Gavin Terrill's Blog

Lindaspaces eBook Now Available

I believe the hardcopy is now out of print, so this is pretty sweet - one of the seminal books on the ideas behind Tuplespaces by Nicholas Carriero and David Gelernter is now available online: "How to write parallel programs - a first course". The last edition was written in 1992, but this is still very relevant today.

Reference Architecture circa 2008

Archetype: High availability, scalable enterprise application supporting '000s of users; running on multi-core b0xen.

• Ajaxy ExtJS front-end.
• Client talks REST.
• Request handling via Lift REST - dispatch to business logic via Scala Event Based Actors.
• Lift JPA integration for persistence.
• Comet for push.

Does documenting your architecture make it high quality?

I just posted a piece to InfoQ on the SEI's new 87 page "Evaluating and improving architectural competence" paper. Towards the end of the report there is a section that discusses principles embodied in the models, expressing that goals should be clearly articulated. One of the examples they used made me chuckle:

Documenting the architecture is likely to lead to high-quality architectures because documentation is essential to effective communication, which is essential to effective understanding and use by the architecture’s stakeholders, which is in turn essential to providing timely and useful feedback.

Don't get me wrong - I love most every aspect of my job, but "documenting the architecture" is not one of my favorite activities by a long shot. Its right up there with filling in timesheets and doing SR&ED claims.

I think part of the problem is figuring out what to document. Most of the stuff I've read suggests spending weeks working through the various architectural views. Obvious guy (aka Heraclitus) says that since change is the only constant, you are taking on a significant maintenance burden in synchronizing the document with the evolving system. Meanwhile, the developers and other architects have filed your treatise in the tl;dr folder.

I agree that "effective communication" is "essential to effective understanding and use by the architecture’s stakeholders", but I'll take providing working code anyday.

BAUF - Big Architecture Up Front

Bob Warfield has just written how the lead architect of Twitter has bolted, leaving behind speculation that there are deep scalability issues that need to be overcome before Twitter can achieve world domination.

I don't think you can take shortcuts when it comes to doing due diligence on your architecture choices before building a solution. I'm all for the YAGNI mindset and avoiding BDUF when building out feature, but there are some things that are simply not possible to change after the architectural foundations have been set without incurring significant renovation cost.

To make matters worse, it is extraordinarily difficult to get this right as you typically have very little information at your disposal when you are making these key decisions. Other factors such as the talent pool available to work on the project further muddy the waters. You end up making trade-offs based on educated guesses, and relying heavily on your intuition.

Some of the key decisions include:

• Framework choice - you get time to market, but the trade off is losing control of architectural decisions. In a Rails app, your CRUD operations are synchronous, so if one day you read about the CAP theorem and think to yourself this eventual consistency thing might be the way to go you are going to have a tough time.  Language choice implicitly follows your framework choice, and it is almost impossible to change that later. Imagine trying to get an MS Access based product to work in a clustered environment for example.
• Caching & Security. Two things that are expensive and extremely difficult to retrofit.
• Client platform. Will you need rich client or offline functionality? How will it communicate with your web based service? If you build it all loosely coupled and cohesive you may have a shot, but these qualities rarely fall out of a system built using an ad-hoc approach.
• Distributed API. In an SOA world, changing the protocol of an established application is pretty much out of the question. So, you are forced to think about future requirements ahead of time. For example, you can pretty much guarantee you have to deal with clients on different versions of the API. Tip: having an extensible data format helps, as well as  making sure your application protocol utilizes caching extensively.

I've also covered the "Premature Scalaculation" idea on InfoQ - some good comments there.

Introductory REST Article

Stefan Tilkov has written an excellent article over at InfoQ covering the ideas behind RESTful architectures: "A Brief Introduction to REST".

The thing I really liked about this article was the low-barrier-to-entry approach Stefan took. For example, instead of starting out with why things are called "resources", he simply says "give every thing an id". I can see this resonating with developers new to REST because words like resource are quite overloaded. Another thing that takes some getting used to is the fact you can also model processes as a resource:

Taking the idea of identifying everything that is worth being identified further leads to the creation of resources that you usually don’t see in a typical application design: A process or process step, a sale, a negotiation, a request for a quote — these are all examples of “things” that merit identification.

Stefan also covers several other fundamental REST ideas:

• Using links to traverse the web of relationships between things.
• The fact that there is a standard set of operations (GET, PUT, POST, DELETE, HEAD, OPTIONS), each with formal semantics, that are sufficient to express your (web) services and can be used by ubiquitous clients such as browsers or curl.
• Using content negotiation to ask for different representations of things (e.g. JSON v's XML).
• Communicating statelessly so that the system is more robust and easier to scale.

There is also a nice summary of the theory behind REST. I started getting into REST a few years back when Paul Prescod was educating the great unwashed on xml-dev. This is the best introductory article on REST I've seen so far, so I highly recommend taking a look if you are at all interested in this emerging architectural style.

InfoQ Architecture Team

Over the last few weeks I've been contributing news items to InfoQ:

• Erlang's Mnesia - a distributed DBMS for highly scalable applications
• Are Mashups EAI 2.0?
• Incremental feature search the next UI paradigm shift?
• Using Amazon Web Services to Implement a Video File Conversion app

(BTW, you can subscribe to new items here)

I'm happy to say that as of Friday I am now officially part of the InfoQ Architecture team. The Architecture team is focussed on forward looking, strategic, and higher level topics related to architecture and design. My motivation for joining the team is around becoming more engaged with the architecture community. I find that I really enjoy doing the research for the news items, but they sure do take up a lot of my spare time!

If there is anything you'd like me to cover or take a look at, please feel free to drop me a note or leave a comment.

InfoQ OSGi Interview

I was interviewed by InfoQ a couple of weeks ago on why we are choosing OSGi for our next generation architecture.

"I believe that in a couple of years time it (OSGi) will also be seen as the de facto standard when architecting IT friendly, service oriented enterprise-scale Java apps"

Read the article: OSGi for application modularity - one company's design choice.

A Rose by any other name

Is it just me, or does the underlying technology behind "The Most Revolutionary Microsoft Technology You've Never Heard Of" strike you as  being awfully similar to Jini/Javaspaces too? Funnily enough, Sun went down the path of pitching their distributed programming toolkit as a hardware solution initially as well. Hey, so what ever happened to UPnP DPWS anyway?

Ajax type-ahead done REST style

It was Archimedes when referring to levers who said "Give me the place to stand, and I shall move the earth".  Joe Gregorio 's piece - REST Tips:  URI space is infinite - is well worth spending 10 minutes on, even if you don't speak Python. I can't promise it will be an earth moving experience for you, but it is definitely a good lever and fulcrum (in terms of knowledge) to have at your disposal.

Where is the REST toolkit?

I've been musing on what a REST toolkit/framework would look like for some time. I believe that adoption by the unwashed masses will only happen when a suitable toolkit arrives on their platform of choice. Of course, the very notion of a REST toolkit/framework is a bit specious to the REST zealots, so today I was not surprised to see the newly introduced JSR-311, Java API for RESTful Web Services  promptly lambasted by the ever engaging ERH.

One thing I do agree with from the JSR is that the facilities available for easily building RESTful services in Java are too low level. I like Bill de hÓra's ideas in REST wins, noone goes home:

When REST becomes a mainstream/default development approach, as opposed to a fringe/implicit one, a couple of things come to mind on the plus ca change front. First off, I think you’ll see better support for URI design and mapping onto code in server libraries. Also, better access to headers (especially caching and timestamping directives). More awareness and support for the full range of HTTP methods and response codes. Attribute modifers and declarations on methods that have side-effects. Exception designs based around 5xx and 4xx response codes. Media type dispatching. Less emphasis on cookies, more support for digest authentication. More flexbility in LAMP stacks for writing out stuff that isn’t HTML (esp. JSON and Atom serializers). That sort of thing. Overall a gradual lifting on HTTP/REST concepts and idioms into application code.

I'll be keeping a close eye on this JSR to see who gets involved. Hopefully this will not become the white elephant that EJB became.