ThingDB - you haven't slayed SQL just yet

I came across ThingDB the other day (via Reg, who pretends he isn't blogging, but one wonders how long until the siren call beckons?!). It is part of the Pharos project from the reddit guys. ThingDB is basically an abstraction layer that sits on an entity table that has a one-to-many relationship to a properties table, with versioning thrown in for good measure. Anyone who has built a system requiring extensible user defined attributes or configurable metadata (e.g. a document management system) would instantly recognize this. It partitions data (i.e. shards) to make it scale beyond a single machine, and they talk of using bloom filters, so it seems pretty cool.

However... what got me to post about this was not so much the technology, but the statements made about SQL. Consider:

But SQL has some problems. For one thing, it's inflexible and poorly-integrated into most programming languages.

Reality check guys: SQL is one of the most successful technical things we have done in our field. I credit that to its flexibility. In fact, I can't think of a situation in my 25 year programming career where I have found SQL to be so inflexible that I need to use something else. Maybe it is because I've predominantly done mainstream CRUD-style enterprise apps, but guess what - that is what the majority of the work is.

I don't get the claim that ThingDB is superior from a language integration point of view. Take a look at the top 9 programming languages (constituting 80% of the space) as at November '08 (according to TIOBE):

1. Java (20.30%)
2. C  (15.28%)
3. C++  (10.36%)
4. (Visual) Basic (9.27%)
5. PHP (8.94%)
6. Python (5.14%)
7. C# (4.03%)
8. Delphi (4.01%)
9. Perl (3.88%)

Everyone one of those languages has excellent support and integration with SQL. In some cases, you can even embed SQL directly into the code (the merits of doing this nowadays are a bit dodgy, but I will say we built a very clean, fast, client/server app in the eighties using this approach). I would go so far as to say that LAMP would not have happened were it not for the integration between the M and P in LAMP.

Side note: check out this cool feature in Netbeans.next - SQL code completion in PHP.

Anyways, a final note on ThingDB. The discussion about how it can be scaled up is very interesting, and I don't want to detract from that as it is very leading edge and well... cool. But, and there is always a 'but', for the average corporate application this is going to be massive overkill. Your typical RDBMS is a very capable technology and isn't going to run out of steam anytime soon. For example, we have a deployment of our product that is tracking issues for 50,000 users worldwide. They are happily running it using Sybase on a 4-way box, and with millions of rows performance is still great.

Bottom line: In spite of claims to the contrary, SQL is still a very useful technology that is entirely appropriate for a large class of applications, and will be be for the foresable future.

Comments

[Joar Swenning]

Yes, the majority of the work is about CRUD-style enterprise apps. And no, SQL hasn't been slayed just yet. Seems strange that it needs to be pointed out but, you do it well!