Gavin Terrill's Blog

G'day Mate

I see it is already April 1st back home: Search tomorrow’s web, today!

Learning Scala

One of my new year revelations is to learn a new language (I'm investing in my knowledge portfolio) and I've decided to go with Scala. I was also looking at Erlang, but in the end Scala made the most sense to me because:

• It runs on the JVM. This is important to enterprise customers, and if Scala works out I'd like to be able to use it for our existing products.
• I am more comfortable with the language syntax, coming from a Java background.
• I can mix and match parts of our existing Spring based Java application with Scala bits. I'm actually interested in how far into component reuse utilizing Spring Dynamic Modules for OSGi I can get - swapping out Java backed components with ones written in Scala.
• I am interested in exploring Scala Closure and Actor idioms for revalidate (ETag helper for Spring & Hibernate applications), inspired by this post on using Scala for HTTP Precondition processing in Jersey.
• I'd like to explore the benefits in terms of code quality using Scala that Dave Pollack reports in building lift using Scala.

Ted Neward's introductory article on developerworks "The busy Java developer's guide to Scala: Functional programming for the object oriented" is easy to digest, and timely. After reading that, I followed the tutorials on the Scala site.

So far, the learning curve has been gentle. Type inference is cool. I find myself thinking about how I can apply the Scala way of doing things to problems areas in our product. I'm currently reading about "mixins", which allow you to treat classes as if they implemented an interface, even when they don't.

The next step is to install the Scala Eclipse plugin and start getting into more meaty examples (Updated a few hours later: see here for instructions on getting around the "org.eclipse.pde.runtime is missing" problem).

A real RESTful API for SimpleDB

Amazon is catching a lot of flak over their implementation of a RESTful API for SimpleDB.  Stefan calls it "the shittiest REST API in a long time". Bill says that "Ascribing REST to the SimpleDb API might be down to lack of knowledge or marketecture". 

So what is the fuss about? Take a look at the following sample code from the Amazon SimpleDB developer guide:

https://sdb.amazonaws.com/?Action=PutAttributes
    &DomainName=MyDomain
    &ItemName=Item123
    &Attribute.1.Name=Color&Attribute.1.Value=Blue
    &Attribute.2.Name=Size&Attribute.2.Value=Med
    &Attribute.3.Name=Price&Attribute.3.Value=14.99
    &AWSAccessKeyId=
    &Version=2007-11-07
    &Signature=Dqlp3Sd6ljTUA9Uf6SGtEExwUQE=
    &SignatureVersion=1
    &Timestamp=2007-06-25T15%3A01%3A28-07%3A00

Can you see what the problem is? It's kind of subtle, so I'll give you a hint - what is  up with that "Action=PutAttributes" bit in the URL?

The problem is that in REST you use the HTTP methods GET, PUT, POST, DELETE, HEAD and OPTIONS. In the above example, GET is being used. Now, the interesting thing about the GET method is that according to the HTTP spec (RFC 2616), it is "safe":

GET and HEAD methods SHOULD NOT have the significance of taking an action other than retrieval

In other words, the GET method is what you use for reading things (e.g. queries or looking up items). GET should not be used to update things. Lots of web infrastructure rely on these semantics to do their job. Breaking the contract hurts the web.

So how would you make it RESTful? First, I'd recommend taking a look at Joe Gregorio's excellent advice from 2004 on designing a REST API. Joe suggests asking four questions:

1. What are the URIs?
2. What's the format?
3. What methods are supported?
4. What status codes should be returned?

So lets map out a strawman SimpleDB REST API using these questions as a guideline.

The URI

In this case, we are updating something in MyDomain which is identified by ItemName.

Resource	URI
MyDomain List	/mydomains/
MyDomain	/mydomains/{item-name}

 

The Format

These days, JSON is a pretty decent format to use as a generic starting place. XML is still fine & dandy, but is a little harder to work with in browser clients. You might want to take a look at Atom as well. In any case, clients can use Content Negotiation to ask for their preferred representation.

A JSON format for a SimpleDB item could look like this:

  {
      "DomainName": "MyDomain",
      "ItemName": "Item123",
      "Timestamp": "2007-06-25T15%3A01%3A28-07%3A00",
      "Attributes": {
		"Color": "Blue",
		"Size": "Med",
		"Price": "14.99”
	}
  }

The list of items would look like this:

  [
    {"ItemName": "Item123", "href": "https://sdb.amazonaws.com/mydomains/123"},
    {"ItemName": "Item321", "href": "https://sdb.amazonaws.com/mydomains/321”},
    ...
  ]

Notice that in the list we have indicated the href of each specific item.

The Methods

Mapping the available operations to the MyDomain items/item we get the following:

Resource	URI	Method	Description
MyDomain List	/mydomains/	GET	Retrieve the list of MyDomain items
		POST	Create a new MyDomain item
MyDomain	/mydomains/{item-name}	GET	Retrieve a specific MyDomain iem
		PUT	Update a MyDomain item
		DELETE	Remove a MyDomain item

 

The Status Codes

Joe recommends that you document the status codes that you expect to return for each operation. Some of the more common ones are:

• 200 - OK.
• 201 - Created. When an item is successfully created return a 201 and the response will include a Location: header with the URI of the newly created item resource.
• 204 - No Content. Used for a successful DELETE.
• 301 - Moved Permanently.
• 400 - Bad Request.
• 410 - Gone.

Note that in HTTP there is the idea of status code fallback. The 2xx series means success, 3xx means a redirection and the 4xx series means a client error. If a client doesn't understand 201 for example, it can still safely assume that the operation was a success.

So there you have it. My version of a RESTful API for SimpleDB. SimpleDB seems like a very cool idea to me. Let's hope they sort this mess out soon.

Cool Image Resizing Technique

"Seam Carving" is a technique for resizing images based on "seams". Seams are runs of similar pixels, and are identified by using gradient magnitude. When you cut out less significant pixels in a seam, you end up being able to resize an image in a non-uniform manner, but retain important features. You can even use it to remove items, like say an ex from that last vacation...

It is a lot easier to see it in action. Take a look:

Larry O'Brien talks about the algorithms behind this here. I had never heard of A* before this. Interesting stuff.

(via hackzine.com)

BFGs on a plane

It was Socrates who said "It's the journey, not the destination." I believe he may have been referring to Virgin America, who apparently have a mp3 playlist you can set up before the flight, chat and Doom (amongst other things including internet access at some point) on their inflight entertainment system. Oh, I am so flying with these guys. Hope they come to Toronto.

No backup, no problem - thanks to the wayback machine

I did the original web site for Karora Technologies just after we got the company incorporated. It was nothing fancy, and I mostly used Microsoft Publisher because it came bundled with the first laptop we purchased. Later on, we hired a graphic design firm to come up with a clean, simple design that incorporated Aboriginal imagery. The designers delivered and we ended up with an awesome website in my opinion.

Later on as the company got bigger, management of the website was handed over to the marketing/sales area who had someone on the team who had "done websites in the past" using FrontPage. The decision was made to "revamp" the site, and along the way we lost the simplicity and elegant of the "Karora Dreaming" site. In fairness, the requirement was to be able to easily update the site, and this was difficult at the time because it was using bare HTML.

Anyway, five years down the track and the new site badly needs to be updated. Karora has since downsized, and no longer has FrontPage people on staff, so I offered to help out in my copious amounts of spare time (yeah, right!).

The first thing I wanted to do was go back to the look and feel the design firm had come up with so I asked for the backups. Backups? You guessed it - there were no backups. Sure, they were done at the time the site was swapped over, but the machine they were on has since disappeared into the ether. The design firm didn't have anything either.

Then I remembered reading about the "Wayback Machine". This is an initiative of the Internet Archive organization. Basically, they take snapshots of the web so you can see what a site looked like at a particular point in time. I wondered if the Karora site was on it, and sure enough there were entries going way back to my initial dodgy version from 1999.

So, I was able to recover our old website design (with a little help from HTTrack). Thanks WayBack Machine!

Boggles the mind

These guys are messing with the theory of relativity: Scientists break speed of light. I guess that's why it is still a theory, and not a law.

Practical applications? Two words: Fibre Optic.

File that under "how would you measure it?".

Serial Killer or Killer Hacker?

Came across this fun site: Programming Language Inventor or Serial Killer?

My score: 7/10.

Squielimojo

(via Stefav Tilkov's Random Stuff)

Pandora's Box

At same stage, I got frustrated with the inane chatter of mainstream commercial radio and now listen to CBC and Jazz.fm which offer more content and less ads. The problem is that I am not being exposed to that much new material (outside of jazz), so lately I've been tuning into Pandora - the Music Genome Project's music discovery project (hat tip to mocca).

Pretty cool. Basically, you tell it the name of an artist or song you like and it uses its database containing 400 attributes of around 300,000 songs from 10,000 artists to figure out other material you might like.

Pandora bills itself as "the most comprehensive analysis of music ever undertaken", so I was a bit disappointed to find the underlying science quite elusive. I imagine they are using something like the "least squares" method for finding songs with similar attributes, but how would that work for an artist whose material may vary substantially over time (e.g. Madonna)? Maybe that has something to do with some users suggesting that seeding your channels with songs instead of artists improves results. Personally, I've found both song and artist founded channels have led me to discover great new artists that I would never have known about before.