Gavin Terrill's Blog

Phusion Passenger RewindableInput Error Parsing with REXML

Ran into this funny little bug and didn't find too much about it on the interwebs, so for prosperities' sake here are the gory details...

The error is "RuntimeError: PhusionPassenger::Utils::RewindableInput is not a valid input stream. It must walk like either a String, an IO, or a Source."

The offending line of code occures when using REXML to parse an XML document as part of a webhook controller:

doc = REXML::Document.new request.body()

Under the covers, it is this line of code that generates the error:

/opt/ruby-enterprise-1.8.7-2011.03/lib/ruby/1.8/rexml/source.rb:21:in `create_from'

This is a rails 3 app, running under Phusion Passenger 3.0.10 and REE 1.8.7.

The solution, courtesy of jsierles, is to fixup Passenger with an ImprovedRewindableInput: https://gist.github.com/281126 Basically, create a /config/initializers/phusion-rexml-phixup.rb file and paste the code from that gist.

 

 

Remembering Carl Adam Petri

Just found out that Carl Adam Petri passed away July 2, 2010. Petri invented the "Petri Net" - a tool for modeling process flow that is used in a wide variety of applications, from circuit design to traffic light coordination to business process management.

I was introduced to Petri Nets by a grizzled veteran software developer back when I knew everything. I looked into it and was struck by the simplicity of the model, plus the mathematics. You see, Petri Nets can be formally verified so you can prove that your flow is "correct", meaning it will complete and doesn't get stuck in an endless loop for example. You can even do simulations with it to help predict capacity requirements.

The basic idea is you have a diagram with boxes that represent "doing something", called a Transition. Transitions are connected to other transitions via a circle, called a Place. The connecting lines show the direction of the flow and are called Arcs. A network has start and end places and can not have cycles (i.e a Petri Net is a DAG).

Now, to model the process flow you place a Token in the starting place. The token represents a unit of work and moves through the network of transitions and places until it gets to the end place. As each transition fires the the token moves to one of the output places.

 

What is interesting is that a place can hold more than one token, so you can add another token to the starting place before the first one has reached the end place. This is why a Petri Net is good for modeling concurrent processes.

Wil Van Der Aalst has written some great papers on applying Petri Net theory to business process management. A few years ago I was inspired by one of his great books and created a Swing based Petri Net editor. The code has long since been lost unfortunately, but I remember feeling quite chuffed at being able to simulate and optimize an insurance claims process by playing with the capacity of certain transitions in order to improve overall system throughput. It was very Kanban like.

Petri was a true pioneer of our industry, and the amazing thing to me is that he came up with the idea when he was just 13! 

37 Signals Interviews Shopify

There is a nice Q & A with Shopify's CEO Tobias Lütke over at 37 Signals talking about the history, culture, product and philosophy of the company.

It's a great, motivating success story, and I'm proud to have played a small part of that by making the introduction to John Phillips after I had experienced the product and was so impressed with the company and technology choices:

We ended up taking an investment from an angel investor, who just called us up one day because he heard about our company. This is certainly an outlier, but taking this investment from John H. Phillips was probably the best decision we ever made. Apart from allowing us to meet payroll for the first year, he has spent countless hours teaching me all the things I needed to know about running a company. Without him, we simply could not have survived the years and would not be around today.

There are some interesting comments on Hacker News, including Tobi talking about a momentous occasion in their lifetime when the new business model was put in place (which I blogged about at the time):

The problem was that our old system was setup so that Shopify was very cheap for people who weren't successful and became quickly expensive for people who actually sold a lot of product. Everyone who expected to setup a 1M revenue store ( e.g. everyone ) picked up a calculator and realized that.

I really admire how Shopify stuck to their guns on the new pricing structure despite huge pushback from the community at the time. Now with the benefit of hindsight, it was clearly a very smart move.

Target non-mobile Devices using JS

Some regex to use when you don't want something to happen on a mobile device:

if (!/android|iphone|ipod|series60|symbian|windows ce|blackberry/i.test(navigator.userAgent)) {
 // regular browser stuff here...
}

I'm All Over It Too

A little slip by one of the bigger banks down here, Westpac, with a wayward tweet (or maybe it was on purpose?!). It's been removed now, but not before I managed to get a screen shot:

Hopefully whoever screwed it up isn't reprimanded too strongly. I think it is nice to see the humans behind the curtain come out every now and again.

 

10 Critical Thinking Traps & Tips

There is a great list of critical thinking traps & tips over at litemind.com. I'm summarizing them here so I remember them!

1. The Anchoring Trap: Over-Relying on First Thoughts.

initial impressions, ideas, estimates or data “anchor” subsequent thoughts

• Always view a problem from different perspectives
• Think on your own before consulting others
• Seek information from a wide variety of sources

2. The Status Quo Trap: Keeping on Keeping On

 status quo automatically has an advantage over every other alternative

• Consider the status quo as just another alternative.
• Know your objectives.
• Avoid exaggerating switching costs.

3. The Sunk Cost Trap: Protecting Earlier Choices

it's a sunk cost, and it shouldn't influence your decision

• Be OK with making mistakes.
• Listen to people who were not involved in the earlier decisions.
• Focus on your goals.

4. The Confirmation Trap: Seeing What You Want to See

looking for information that will most likely support your initial point of view

• Expose yourself to conflicting information.
• Get a devil's advocate.
• Don't ask leading questions.

5. The Incomplete Information Trap: Review Your Assumptions

(my favorite, but hard to execute!)

overlooking a simple data element can make our intuitions go completely astray

• Make your assumptions explicit.
• Always favor hard data over mental simplifications.

6. The Conformity Trap: Everybody Else Is Doing It

other people's actions do heavily influence ours

• Discount the influence of others.
• Beware "social proof".
• Be courageous.

7. The Illusion of Control Trap: Shooting in the Dark

we still tend to irrationally believe that we can somehow influence results. We just love to feel in control

• Understand that randomness is part and parcel of life.
• Beware of superstitions.

8. The Coincidence Trap: We Suck at Probabilities

Probabilities can be unintuitive.

• Don't over-rely on gut estimates.
• Beware of "after the fact" probabilities.

9. The Recall Trap: Not All Memories Are Created Equal

we analyze information based on experience, on what we can remember from it

• Get hard data. (there's that hard data thing again - see # 5)
• Be aware of your emotions.

10. The Superiority Trap: The Average is Above Average

people have much inflated views of themselves

• Be humble.
• Don't go overboard. (wrt applying rigor and rational thinking)

I encourage you to read the full article as it contains several good examples and great explanations of the tips.

Giving the Benefit of the Doubt

There was a lively thread on proggit the other day about how difficult programmers can be to work with:

• "always trying to one up each other and prove how smart they are"
• "petty infighting and ruthless backstabbing"
• "constantly seek refuge in argument by authority, blindly quoting their favorite gurus to support their positions"
• "desperate need to believe in absolutes"

I have to admit, I am certainly guilty of pretty much all of the traits listed in the post. Some more than others, much to my chagrin.

It occurred to me that as I have gained more experience (read: got older) these tendencies have tended to diminish. I have realized the more aware I have become with the business side of the equation, ideals I was once passionate about now seem rather insignificant in the grand scheme of things (and of course with the benefit of 20/20 hindsight).

My interaction style has also changed a lot. Whereas before I would quickly jump to conclusions when I saw things that were "wrong", I now try to come up with possible explanations for why it is the way it is before flying off the handle. What I have found is that I often learn something new - if not with the code or process then certainly about the way the individual works.

So today I came across a post in The Fishbowl about a thing called the "Principle of Charity". It's a good read. Fundamentally it says be charitable to whom it is you are dealing with and assume the best of them:

1. They are not idiots, brain dead or morons.
2. They are honest and not trying to scam, manipulate or deceive you.
3. They did their homework.

I like it! It is nice to find out about things you have a sense of but can't quite put a finger on. Getting back to that proggit thread, I wish someone had told me about this 20 years ago. I dare say my technical discussions would of been much more efficient and less aggravating had I afforded my poor colleagues the benefit of the doubt occasionally.

Charles says his challenge is to remember the principles. I think my challenge (now that I am enlightened) is how to respond to individuals who violate those principles when dealing with me. Let me clarify that. I mean to say how to respond "politely".

jQuery Cycle Plugin - Adding Image Links

The jQuery cycle plugin is pretty freakin' awesome, but I couldn't figure out how to put a link behind each image. Sure there is a tutorial, but the images refused to show up no matter how much CSS tweaking I applied.

So I came up with a quick and dirty solution that uses jQuery to handle the click event for each image in the slideshow.

First the markup for the slideshow images:

<div id="slideshow">
  <img src="'banner1.jpg" alt="" rel="/" />
  <img src="'banner2.jpg" alt="" rel="/collections/skincare" />
  <img src="'banner3.jpg" alt="" rel="/collections/men" />
  <img src="'banner4.jpg" alt="" rel="/collections/men" />
  <img src="'banner5.jpg" alt="" rel="/products/the-ultimate-guide-to-beauty-by-anjana-gosai" />
</div>

It is basically the same as what you normally use, but each image has an additional "rel" attribute with the location to link to.

Now the javascript:

<script type="text/javascript">
  jQuery.noConflict();
  jQuery(document).ready(function() {
    jQuery('#slideshow').cycle({fx: 'fade', timeout:5000, speed:1000});
    jQuery('#slideshow img').click(function (){
      document.location.href = jQuery(this).attr('rel');
    }).css('cursor', 'pointer');
  });
</script>


I am adding a click handler to each <img> element in the slideshow div, and using whatever is specified in the rel attribute as the location to go to. Finally, I make the cursor a pointer for each image so the user knows it is click-able.

You can see it in action here.

I'll see your Groovy and raise you a Python (and Ruby, PHP for good measure)

Inspired by what is clearly an interesting trend in want ads mentioning Groovy, I ran the same query against Python, Ruby & PHP. Groovy is yet to be where Ruby was five years ago but has at least managed to break out from the x axis. Python and PHP keep on gaining traction.

ruby, python, php, groovy Job Trends

ruby jobs - python jobs - php jobs - groovy jobs

Handling Errors Using jQuery.load

Following on from my last post on integrating HTML content from another  system into an existing site, I realized I wasn't handling failures very gracefully. To recap, I was using the load method of jQuery:

$(document).ready(function() {
 $('#races a').click(function () {
  $('#results').load('Racing/' + $(this).attr('href') + ' #wrap');
  return false;
 });
});

I was surprised to find there wasn't much information about how to go about capturing an error if it occurs. It turns out you can get it by passing a callback function to the load function. This function accepts 3 parameters: responseText, textStatus and an XMLHttpRequest. I'm not exactly sure why they decided to give you responseText and textStatus when you have an XMLHttpRequest (an object that already encapsulates both of these values so passing them as parameters is a violation of the DRY principle).

Anyways, here is how to do it using the callback function:

$(document).ready(function() {
 $('#races a').click(function () {
  $('#results').load('Racing/' + $(this).attr('href') + ' #wrap', "", function(responseText, textStatus, XMLHttpRequest) {
   switch (XMLHttpRequest.status) {
    case 200: break;
    case 404:
     $('#results').html('<p>Looks like the results have not been uploaded to the server yet. Please check back later.</p>');
     break;
    default:
     $('#results').html('<p>' + XMLHttpRequest.status + ': ' + XMLHttpRequest.statusText + '. Please contact the club and let them know.</p>');
     break;
   }
  });
  return false;
 });
});

Now you can get specific about how you report your error based on the response code. For example, if you see something in the 400 range you know there has been a client problem (e.g. 404 not found), versus a 500 series status for server side problems. In this case, I explicitly handle 404 errors and have a generic catch-all for anything else.