Gavin Terrill's Blog

Wide Finder, Two.Oh!

Tim Bray started a project last year that attempted to find the fastest way to do string wrangling on log files using Erlang. The idea is to see how languages (like Erlang) that work  well on multi-core machines  compare with more traditional languages; in terms of designing a solution and performance. Pretty soon, Really Smart People were contributing solutions, including those written in languages that are not concurrent savvy per se (e.g. Perl). As it turns out, a Perl implementation kicked Erlang's butt, but the project was somewhat flawed as Tim points out in "Wide Finder 2":

There were a few problems last time. First, the disk wasn’t big enough and the sample data was too small (much smaller than the computer’s memory). Second, I could never get big Java programs to run properly on that system, something locked up and went weird. Finally, when I had trouble compiling other people’s code, I eventually ran out of patience and gave up. One consequence is that no C or C++ candidates ever ran successfully.

This time, we have sample data that’s larger than main memory and we have our own computer, and I’ll be willing to give anyone who’s seriously interested their own account to get on and fine-tune their own code.

What I loved about this the first time around was the discussion around how people approached the problem and their refinement of strategies based on some serious analysis of the bottlenecks. This is kind of geeky, but this is the type of stuff I find fascinating. Tim must have spent a fair bit of time lobbying for the corporate resources needed to get round 2 off the ground, so hat's off to you Tim, and well done Sun for having the foresight to back this project.

Simple Spring 2.5 App with 2.5

Wanted to send some link love to Cagatay Civici who has created a really nice, minimal, concise sample application that showcases a bunch of latest and greatest Java webapp technologies:

• JSF (MyFaces 1.2.2 with Facelets)
• Spring 2.5
• JPA with Hibernate
• HSQLDB for persistence
• Jetty for deployment
• Maven2 for build

I like the Generic DAO idea. Get it here: Annotation Driven JSF-Spring-JPA.

JSR 311 not RESTful?

Elliot thinks JSR-311, JAX-RS: The Java API for RESTful Web Services isn't really that RESTful at all, so the name should be changed to something along the lines of "Java HTTP Server API". Initially, I thought no way, but I think Elliot may be on to something.

I've done a little bit of playing around with Jersey, and I've found the annotations to be quite helpful in building a RESTful interface to our domain objects. Here, for example, are the relevant bits of our UsersResource class which sits the behind the /users uri:

@UriTemplate("/users")
public class UsersResource {
    @HttpContext 
    private UriInfo uriInfo;
    
    private FormatterFactory formatterFactory = new DefaultUserFormatterFactory(); // Spring injected
    private RealmManager realmManager = RealmManager.instance(); // Spring injected    

    @HttpMethod("GET")
    @ProduceMime("application/json")
    public JSONObject getUsersAsJsonArray() throws JSONException {
    	JsonFormatter formatter = formatterFactory.getSummaryFormatter();
    	
    	JSONObject o = new JSONObject();
        JSONArray objectArray = new JSONArray();

    	Search search = new Search(createFilter());
    	search.setFilter(filter);
    	realmManager.execute(search);

        for (Object item : search.getResults()) {
            objectArray.put(formatter.format(uriInfo, item));
        }
    	o.put("totalCount", search.getTotal());
    	o.put("users", objectArray);
		
        return o;
    }

    @UriTemplate("{username}")
    public UserResource getUser(@UriParam("username") String username) {
	User user = realmManager.getUser(username);
        if (null == user) {
            throw new WebApplicationException(new RuntimeException("user '" + username + "' does not exist."), 404);
        }
    	
        return new UserResource(formatterFactory, uriInfo, user);
    }

    private UserFilter createFilter() {
        UserFilter filter = new UserFilter();
    
        // Populate a filter using uri parameters
        try {
            MultivaluedMap<String, String> queryParameters = uriInfo.getQueryParameters();
            for (Map.Entry<String, List<String>> entry : queryParameters.entrySet()) {
                String value = entry.getValue().get(0);
                PropertyUtils.setSimpleProperty(filter, entry.getKey(), value);
            }
        } catch (IllegalAccessException e) {
            throw new WebApplicationException(e);
        } catch (InvocationTargetException e) {
            throw new WebApplicationException(e);
        } catch (NoSuchMethodException e) {
            // ignore - this is a parameter that doesn't match a setter on the filter
	}
        return filter;
    }
}

I've bolded that JAX-RS annotations. Is there anything there that is REST specific? Not really. It is mostly uri munging. Then again, you could also say the same of REST when it gets down to the nuts and bolts of implementation.

On the other hand, REST is more than just uri munging - it is an architectural style, and Elliot is right when he says:

An API is probably the wrong place to enforce architectural decisions

.htaccess settings for parsing PHP script in HTML

I have a simple website with a common header and footer hosted on www.netnation.com. Instead of duplicating the common bits on every page and creating a maintenance nightmare, I opted to use php include statements. The problem was I don't particulary want to bother users with the implementation details, so I want my URLs to end with ".html" (not ".php"). After racking my brains out and trying every combination I could find on Google, it turns out the following is the magic incarnation:

AddHandler php-script .html
Action php-script /cgi-bin/php

Put those lines in your .htaccess file (in every directory), and your embedded php script in .html files will be parsed correctly.

If anyone knows why this works and the other stuff doesn't, please feel free to educate me and future visitors by commenting.

Acegi Petclinic JUnit

My last post on how to use Acegi Security while JUnit integration testing drew a fair bit of traffic, so I thought I'd write a small Eclipse project that pulls all the bits together, and updated to run with Spring 2.0 and Acegi 1.0.3.

To build this project I followed the following steps:

1. Got petclinic 2.0.2 going (download Spring here) and followed the instructions in the readme.txt files.
2. Got acegi 1.0.3 (download Acegi here) and followed the instructions on integrating it with petclinic as described here (including the optional bonus section - "securing the middle tier").
3. (optional) Ran the JdbcClinicTests from within Eclipse. This allowed me to step through the code in the debugger.
4. Modified JdbcClinicTests so that it used the Acegi post-interceptor.

To get JdbcClinicTests working with Acegi (Step 4) I made the following changes:

1. Add

   /WEB-INF/applicationContext-acegi-security.xml

to JdbcClinicTests.getConfigLocations.

2. Applied the same changes to /test/../applicationContext-jdbc.xml that were done in step 2 above - i.e. add a "postInterceptor" entry to the clinic bean and create a "methodSecurityInterceptor" bean.
3. Add the acegi libraries to the debug profile, and include the war directory (so that it can find users.properties and the acegi context file). Here is a screen grab of my settings: Download acegi-debug-settings.gif

If you ran the project at this point you would get a AuthenticationCredentialsNotFoundException for every single test. To prevent this, we need to use a TestingAuthenticationProvider instead. I created a class called AcegiTestIntegration that does the replacement on the fly in its' constructor, and has a setAuthenticationToken method that sets up the security context.

For example, here is how it is initialized in the JdbcClinic.setClinic method:

acegiTestIntegration = new AcegiTestIntegration(getApplicationContext());
acegiTestIntegration.setAuthenticationToken(dianne);

The variable dianne is declared as:

private TestingAuthenticationToken dianne = new TestingAuthenticationToken("dianne", "emu", new GrantedAuthority[] {ROLE_USER});

Now if you ran the project, every test would pass except the last one - testInsertVisit - which would get an AccessDeniedException. Success! You have validated that only users with the supervisor role can invoke the method. Now, to get the unit test to pass, we simply catch the exception, and set up a token that should pass:

try {
 this.clinic.storeVisit(visit);
 fail("Should of thrown an AccessDeniedException because dianne is not ROLE_SUPERVISOR");
} catch (AccessDeniedException e) {
 // expected - dianne is ROLE_USER, not ROLE_SUPERVISOR
}
acegiTestIntegration.setAuthenticationToken(marissa); // ROLE_SUPERVISOR
this.clinic.storeVisit(visit);

That's it.

Choose from one of the following downloads:

• Download petclinic-acegi-junit.zip - The full source tree (including jar files) and eclipse project (12,603 KB).
• Download AcegiTestIntegration.java - AcegiTestIntegration source file (put this under petclinic/test/org etc) (2 KB).

Additional notes:

• An alternative approach you may want to consider is having seperate applicationContext-acegi-security.xml files - one for production and one for testing. The testing one would be configured to use a TestingAuthenticationProvider. The down side to this is having to move different configuration files around in the build script and/or keeping changes in sync between the files.
• AcegiTestIntegration could be configured as a bean and injected into the test class. In this case, make it ApplicationContextAware and move the code that is in the constructor to it's setApplicationContext method, then get rid of the constructor.
  

Please leave a comment if you find this useful, or have suggestions for improvements. Thanks.

ETags considered useful

Nice post about how using ETags can save you bandwidth and computation.

Summary: when you send something to a client, give them a token they can use later to see if that something has changed. If nothing has changed, then don't send anything.

I'd love to turn this on for our application somehow, but given the dynamic nature of it I suspect it could get a bit tricky. The stats Joe shows indicate that roughly 1 out 4 requests don't need a reply. I'd be interested to know what the breakdown of "user agents" making the requests were, as not every browser is ETag savvy. My other concern would be that once I got into the messy details, I would discover there are problems using this type of caching mechanism in a clustered environment.

How does that quote from go? - "If it was easy, everyone would be doing it."

How to fix schema info messages in Visual Studio 2005 when using App.config and remoting

Sorry for the title, but I'm hoping to save some time for people who are searching for a solution to this problem in the future.

I'm using Visual Studio 2005 (build 8.0.50727-4200) to create a project that uses .Net remoting. When I try to configure remoting using App.config via RemotingConfiguration.Configure(...), I get annoying messages in the error list window telling me that it "Could not find schema information for the element 'application'".   The problem is because DotNetConfig.xsd in c:\program files\microsoft visual studio 8\xml\schemas doesn't have all the bits required to understand remoting.

Here is how to fix the problem:

1. Open App.config
2. Select the XML | Create Schema menu option
3. Copy everything inside the <xs:element name="system.runtime.remoting"> tag to the clipboard
4. Open C:\Program Files\Microsoft Visual Studio 8\Xml\Schemas\DotNetConfig.xsd
5. Search for "remoting"
6. Change the line that says

   <xs:element name="system.runtime.remoting" vs:help="configuration/system.runtime.remoting"/>

   to

   <xs:element name="system.runtime.remoting" vs:help="configuration/system.runtime.remoting">
   </xs:element>

7. Paste the clipboard contents inside the remoting element

This will only pick up the elements and attributes you are currently using in App.Config. If you change the file you will have to go through this rigmorale again, or edit DotNetConfig.xsd by hand.

I imagine this will get fixed eventually, but in the mean time I hope this helps.

How to slow down your application - Part 1

I promise this will deliver at least a 200% slow down to your Hibernate app. Just add the following to your log4j file:

log4j.logger.net.sf.hibernate=DEBUG

My oh my, my log file overfloweth...

JUnit testing with Acegi Security

Here's a tip if you are trying to do some unit testing with Acegi Security - particularly if you are doing role based authorization of method calls on your manager objects via interception.

Basically, a secure method interceptor will a) need an authentication token to play with, and b) a way to  find out what authorities the user has.  We need to cater for this when running the tests.

Acegi Security conveniently provides a TestingAuthenticationToken for unit testing purposes so we are Ok with the token part. You will now need to provide a suitable authentication provider. Somewhat unsurprisingly, there is a TestingAuthenticationProvider that you can add to your Spring config file to accommodate this.

But here is the problem: we already have this configured for production using a different provider (obviously), so we would have to resort to changing the config files depending on if you are running tests or not. This is too fragile for me, so here is the essence of the tip: leave the configuration file alone and dynamically changes the provider in the setup of the unit test.

Here is some code that I have in my JUnit setUp() method:

        // Grant all roles to noddy.
        TestingAuthenticationToken token = new TestingAuthenticationToken(
                "noddy", "test", new GrantedAuthority[] {
                        new GrantedAuthorityImpl("User"),
                        new GrantedAuthorityImpl("Administrator") });
        
        // Override the regular spring configuration 
        ProviderManager providerManager = (ProviderManager) ctx.getBean("authenticationManager");
        List list = new ArrayList();
        list.add(new TestingAuthenticationProvider());
        providerManager.setProviders(list);
        
        // Create and store the Acegi SecureContext into the ContextHolder. 
        SecureContextImpl secureContext = new SecureContextImpl();
        secureContext.setAuthentication(token);
        ContextHolder.setContext(secureContext);

So... it sets up an authentication token, then resets the authentication managers' provider list with one that contains the TestingAuthenticationProvider. It then creates the context for the method execution.

The problem with this is it does it for every single test method - that is how JUnit works for better or worse.

Hope it proves useful for you.

Update 2007-01-31: I've updated this to work with Acegi 1.0.3 and Spring 2.0.2.

Eclipse 3.1

Wow - I just checked out the "New and Noteworthy" page for Eclipse 3.1 Milestone 4.

Looking good:

• Ant debugger
• Java 5 Annotation support
• Grouping multiproject projects
• Spell checker for property files (wonder how this works for I18n)
• Rerun failed JUnit tests first

M4 was released right on schedule, so I'm hoping to see M5 released on the 18th. (Salivating...) "Late June" can't come quick enough.