Technical Notes

Getting started with Mechanize

I think this will start a short series on black-box webapp testing.  This is just the first in a series and we’ll add quite a bit of content to it in the next week or so.

A weeks back I helped a development team set up a testing environment for their Ruby on Rails webapp.  The webapp is about 18 months old and had exactly zero tests.  Reverifying its intended behavior was a full time job for some of the developers on the team because there was no way to prove any behavior worked as intended at any given time.  Worse, behavior verification was a manual process and obviously error-prone.

Repeatable tests that prove the software is correct are of high value to this customer.  Testing their application had to be easy.  Writing tests and running them had to be easy.  For an existing webapp with little to no test coverage a black-box testing approach provides a  lot of return for the effort.  Black-box testing allows behavior testing.  We test  webapp behavior by making GET and POST requests to the known URLs the webapp exposes.

Before we set up a test environment for the existing webapp we must decide how to exercise its behavior.  There are a plenty of HTTP libraries available for different languages and platforms.  When you don’t have any tests any testing is better than nothing.  Don’t be afraid if your initial testing framework doesn’t match your webapp language and platform.  We can just as easily test a JSF webapp hosted on Windows with a Ruby client on Linux.  In fact, diversity in the test environment is a good thing.  Multiple black-box clients show how a webapp will perform in the wild.

Automated webapp testing with the Ruby Mechanize library is very easy.  The Ruby environment with RubyGems is required to use the Mechanize library.  There are instructions for setting up a Ruby environment on the Ruby on Rails getting started page.  Follow the instructions for Ruby and RubyGems.

Installing the Mechanize gem is easy.  On Mac OS X 10.5 and Linux the command is:
sudo gem install mechanize

Once you install Mechanize you probably want to play around with it.  You can do so with irb, the interactive Ruby interpreter.  Start irb on the command line by typing irb.  We write Ruby programs directly in irb and irb executes each line as we enter it.  Think of it like a bash shell for Ruby.

Initially irb loads only the core Ruby libraries.  We need to load Mechanize manually with the ‘require’ keyword.

>> require 'mechanize'

I like API documentation in front of me when I code.  The documentation for the version of Mechanize you’re using is on your local box if you installed it with the gem install command as it appeared a few paragraphs up.  Start the gem documentation server by typing ‘gem server’ on the command line.  Now visit http://localhost:8808 in your web browser.  Click the rdoc link in the Mechanize section for the API doc.

The fun stuff happens in the WWW::Mechanize class.  Let’s go back to irb and create an instance of this class:

>> agent = WWW::Mechanize.new

Now we have an instance of an object that makes HTTP GET and POST requests.  Start up your webapp and we can make a request.  Starting with a GET request we call the get method on the Mechanize object.  The get method returns an instance of WWW::Mechanize::Page.

>> page.header.class
=> WWW::Mechanize::Headers
>> page.header
=> {"last-modified"=>"Tue, 16 Jun 2009 02:41:30 GMT", "connection"=>"close", "date"=>"Fri, 17 Jul 2009 00:42:43 GMT", "content-type"=>"text/html", "content-length"=>"7466"}
>> page.body.class
=> String
>> page.body

Go ahead and enter page.body.  I omit the output here because it is a little long.  What you should see is the HTML of the web page you requested with the agent.get(…) method.  agent.get(…) assembles the HTTP headers and makes an HTTP GET request.  This method crosses the network if your webapp runs on a different box.  You can test remote webapps just as easily as local.

Play around with the API.  Next time we’ll cover HTTP POST requests.