Running FitNesse tests from the command line with PyFIT

After a few back-and-forth posts on the FitNesse mailing list and a lot of assistance from John Roth, the developer and maintainer of PyFIT, I managed to run FitNesse tests from the command line so that they can be integrated in buildbot.

First off, you need fairly recent versions of both PyFIT and FitNesse. I used PyFIT 0.8a1, available in zip format from the CheeseShop. To install it, run the customary "python setup.py install" command, which will create a directory called fit under the site-packages directory of your Python installation. Note that PyFIT 0.8a1 also includes a Python port of Rick Mugridge's FIT library.

As for FitNesse, you need at least release 20050405. I used the latest release at this time, 20050731.

One word of advice: don't name the directory which contains your FitNesse fixtures fitnesse; I did that and I spent a lot of time trying to understand why PyFIT suddenly can't find my fixtures. The reason was that there is a fitnesse directory under site-packages/fit which takes precedence in the sys.path list maintained by PyFIT. Since that directory obviously doesn't contain my fixtures, they were never found by PyFIT. The fitnesse directory didn't use to be there in PyFIT 0.6a1, so keep this caveat in mind if you're upgrading from 0.6a1 to a later version of PyFIT.

Since PyFIT doesn't have online documentation yet, I copied the documentation included in the zip file to one of my Web servers, so you can check out this FIT Overview page.

Let's assume that your FitNesse server is running on localhost on port 8080, and that you have an existing test suite set up as a FitNesse Wiki page (of type Suite) which contains 4 sub-pages of type Test. Let's say your test suite is called YourAcceptanceTests, and it resides directly under the FitNesseRoot directory of your FitNesse distribution, which means that its URL would be http://your.website.org:8080/YourAcceptanceTests. Let's say your test pages are called YourTestPageN with N from 1 to 4.

If you want to run your FitNesse tests at the command line, you need to use the TestRunner. The command line to use looks something like this:

python /usr/local/bin/TestRunner.py -o /tmp/fitoutput localhost 8080 YourAcceptanceTests

in connect. host: 'localhost' port: '8080'
http request sent
validating connection...
...ok
Classpath received from server: /proj/myproj/tests:fitnesse.jar:fitlibrary.jar
processing document of size: 4681
new document: 'YourTestPage1'
processing document of size: 11698
new document: 'YourTestPage2'
processing document of size: 6004
new document: 'YourTestPage3'
processing document of size: 3924
new document: 'YourTestPage4'
completion signal received
Test Pages: 4 right, 0 wrong, 0 ignored, 0 exceptions
Assertions: 239 right, 0 wrong, 0 ignored, 0 exceptions

The general syntax for TestRunner.py is:

TestRunner.py [options] host port pagename

From the multitude of options for TestRunner I used only -o, which specifies a directory to save the output files in. By default, TestRunner saves the HTML output for each test page, as well as a summary page for the test suite in XML format.

Another caveat: there is a bug in PyFIT 0.8a1 which manifests itself in PyFIT not recognizing the elements of your test tables if they're not in camel case format. So for example if you have a test table like this:

!|MyFitnesseFixtures.SetUp|
|initialize_test_database?|
|true|

you will need to change it to:

!|MyFitnesseFixtures.SetUp|
|initializeTestDatabase?|
|true|

You will of course need to have a corresponding method named initializeTestDatabase defined in the SetUp.py fixture and declared in the _typeDict type adapter dictionary.