Monday, November 14, 2005

Exciting times in the Python testing world

If you are a developer or tester using Python, you live in exciting, ebullient times. There are Python-based testing frameworks newly-announced or recently-updated almost every day. Here is a rundown of the latest I'm aware of:

Unit testing
  • py.test: no recent new release, but changes are happening almost daily in svn
  • TestOOB: version 0.7 was released recently (TestOOB is an enhancement to the standard unittest module, offering many features that py.test offers)
  • nose: version 0.7.2 was freshly released yesterday (nose, in its author's words, "provides an alternate test discovery and running process for unittest, one that is intended to mimic the behavior of py.test as much as is reasonably possible without resorting to too much magic"; nose will become, if it's not already, the official test framework for TurboGears)
Web application testing
  • twill: version 0.7.4 was released on Nov. 11th, with unit tests that use nose, and with new commands to help developers use twill to unit test their own Web apps; also, Titus Brown announced yesterday that he extended twill to add in-process testing of WSGI applications (I blogged about twill here)
  • FunkLoad: version 1.3.1 was released on Nov.10th (FunkLoad offers functional, performance, load and stress testing of Web applications)
  • zope.testbrowser: version 0.9.0 was released on Nov.12th (zope.testbrowser is the stand-alone version of the Zope 3 functional testing framework)
  • Sancho: version 2.1 was released on Nov. 2nd (Sancho is the unit test framework for the MEMS Exchange applications; for those who are not familiar with MEMS Exchange, they are the guys behind Quixote, Durus and other Python apps)
  • Django's own doctest-based test framework: read my blog post about it
  • ibofobi's doctest-based framework for Django: blog post from Nov. 8th announces a new unit test framework for Django-based Web applications
  • [Update from Ian Bicking, the creator of Paste]: paste.test.fixture: the unit test framework used in Paste (read "What is Paste, yet again" on Ian's blog)
  • [Update from Robert Brewer, the creator of CherryPy]: webtest is a small, but helpful and isolated web-application-test module (it extends unittest) used to test CherryPy
GUI testing
  • guitest: version 0.3 was released on Nov. 13th (guitest is a Python helper library for testing Python GUI applications, with pyGTK support being the most mature)
  • retest: version 0.5.1 was released on Sept. 23rd (retest enables tests of Python regular expressions in a webbrowser; it uses SimpleHTTPServer and AJAX)
I also want to mention MochiKit as an example of an application that makes it a point to offer top-notch tests and documentation. MochiKit is a JavaScript library that is very "Pythonical" in nature, which is not surprising given that one of its main developers is Bob Ippolito, well-known for his contributions in the Python community. One of the goals of MochiKit is to maintain 100% documentation coverage, and another is to test itself mercilessly. If only all applications followed these tenets, the world would truly be a better place :-)


Ian Bicking said...

Hey, you forgot paste.fixture:

There's also a command-line tester in there, but I guess I haven't documented it. There's some notes in the docstrings:

Sancho seems more like a test runner (like py.test) than something specifically web testing oriented.

Anonymous said...

CherryPy has a small, but helpful and isolated web-application-test module (it extends unittest):

Benji York said...

Another one for your list: zope.testbrowser ( It's part of Zope 3, but is usable outside of Zope.

Grig Gheorghiu said...

Benji: zope.testbrowser was already mentioned in my post, under "Web application testing".

Paulo Eduardo Neves said...

Great! Now we managed to get more tests frameworks than web frameworks in python!

Modifying EC2 security groups via AWS Lambda functions

One task that comes up again and again is adding, removing or updating source CIDR blocks in various security groups in an EC2 infrastructur...