Python Fuzz Testing Tools

Ian Bicking suggested I create a new category in the Python Testing Tools Taxonomy: Fuzz Testing or Fuzzing. Done. If you're not familiar with the term, see the Wikipedia article which talks about this type of testing. Here's an excerpt: "The basic idea is to attach the inputs of a program to a source of random data ("fuzz"). If the program fails (for example, by crashing, or by failing built-in code assertions), then there are defects to correct. The great advantage of fuzz testing is that the test design is extremely simple, and free of preconceptions about system behavior."

Ian told me about the Peach Fuzzer Framework. I was familiar with Pester (the home page talks about a Java tool called Jester, and it has links to the Python version called Pester); I also googled some more and found other Python fuzzing tools such as antiparser and Taof, which are both geared towards fuzzing network protocols. In fact, many fuzzing tools are used in security testing because they can aid in attacking software via random inputs. See this Hacksafe article on "Fuzz testing tools and techniques" and this PacketStorm list of fuzzing tools. Another good overview is Elliotte Harold's developerWorks article on fuzz testing. Very interesting stuff. If the "Python Testing Tools" tutorial Titus and I proposed for PyCon gets accepted, expect to see some fuzz testing included in our arsenal :-)

I also added Ian's minimock tool to the PTTT page. Very cool minimal approach to mock testing, achieved by embedding mocking constructs in doctests.

In other testing-related blog posts, Titus talks about the difficulty of retrofitting testing to an existing application (even when you wrote the testing tools!), and Max Ischenko presents some uber-cool plugins which integrate nose into vim.