Showing posts from September, 2005

HoneyMonkeys: an adventure in black box testing

HoneyMonkeys is the name of a Microsoft research project in computer security. It combines the concept of honeypots with an attitude of "monkey see, monkey do". Specifically, it consists of a cluster of WinXP machines with various configurations (SP1, SP2 non-patched, SP2 partially patched, SP2 fully patched) running as Virtual Machines for easy rollout and reloading.

The XP machines run the IE browser in an automated fashion, pointing it to sites known or suspected for hosting malware. Each machine also runs monitoring software that records every single file and Registry read/write, as well as any attempt to hook malware into Auto-Start Extensibility Points -- for many more details on this see this research report from Microsoft. The machines act as "monkeys" by merely pointing the browser to suspected malicious Web sites and then waiting for a few minutes. The automated IE drivers do not click on any dialog box elements that might prompt for installation of soft…

Oblique Strategies and testing

A message posted to comp.lang.python pointed me to a post by Robin Parmar on Oblique Strategies. I had read about this concept before, but I didn't really delve into it, so it was nice to see it mentioned again. The Oblique Strategies are one-line sentences devised by Brian Eno and Peter Schmidt as ways to "jog your mind" and get you unstuck in moments when your creative juices don't flow as much as you would like to. They offer "tangential" solutions to problems, as opposed to the more obvious, and oftentimes futile, "head-on" solutions.

It strikes me that the Oblique Strategies could be an important tool in a tester's arsenal. After all, good testers should be able to "sniff" problems that are not obvious; they should be able to go on "tangents" at any time, to follow their intuition in finding bugs that might be due to subtle interactions. I find it funny that, according to Brian Eno, the very first Oblique Strategy he wr…

Web app testing with Python part 3: twill

In a recent thread on comp.lang.python, somebody was inquiring about ways to test whether a Web site is up or not from within Python code. Some options were proposed, among which I referred the OP to twill, a Web application testing package written in pure Python by Titus Brown (who, I can proudly say, is a fellow SoCal Piggie).

I recently took the latest version of twill for a ride and I'll report here some of my experiences. My application testing scenario was to test a freshly installed instance of Bugzilla. I wanted to see that I can correctly post bugs and retrieve bugs by bug number. Using twill, all this proved to be a snap.

First, a few words about twill: it's a re-implementation of Cory Dodt's PBP package based on the mechanize module written by John J. Lee. Since mechanize implements the HTTP request/response protocol and parses the resulting HTML, we can categorize twill as a "Web protocol driver" tool (for more details on such taxonomies, see a previous…

CherryPy, Cheetah and PEAK on IBM developerWorks

I haven't read these articles yet and I wanted to have the links in one place for future reference:
CherryPy article by Leonard Richardson
Cheetah article by the samePEAK article in the Charming Python series by David Mertz

Jakob Nielsen on Usability Testing

Do you spend one day per week on observing how new users interact with your product? In fact, do you have any usability testing at all in your budget? In the rare event that you do run usability testing sessions, do you focus on actual user behavior (and not waste time by having users fill in endless questionnaires)? If you tend to answer "no" to these questions, read Jakob Nielsen's article on how to properly conduct usability testing sessions.

Running a Python script as a Windows service

This is a message I posted to regarding ways to run a regular Python script as a Windows service.

I will assume you want to turn a script called into a service.
1. Install Win2K Resource Kit (or copy the 2 binaries instsrv.exe and srvany.exe).

2. Run instsrv to install srvany.exe as a service with the name myscript:
C:\Program Files\Resource Kit\instsrv myscript "C:\Program Files\Resource Kit\srvany.exe"

3. Go to Computer Management->Services and make sure myscript is listed as a service. Also make sure the Startup Type is Automatic.

4. Create a myscript.bat file with the following contents in e.g. C:\pyscripts:

C:\Python23\python C:\pyscripts\

(replace Python23 with your Python version)

5. Create new registry entries for the new service.
run regedt32 and go to the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\myscript entry add new key (Edit->Add Key) called Parameters add new entry for Parameters key (Edit->Add Value) to set the …

Michael Feathers on Unit Testing Rules

Short but high-impact post from Michael Feathers (of "Working Effectively with Legacy Code" fame). His main recommendation is to have unit tests that do not interact with the OS or other applications. Interactions to avoid include databases, sockets, even file systems. When you have a set of unit tests that run in isolation (and thus run very quickly), and when you have other sets of tests that do exercise all the interactions above, you are in a good position to quickly pinpoint who the culprit is when a test fails.

Recommended site: QA Podcast

I got an email from Darren Barefoot pointing me to a site he helped put together: QA Podcast. Very interesting stuff: interviews/conversations about software testing with folks who care and have something to say on this subject. I was glad to see that the podcasts published so far cover subjects such as performance testing and exploratory testing. I listened so far to a conversation on exploratory testing with James Bach and I already took away a ton of ideas I can apply in my testing activities.