Posts

Showing posts from August, 2006

On the importance of functional testing

I did not need further proof of the fact that functional tests are a vital piece in a project's overall testing strategy. I got that proof anyway last night, while watching the Pybots buildmaster status page. I noticed that the Twisted unit tests were failing, but not because of errors within the Twisted package, but because pre-requisite packages such as ZopeInterface could not be installed anymore. If you followed my post on setting up a Pybots buildslave, you know that before running the Twisted unit tests, I attempt to instal ZopeInterface and other packages using the newly-built python binary, via "/tmp/python-buildbot/local/bin/python setup.py install".

Well, all of a sudden last night this last command was failing with errors such as:
error: invalid Python installation:
unable to open /tmp/python-buildbot/local/lib/python2.6/config/Makefile
(No such file or directory)

This proved to be was a transient error, due to some recent checkins that modified the release numbers…

QA blog at W3C

Karl Dubost sent me a message about some issues he had with running Cheescake on a Mac OS X machine. It turned out he was using an ancient version of Cheesecake, although he ran "easy_install Cheesecake". I told him to upgrade to the latest version via "easy_install Cheesecake==0.6" and his problems dissapeared.

Anyway, this is not what I was trying to blog about. Reading his email signature, I noticed he works as a Conformance Manager at W3C. Karl also mentions a QA blog at W3C in his signature. Very interesting blog, from the little I've seen so far. For example, from the "Meet the Unicorn" post, I found out about a W3C project (code-name Unicorn) which aims to be THE one tool to use when you want to check the quality -- i.e. the W3C conformance I suspect -- of web pages. This tool would "gather observations made on a single document by various validators and quality checkers, and summarize all of that neatly for the user." BTW, here is a l…

Setting up a Pybots buildslave

If you're interested in setting up a buildbot buildslave for the Pybots project, here are some instructions:

Step 1

Install buildbot on your machine. Instructions can be found here, here, here and here.

Step 2

Create a user that will run the buildbot slave process. Let's call it buildslave, with a home directory of /home/buildslave. Also create a /home/buildslave/pybot directory.

Step 3

Create the file buildbot.tac in /home/buildslave/pybot, with content similar to this:

from twisted.application import service
from buildbot.slave.bot import BuildSlave

# set your basedir appropriately
basedir = r'/home/buildslave/pybot'
host = 'www.python.org'
port = 9070
slavename = 'abc'
passwd = 'xyz'
keepalive = 600
usepty = 1

application = service.Application('buildslave')
s = BuildSlave(host, port, slavename, passwd, basedir, keepalive, usepty)
s.setServiceParent(application)

Step 4

Create a python-tool directory under /home/buildslave/pybots. You must name this directo…

Cheesecake case study: Cleaning up PyBlosxom

Will Guaraldi wrote an article on "Cleaning up PyBlosxom using Cheesecake". Cool stuff!

Will, I hope we meet at the next PyCon, I owe you a case of your favorite beer :-)

Pybots -- Python Community Buildbots

The idea behind the Pybots project (short for "Python Community Buildbots") is to allow people to run automated tests for their Python projects, while using Python binaries built from the very latest source code from the Python subversion repository.

The idea originated from Glyph, of Twisted fame. He sent out a message to the python-dev mailing list (thanks to John J. Lee for bringing this message to my attention), in which he said:

"I would like to propose, although I certainly don't have time to implement, a program by which Python-using projects could contribute buildslaves which would run their projects' tests with the latest Python trunk. This would provide two useful incentives: Python code would gain a reputation as generally well-tested (since there is a direct incentive to write tests for your project: get notified when core python changes might break it), and the core developers would have instant feedback when a "small" change breaks more co…

Dave Nicolette's recommended reading list on agile development

Worth perusing. I always enjoy Dave's blog posts on agile development, so I trust his taste :-)

Cheesecake 0.6 released

Thanks to Michał's hard work, we released Cheesecake 0.6 today. The easiest way to install it is via easy_install: sudo easy_install Cheesecake

Update: Please report bugs to the cheesecake-users mailing list.

Here's what you get if you run cheesecake_index on Cheesecake itself:

$ cheesecake_index -n cheesecake
py_pi_download ......................... 50 (downloaded package cheesecake-0.6.tar.gz directly from the Cheese Shop)
unpack ................................. 25 (package unpacked successfully)
unpack_dir ............................. 15 (unpack directory is cheesecake-0.6 as expected)
setup.py ............................... 25 (setup.py found)
install ................................ 50 (package installed in /tmp/cheesecakeNyfM4f/tmp_install_cheesecake-0.6)
generated_files ........................ 0 (0 .pyc and 0 .pyo files found)
---------------------------------------------
INSTALLABILITY INDEX (ABSOLUTE) ........ 165
INSTALLABILITY INDEX (RELATIVE) ........ 100 (16…

A couple of Apache performance tips

I had to troubleshoot an Apache installation recently. Apache 2.0 was running on several Linux boxes behind a load balancer. If you ran top on each box, the CPU was mostly idle, there was plenty of memory available, and yet Apache seemed sluggish. Here are a couple of things I did to speed things up.

1. Disable RedirectMatch directives temporarily

All the Apache servers had directives such as:

RedirectMatch /abc/xyz/data http://admin.mysite.com/abc/xyz/data

This was done so administrators who visited a special URL would be redirected to a special-purpose admin server. Since the servers were pretty much serving static pages, and they were under considerable load due to a special event, I disabled the RedirectMatch directives temporarily, for the duration of the event. Result? Apache was a lot faster.

2. Increase MaxClients and ServerLimit

This is a well-known Apache performance optimization tip. Its effect is to increase the number of httpd processes available to service the HTTP requests.

Ho…