Porting to the Linux Standard Base

IBM developerWorks offers a tutorial (free registration required) on "Porting to the Linux Standard Base". Excerpt from the introduction:

"Because Linux® is an open operating system, you can configure and assemble it to suit specialized purposes. However, while variety and choice are beneficial for users, heterogeneity can vex software developers who must build and support packages on a multitude of similar but subtly different platforms. Fortunately, if an application conforms to the Linux Standard Base (LSB), and a flavor of Linux is LSB compliant, the application is guaranteed to run. Discover the LSB, and learn how to port your code to the standard."

Adhering to standards -- I'm all about that, although I've been called names before for showing enthusiasm for Python project layout standardization...Anyway, I'm glad to see the Linux community pushing the LSB, since this will benefit both distribution creators and application writers.

GK-H on "Myths, lies and truths about the Linux kernel"

"Myths, lies and truths about the Linux kernel" is the title of Greg Koah-Hartman's closing keynote at OLS 2006. Fascinating read, especially when Greg talks about the apparently chaotic Linux kernel development process, which turns out to be amazingly flexible and evolutionary. I was also impressed by the arguments for open-source drivers that are maintained and modified at the same time with the kernel -- this make a stable internal kernel API unnecessary, and allows the kernel to evolve.

Titus sent me the link to the keynote, and he also underlined this paragraph related to testing:

"Now, this is true, it would be great to have a simple set of tests that everyone could run for every release to ensure that nothing was broken and that everything's just right. But unfortunately, we don't have such a test suite just yet. The only set of real tests we have, is for everyone to run the kernel on their machines, and to let us know if it works for them."

It's pretty amazing to me that the Linux kernel manages to be that stable without a regression test suite. Imagine how much better it would be with such a regression suite. Clearly, a community project waiting to happen.

In the mean time, back to my pybots.... also waiting for that to happen -- but at least it's a bit more realistic from my perspective, and I hope to be able to give you more details as the PSF is working on getting a server to configure the buildmaster on.

Marick on refactoring

Brian Marick just posted his definition of refactoring: "A refactoring is a test-preserving transformation." It very succintly expresses the critical need for tests. If you don't have tests, how do you know refactoring preserves anything? Great stuff as usual from Brian M.

Emmental

The fourth Cheesecake/SoC iteration has been completed -- code name emmental. Here are some stories that Michał implemented in this iteration:

• Implement --static command line flag, which makes Cheesecake do only static tests, that don't execute any of package code. This is useful for example for the Cheesecake/PyPI integration, where we'll only look at 'static' indexes such as documentation and installability, as opposed to 'dynamic' indexes which involve code execution, such as unit test coverage;
  • Make execution of some parts of code depending on "static" flag.
  • Implement static "profile" - a subset of all indices that scores only statically.
  • Technical detail related to this story: a nice touch from Michał was the implementation of index dependencies via this changeset
    

• Implement --lite command line flag, which makes Cheesecake ignore time-consuming tests, such as the pylint test;

• Static unit test analysis (lots of work still to be done here);
  • Use Michael Hudson's AST-based pydoctor package
  • Compute proportion of number of code/functions and tests.

OpenWengo Code Camp

Found this via the buildbot-devel mailing list: OpenWengo Code Camp. It seems similar in philosophy and goals to the Google Summer of Code. Excerpt from the home page:

"OpenWengo Code Camp is a friendly, challenging and mind-stimulating contest aimed at pushing open source software projects forward.
Students apply for proposed software development subjects for which they have a particular interest in. These subject proposals describe ways to bring enhancements to existing or new FOSS projects, generally by writing source code.

If their application is accepted, they get the chance to be mentored by open source software contributors to work during 2 months on the subject for which they applied. At the end of summer, mentors give their appreciation: if goals were successfully reached, students get 3500 euros of cash.

Mentors get 500 euros of cash if they played their role which consist mainly in helping students to complete their work successfully and evaluating their work at intermediate and final stages."

Sounds pretty reasonable to me, and the cash is not bad either :-)

Looks like one of the proposals involves building a SQL backend for buildbot -- hopefully the project will go through.

Devon

The 3rd milestone for the Cheesecake/SoC project has been completed -- code name devon. This iteration had 3 stories:

1. Create functional tests that actually execute cheesecake_index script. Check that Cheesecake is:

• properly cleaning up
• leaving log file when package is broken and is removing it otherwise
• computing score properly
• handling its command line options properly

2. Write script that will automatically download and score all packages from PyPI.

• Each package should have its score and complete Cheesecake output logged.
• Gather time statistics for each package.
• Make a summary after scoring all packages:
  • number of packages for which Cheesecake raised an exception
  • manually check first/last 10 packages and think about improving scoring techniques

3. Add support for egg packages

• Refactor supported packages interface
• Add support for installing eggs via setuptools easy_install

As far as story #2 is concerned, Michał and I discussed some modifications and tweaks we need to do to the scoring algorithms so that more packages get higher scores. Here are some ideas that we already implemented:

• don't decrease installability score if a package is not hosted on PyPI (the package still needs to have a valid download link on its PyPI page);
• split required files and directories into 3 categories: high, medium, and low importance, each category getting a score of 30, 20, and 10 points respectively;
  
• here is the current classification, where Doc means the file can also have a 'txt' or 'html' extension, and OneOf means the score is given if any one of the files/directories in the specified list is found:

cheese_files = {
  Doc('readme'): 30,
  OneOf(Doc('license'), Doc('copying')): 30,
  OneOf(Doc('announce'), Doc('changelog')): 20,
  Doc('install'): 20,
  Doc('authors'): 10,
  Doc('faq'): 10,
  Doc('news'): 10,
  Doc('thanks'): 10,
  Doc('todo'): 10,
}
cheese_dirs = {
  OneOf('doc', 'docs'): 30,
  OneOf('test', 'tests'): 30,
  'demo': 10,
  OneOf('example', 'examples'): 10,
}

We're getting ready to release cheesecake out in the wild pretty soon, I'd say in a couple of weeks -- so stay tuned!

We've also seen some activity on the cheesecake-users and cheesecake-dev mailing lists, and as always we encourage people interested in this project to send us feedback/suggestions/criticisms. We've been known to always take constructive criticism into account :-)

Update

Read also Michał's post on devon.

Camembert

The second week of the Cheesecake/SoC project has ended, and all the stories have been completed. We chose the name camembert for this iteration. It included some very tasty (or should I say tasteful) refactoring from Michał, who sprinkled some magic pixie dust in the form of metaclasses and __getitem__ wizardry. It also included some development environment-related tasks, all of them executed via buildbot: automatically generating epydoc documentation and publishing it, running coverage numbers and publishing them, and converting the reST-based README file into Trac Wiki format. This last task had as a side-effect the creation of a little tool that Michał called rest2trac, which will be made available in the near future. Currently it does the conversions that we need for the markup we use in the README file.

All in all, another productive week, and lots of good work from Michał. Check out his Mousebender blog for more information.

Xen installation and configuration

Courtesy of my co-worker Henry Wong, here's a guide on installing and configuring Xen on an RHEL4 machine.

Introduction

Xen is a set of kernel extensions that allow for paravirtualization of operating systems that support these kernel extensions, allowing for near-native performance for the guest operating systems. These paravirtualized systems require a compatible kernel to be installed for it to be aware of the underlying Xen host. The Xen host itself needs to be modified in order to be able to host these systems. More information can be found at the Xen website.

Sometime in the future, XenSource will release a stable version that supports the installation of unmodified guest machine on top of the Xen host. This itself requires that the host machine hardware have some sort of virtualization technology integrated into the processor. Both Intel and AMD have their own versions of virtualization technology, VT for short, to meet this new reqirement. To distinguish between the two competing technologies, we will refer to Intel's VT as its codename, Vanderpool, and AMD's VT as Pacifica.

---

Installation

Before starting, it is highly recommended that you visit the Xen Documentation site. This has a more general overview of what is involved with the setup, as well as some other additional information.

Terminology

• domain 0 (dom0): In terms of Xen, this is the host domain that hosts all of the guest machines. It allows for the creation and destruction of virtual machines through the use of Python-based configuration files that has information on how the machine is to be constructed. It also allows for the management of any resources that is taken up by the guest domains, i.e. networking, memory, physical space, etc.

• domain U (domU): In terms of Xen, this is the guest domain, or the unpriviledged domain. The guest domain has resources assigned to it from the host domain, along with any limits that are set by the host domain. None of the physical hardware is available directly to the guest domain, instead the guest domain must go through the host interface to access the hardware.

• hypervisor: Xen itself is a hypervisor, or in other words, something that is capable of running multiple operating systems. A more general definition is available here.

Prerequisites

Xen Hypervisor Requirements

• A preexisting Linux installation, preferably something running 2.6. In this case, we'll be running with Redhat Enterprise Linux 4 Enterprise Server Update 3.

• At least 1GB or more of RAM

• 40GB+ disk space available

• (OPTIONAL) Multiple CPU's. Hyperthreading doesn't count in this case. The more, the better, since Xen 3.0 is capable of virtualized SMP for the guest operating system.

Guest Domain Requirements

• A preexisting Linux installation, preferably something running either the same kernel version as the host-to-be or newer. More on this later in the page.

• Some storage for the guest domain. An LVM-based partitioning scheme would be ideal, but you can use a file to back the storage for the machine.

Xen Hypervisor Installation Procedure

1. Obtain the installation tarball from XenSource Download Page. In this case, grab the one for RHEL4.

2. Extract the tarball to a directory with sufficient space and follow the installation instructions that are provided by XenSource. For RHEL4, it is recommended that you force the upgrade of glibc and the xen-kernel RPMs. This will be explained in detail further in the page.

3. Append the following to the grub.conf/menu.lst configuration file for the GRUB bootloader:

   title Red Hat Enterprise Linux ES-xen (2.6.16-xen3_86)
     root (hd0,0)
     kernel /xen-3.0.gz dom0_mem=192M
     module /vmlinuz-2.6-xen root=/dev/VolGroup00/LogVol00 ro console=tty0
     module /initrd-2.6-xen.img
   

   This might change depending on the version that is installed, but for the most part, using just the major versions should work. Details about the parameters will be explained later in the page.

4. Reboot the machine with the new kernel.

The machine should now be running the Xen kernel

Guest Domain Storage Creation Procedure

LVM Backed Storage

By default, RHEL4 (and basically any new Linux distribution that uses a 2.6 kernel by default) uses the LVM (Logical Volume Manager) in order to keep track of system partitions in a logical fashion. There are two important things about LVM, the logical volume and the volume group. The volume group consists of several physical disks that are grouped together during creation, with each volume group having a unique identifier. Logical volumes are then created on these volume groups, and can be given a unique name. These logical volumes are able to grab a pool of available space on the volume group, with any specified size, properties, etc. If you wish to learn more about LVM, a visit to the LVM HOWTO on the Linux Documentation Project site is recommended.

Physical Partition Backed Storage

Far easier to create than an LVM, but with a little less flexibility, the physical partition backed storage for a guest machine just uses a system partition to store the data of the virtual machine. This partition needs to be formatted to a filesystem that is supported by the host, if you are to use the paravirtualization approach for domain creation.

File-Backed Storage

By far the easiest way to get a guest domain up and running, a file-backed store for the guest allows you to put the file anywhere where there is space. You wouldn't have to give up any extra partitions in order to create the virtual machine. But, this incurs a performance penalty.

Guest Domain Installation Procedure

1. Create an image tarball from the preexisting Linux installation for the guest. Use tar along these lines:

   tar --exclude=/ --exclude=/sys/* --exclude=/tmp/* --exclude=/dev/* --exclude=/proc/* -czpvf  /
   

   Note that the excludes are before rather than after the short flags. This is because the -f short option is positional, and thus it needs a name immediately after the option.

2. Move the tarball over to the Xen hypervisor machine.

3. Mount the desired location of the guest storage on the hypervisor.

4. Unpack the tarball into the guest storage partition.

5. Copy the modules for the Xen kernel into the guest's /lib/modules directory. You can use the following command to copy the modules directory, replacing with the guest storage mount point:

   $ cp -r /lib/modules/`uname -r`/ /lib/modules/
   

6. Move the /lib/tls directory to /lib/tls.disabled for the guest. This operation is specific to Redhat-based systems. Due to the way that glibc is compiled, the guest operating system will incur a performance penalty if this is not done. Ignore this step for any non-Redhat systems.

Initial setup of the guest is completed.

---

Running With Xen

Creating and starting a guest domain

1. Create a guest configuration file under /etc/xen. Use the following example as a guideline:

   kernel = "/boot/vmlinuz-2.6-xen"                # The kernel to be used to boot the domU
   ramdisk = "/boot/initrd-2.6.16-xenU.img"        # Need the initrd, since most of these systems run udev
   
   memory = 256                                    # Base memory allocation
   name = "xmvm1"                                  # Machine name
   cpus = ""                                       # Specific CPU's to assign the vm, leave blank
   vcpus = 1                                       # Number of available CPU's to the system
   vif = [ '' ]                                    # Defines the virtual network interface
   
   # LVM-based storage
   disk = [ 'phy:VolGroup01/xenvm1-root,hda1,w',   # Guest storage device mapping to the virtual machine
          'phy:VolGroup01/xenvm1-swap,hda2,w' ]
   
   root = "/dev/hda1 ro"                           # Root partition kernel parameter
   

2. Mount the guest storage partition and edit the /etc/fstab for the guest to reflect any changes made to the configuration file. Remove any extraneous mount points that won't be recognized by the guest when the system is started, otherwise the guest machine will not boot.

3. Start the maching using the following command:

   $ xm create -c 
   

   This will create the machine and attach it to a virtual console. You can detach from the console using CTRL-].

Further setup is still required, but it is OS-specific. The network interfaces will need to be setup for the guest machine.

Python at UC Riverside

I interviewed a candidate for a QA position a couple of days ago; he had a bachelor's degree in Comp. Science from UC Riverside. I was happy to find out that Python is the main language taught there, along with C++. They participated in an XP-style project based on Python and Pygame. What's not to like?

One thing though that I don't really get is that they didn't seem to put emphasis at all on unit testing. They had short iterations, customer feedback, pair programming, but no unit tests. How can you teach XP without stressing the importance of unit tests? To me "XP with no unit tests" is an oxymoron, up there with "making soup in a sieve", or even -- dare I say -- "work on the Cheesecake project to keep the cuddly teddy-bear of an effbot happy" (extra Cheesecake points to anyone who can spot the multiple oxymorons in the last phrase.)

Update 06/11/06

Peter Fröhlich contacted me via email, and told me that, in all probability, the XP class I mentioned is one that he taught when we was at UCR. Peter pointed out that unit tests were mentioned in the class, and students were encouraged to use them, but were not forced to do so. I stand corrected in my assessment above. This shows that fact checking should be a practice more widely used by bloggers! :-)

I'm also happy to report that Peter uses Python in the classes he teaches at JHU too. I just wish more people in academia would follow his example.

Got brie?

Cheesecake/Summer of Code project: 1 down, 11 to go. The numbers represent weeks that are allocated for Google SoC projects this year, up to the August 21st deadline.

We decided to have 1-week iterations, which we entered as milestones in Trac. Each iteration consists of several stories which are entered as tickets of type 'enhancement' in Trac. Stories are estimated in points, with roughly 2 points per day. So a 4-point story is estimated at roughly 2 days of work. We'll keep the maximum number of points per iteration to 8, a bit less than the maximum velocity, but more realistic, because there's always something that comes up and needs to be taken care of. Plus, a story is not done if it's not well tested.

Each story is split into tasks that take roughly a few hours each, and the tasks are entered as tickets of type 'task' in Trac (here are all the tickets we've entered so far, by milestone.)

To keep things fun, each iteration has a code name inspired by cheese varieties out of the Monty Python Cheese Shop skit. We assign code names in alphabetical order, skipping certain letters because of obscure/hard-to-remember cheese varieties for that letter -- such as appenzeller.

The first milestone/iteration was the delicious brie. I'm very pleased to report that Michał made great progress and completed the following 2 stories that were selected for this iteration:

• Bugfixes (2 points)
  
  • Score is decresed for .pyc files, the same should be done for .pyo.
  • Filenames checking should be a bit more restrictive than checking regular expression. Readme, README and readme are acceptable names, but ReAdMe is not.
  • Files that change cheesecake index cannot be empty

• Docstring index enhancements (6 points)

• Use the latest pydoctor.
• Check docstrings contents: make sure they're not empty.
• Check docstrings for use of epytext.
• Check docstrings for use of ReST.
• Check docstrings for use of Javadoc.
• Write finer-grained unit tests for docstring index.

I think that splitting the workload into stories, and stories into well-defined tasks, has helped the development and testing process a lot, while keeping the project light (or agile, if you prefer) on project management overhead. Another thing I like about Trac is that it shows closed ticket numbers with a strike-through line, so by enumerating the tasks that a story depends on -- like here -- we can see at a glance when the story has been completed.

We also have a 'product backlog' of sorts: the SummerOfCode06 Wiki page where we jot down ideas about things to implement next. Stories for the next iterations will be chosen out of the pool of stories that are already there. The rough month-to-month schedule is: June is mostly about Cheesecake 'core' enhancements, July will be dedicated to the PyPI integration, and August to investigating and implementing ways to run Cheesecake in a VM/sandbox. Things might shift a bit, but having small iterations and frequent releases will help in adapting to changes that will undoubtedly occur. Speaking of releases, we're planning on releasing Cheesecake itself around the end of June.

Also check out Michał's SoC blog -- aptly named Mousebender -- for more details on what he's working on.

I encourage people interested in this project to start posting to the cheesecake-dev mailing list. The more feedback we get from the community, the more we can fine-tune the various Cheesecake index measurements.

And speaking of feedback: some people expressed their displeasure at having the Cheesecake scores up on PyPI, thinking that this would turn PyPI into a 'hall of shame'. A good compromise might be to make the Cheesecake score for a given PyPI package visible only to the creator of the package (Richard Jones's idea), while showing the top N or top X% packages in each index category to everybody, so that people can have practical examples of packages that scored high (my idea). This would avoid the whole 'public hall of shame' controversy, and turn Cheesecake+PyPI into a 'public hall of fame'. Michał is also thinking about adding links to explanatory pages next to PyPI scores, so that people understand what counts for the score, why it's worth improving it and how to improve it.

Here also is some great feedback from Will Guaraldi. We'll certainly start working on this very soon:

"Maybe create a --recommend option that for each category if the score is less than half (or something along those lines), Cheesecake spits out a short blurb about what the category score means and a url for where to go for more information about how to fix it. Then at that url (it could point to a specific page on this wiki) are resources regarding that category.

For example, somewhere else on this wiki I asked about where I might find documentation on what kind of information should go into files like README, CHANGELOG, ... That information would be really useful to have for all the categories.

If you need help--I'd be happy to help build such documentation. If you we put the information in the wiki, then it can grow and evolve over time which would also be useful because it might provide feedback into those specific categories and tests."

Anyway, for now, enjoy the brie and get ready for some camembert!

Cheesecake mailing lists

If you are interested in the Cheesecake project, you can now subscribe to two mailing lists: cheesecake-dev and cheesecake-users (thanks, Titus!)

In the near future, most of the discussions will take place on cheesecake-dev, since the Cheesecake/SoC project is in full swing.

Michał and I already posted a couple of threads with feedback that we got from the CommentsPage on the Trac Wiki. We'd love to get more feedback, so please don't spare us! You can also add ideas to the SummerOfCode06 Wiki page.

Sparklines and sparkplot

Via Darren Rowse: how infosthetics is using sparklines to display its relative daily Google AdSense earnings. Looks like they're using the PHP sparkline library, but hey, you can always give my sparkplot module a try too! Maybe you want to restore romance to the sports page? Then sparkplot might just be the ticket :-)

Several people expressed interest recently in sparkplot, so I guess it's time for me to dust it off a bit and release it in the wild. Stay tuned.

Treasure trove: AYE conference articles

If you're interested in Amplifying Your Effectiveness, then you'll enjoy these articles from the AYE conference, written by AYE hosts and guests who explore both the technical and the human sides of software and IT development. Also check out the links to various blogs on the AYE conference page. To summarize in 3 short words: Jerry Weinberg rulez :-)

Sandboxes everywhere

Jon Udell's most recent InfoWorld column talks about "Easing app deployment with an open source sandbox". The article talks about a hosting service which offers an automated sandbox installer and thus considerably eases the deployment and testing of packages. This resonates with one of the main goals of the Cheesecake Summer of Code project: to offer a sandbox environment where Python packages can be uploaded and inspected dynamically, by running their unit tests, getting code coverage numbers, etc.

To me, 2006 seems to be the year of the virtual machine. Here's the equation:

commodity hardware + solid open-source virtualization technologies such as Xen = great opportunities for testers

Many companies have already started to capitalize on this equation (Autoriginate/HostedQA is just one example). The beauty of Open Source however makes this opportunity available to anybody who possesses a medium-to-high amount of Linux hacking skillz :-).

I'll post more about this topic as work on Cheesecake/SoC progresses. The way I see it, we'll offer a way for people to post their packages to one of our servers, and we'll compute all the dynamic Cheesecake scores (such as code coverage obtained by running unit tests) in a dedicated virtual machine. These dynamic scores will also be computed when a request comes from the PyPI interface. This is all on the drawing board right now, but that's the general idea.

Another related project that hasn't been started yet, based on an idea that Titus had last year, would be to automatically apply patches to Python core, compile, build and run all unit tests, all of this in a safe sandbox environment. This will hopefully lower the barrier of accepting patches into Python core.

Michael Feathers on refactoring and continuous integration

From the author of "Working effectively with legacy code", Michael Feathers, a blog post on "Refactoring needs more than tests". Interesting point of view: sometimes unit tests are not enough when it comes to refactoring.

M.F. disusses the situation of a project with a large code base, where several teams work on different releases/branches of the code at the same time. How do you refactor with confidence in this case? Another scenario discussed in the post is a project with dependencies on 3rd party code. How do you refactor with confidence, when you know that the 3rd party code keeps changing and needs to be patched everytime it gets updated?

Michael's answer is: integrate frequently. I say: buildbot to the rescue! :-)

Cheesecake and the Summer of Code

I'm very pleased to announce that Michał Kwiatkowski's project "Cheesecake enhancements and its integration with PyPI" was accepted as a Google Summer of Code project under the Python Software Foundation umbrella. Here's a summary of Michał's application:

Cheesecake is an application designed to evaluate and estimate the overall quality (or so called 'kwalitee') of a given software package written in Python. It emphasizes a need for well-written documentation and unit tests, encouraging good programming practices and penalizing sloppy design and careless distribution. Using Cheesecake to check your code gives you confidence that your software doesn't merely run, but is usable and easy to test and modify as well.

Because Python is very easy to learn and use there exists a vast variety of software written in it, most of which was scattered until PyPI was created. Now, when new packages are being indexed on Cheese Shop every day, an effort can be made to spread the spirit of good software design and code reuse among the Python community. This can be achieved by combining the power of Cheesecake and Cheese Shop. Everytime a new version of a package would be uploaded to Cheese Shop, its cheesecake index will be calculated and published on web. Having a way to measure a quality of a package with accordance to other existing packages will be of invaluable help for all developers. It will promote well built packages and in the long run raise the overall quality of Python software.

Adding Cheesecake functionality to PyPI has been already mentioned by Phillip J. Eby on the catalog-sig mailing list. Together with Cheesecake maintainer Grig Gheorghiu we've discussed modifications needed to be done to Cheesecake code to be reliable enough so it could be incorporated into PyPI service. A working copy of our ideas is accessible on the project wiki. It includes enhancing Cheesecake code scoring techniques to take into account unit tests of a package, running tests in secure environment, extending supported archive formats and fixing all known bugs. Development of Cheesecake will adhere to best practices such as unit testing, continuous integration (via buildbot), pylint verification, etc.

The next part of this project will include collaboration with Richard Jones, PyPI maintainer, and merging Cheesecake into PyPI service. Upon completion all PyPI uploads will be automatically scored by Cheesecake. It will be possible to browse packages archive by cheesecake index, sorting results by installability, documentation and code kwalitee index. Statistics in numeric and graphical form will also be made available. This part of a project will involve writing server-side code, with emphasis on security and robustness.

The remaining time will be spent on resolving all problems that would occur during usage of Cheesecake and PyPI. Along with fixing bugs, I will develop a simple Hello world package that can be taken as an example of good development practices for all Python developers. It should also score 100% in the Cheesecake test of course. It will be what hello is for GNU Project.

If you're interested in details, this Cheesecake wiki page contains a lot of ideas which will start being turned into reality as of today :-) Please feel free to edit the page and add your own wishlist-type items.

Here are a few thoughts I had regarding the value of this project:

This project will have 2 very important contributions: first of all, it will integrate with PyPI and help rank the Cheeseshop packages according to various quality criteria. People learn better by example -- and what better examples than tools that score high on a scale that looks at different quality indicators such as documentation, installability, and code 'kwalitee'? Cheesecake will provide a way to identify the best-of-breed packages in those areas.

Second, the project will investigate ways to dynamically assess packages by executing their code in a sandbox environment. This will help mainly with getting code coverage numbers by running a project's unit tests, but one can easily envision many other applications -- one idea that Titus Brown had was to automatically apply and verify patches to Python core, without the fear that the host machine will crash and burn. This will hopefully streamline the process of accepting patches into Python core (a famously complicated process currently).

Michał and I will use Trac to manage this project. The idea is to have short iterations represented as milestones in Trac, with tickets of type 'enhancement' that represent the stories to be done in each iteration. Each story will be split into short tasks that can be accomplished in a matter of hours, and each task will be represented as a ticket of type....'task', what else? This will give us a nice way of watching the progress of the project over the summer. Of course, the criterion for the completion of a given story is: all unit/acceptance/functional tests should pass for that story.

I'm very excited to have Michał work on this project and I'm very hopeful that at the end of this summer we'll have a solid application that will benefit the Python community.

Here is the list of the 25 applications accepted to the Summer of Code under the PSF umbrella.

Dynamically updating buildbot status text

Let's assume you want to update the build step status text displayed in the buildbot HTML status page, based on some information that only the build slave knows -- such as a version number that is computed by the slave during the build step for example.

Note that if you just want to customize the build step status with some text that is known in advance by the master (e.g. "client install" or "twill functional tests"), all you need to do is to subclass from ShellCommand and set the descriptionDone class variable to the desired custom text. See this post for more details on how to do this.

For dynamically updating the status text, the solution I found was to override some of the methods in the ShellCommand class.

My particular scenario is this: the build slave installs some package and identifies its version number. I want to be able to display that version number in the status for that build step.

I defined the following subclass of ShellCommand:

class ClientInstall(ShellCommand):
name = "client install"
description = ["running %s" % name]
descriptionDone = [name]

def __init__(self, **kwargs):
    ShellCommand.__init__(self, **kwargs)
    self.version = None

def createSummary(self, log):
    log_text = log.getText()
    s = re.search("--version=(.*)", log_text)
    if s:
        self.version = s.group(1)

def getText(self, cmd, results):
    text = self.describe(True)[:]
    if results == WARNINGS:
        text.append("warnings")
    if results == FAILURE:
        text.append("failed")
    if self.version:
        text.append("version=" + self.version)
    return text

The two most important methods in this case are createSummary and getText. I chose createSummary for overriding because it has access to the slave's log. In my case, that log contained the version number computed by the slave, so I just introduced a new variable, self.version, and set it to the result of a regular expression search for "--version=(.*)".

The getText method is called by ShellCommand inside the setStatus method, like this (the ShellCommand class lives in the process/step.py file installed under the buildbot root installation directory, in my case /usr/local/lib/python2.4/site-packages/buildbot):

def setStatus(self, cmd, results):
    # this is good enough for most steps, but it can be overridden to
    # get more control over the displayed text
    self.step_status.setColor(self.getColor(cmd, results))
    self.step_status.setText(self.getText(cmd, results))
    self.step_status.setText2(self.maybeGetText2(cmd, results))

My overridden version of getText (shown above) checks to see if self.version is non-empty, and if this is the case, it appends it to the variable text, which is a copy of the list returned by self.describe(True). Copying the list into a variable instead of modifying it in place is very important. Initially, I did something like:

text = self.describe(True)
if results == WARNINGS:
    text += ["warnings"]

The net effect of this was that each of my build slaves was updating this particular build step status with version information from all the other build slaves. It felt like a global or class-wide variable was being trampled under foot by all the build slaves, and indeed this was the case, as I found out when I sent a message to the buildbot-devel list and Brian Warned explained what was going on: the list of strings returned by self.describe() is a class-wide value that's not supposed to be mutated. Brian suggested modifying the above snippet of code to:

text = self.describe(True)
if results == WARNINGS:
    text = text + ["warnings"]

Neal Norwitz suggested the solution I finally adopted, which is to first make a copy of the list returned by self.describe, then append to it. This is more efficient, because it only allocates the list one time, then resizes it if necessary:

text = self.describe(True)[:]
if results == WARNINGS:
    text.append("warnings")

Once again, buildbot proved to be very flexible and customizable -- but not without jumping through some hoops in this particular scenario. In any case, I hope this post will be useful for buildbot users out there who want to display more customized information in their build steps.

SSH tunnelling with Putty

Courtesy of David Hancock, here's a mini-howto on configuring Putty for SSH tunnelling. Let's say you have an account on a Linux box (with an IP address of 192.168.2.100) that you can SSH into. Let's say you want to connect to a Trac instance running on port 8000 on a different box (with IP 192.168.2.200), and you can't get directly to port 8000 on the second IP. You can still use your account on the first box and create an ssh tunnel that will allow you to get to port 8000 on IP #2.

Here's David's howto, almost verbatim:

What we'll do is forward port 9080 on the PC to 8000 on 192.168.2.200 (the host/port for Trac). I'm using Putty version 0.54.

1. Start Putty (so you're looking at the PuTTY Configuration screen.)
2. Enter 192.168.2.100 (the IP of the box you can ssh into) in the Host name / IP address box.
3. Check SSH as the protocol (port number should change to 22.)
4. Enter 'trac-tunnel' as the Saved Sessions name, and click Save.
5. Open the Connection list in the left pane.
6. Open the SSH list in the left pane, Click Tunnels.
7. Check X11 Forwarding (in case you need to run X-based applications.)
8. Back on the right side, at the bottom, enter 9080 for source port (there's nothing special about port 9080, it can be any non-used port on your local machine.)
9. Enter 192.168.2.200:8000 as the Destination, leave Local checked.
10. Click Add.
11. Important, easy to forget: Click Session on the left pane, Click Save.

Now your 'trac-tunnel' session will not only connect you to the .100 box, but when you're logged into the .100, it will mediate a tunnel between your PC's port 9080 and port 8000 on 192.168.2.200.

So, let's try it out:

1. Use Putty to open the 'trac-tunnel' connection, and log in as yourself
2. Point your browser to http://127.0.0.1:9080/ and you'll get right in.

You'd repeat Steps 8-10 to add more local port forwardings. Step 11 is easy to forget, so be warned...

Zen of Unicode

I attended David Goodger's Unicode talk at PyCon earlier this year and I thought I'm well on my way to Unicode enlightenment. It turns out I still need to chop a lot of wood, carry a lot of water before I attain this particular Zen...In the hope that other people will find it useful, here's a mini-tutorial on Unicode in the form of an email message from David, who responded in excruciating detail to some Unicode-related questions I sent him. I tried to copy and paste the text into the Blogger editor, only to get all sorts of markup-related errors, so I just put it on a Trac wiki. Hopefully David will soon publish his Unicode tutorial on the Web. Until then, happy Unicode hacking!

Selenium test creation and maintenance with make_selenium.py

Michał Kwiatkowski is the author of make_selenium.py, a useful Python module that can help you create and maintain Selenium tests. With make_selenium, you can go back and forth between Selenium tests written in HTML table format and the same tests written in Python.

When they start using Selenium, most people are drawn into using the Selenium IDE, which simplifies considerably the task of writing tests in HTML table format-- especially writing Selenium "action"-type commands such as clicking on links, typing text, selecting drop-down items, submitting forms, etc. However, maintaining the tests in HTML format can be cumbersome. Enter make_selenium, which can turn a Selenium test table into a Python script.

Here's an example of an HTML-based Selenium test that deals with Ajax functionality (see this post for more details on this kind of testing):

TestCommentary
open	/message/20050409174524.GA4854@highenergymagic.org
dblclick	//blockquote
waitForCondition	var value = selenium.getText("//textarea[@name='comment']"); value == ""	10000
store	javascript{Math.round(1000*Math.random())}	var
type	username	user${var}
type	email	user${var}@mos.org
type	comment	hello there from user${var}
click	//form//button[1]
waitForCondition	var value = selenium.getText("//div[@class='commentary-comment commentary-inline']"); value.match(/hello there from user${var}/);	10000
verifyText	//div[@class="commentary-comment commentary-inline"]	regexp:hello there from user${var}
clickAndWait	//div/div[position()="1" and @style="font-size: 80%;"]/a[position()="2" and @href="/search"]
type	q	user${var}
clickAndWait	//input[@type='submit' and @value='search']
verifyValue	q	user${var}
assertTextPresent	Query: user${var}
assertTextPresent	in Re: [socal-piggies] meeting Tues Apr 12th: confirmed
open	/message/20050409174524.GA4854@highenergymagic.org
assertTextPresent	hello there from user${var}
assertTextPresent	delete
click	link=delete
waitForCondition	var allText = selenium.page().bodyText(); var unexpectedText = "hello there from user${var}" allText.indexOf(unexpectedText) == -1;	10000
assertTextNotPresent	hello there from user${var}
assertTextNotPresent	delete
clickAndWait	//div/div[position()="1" and @style="font-size: 80%;"]/a[position()="2" and @href="/search"]
type	q	user${var}
clickAndWait	//input[@type='submit' and @value='search']
verifyValue	q	user${var}
assertTextPresent	Query: user${var}
assertTextPresent	no matches

Let's assume we want to add more assertion commands to this table. Let's turn the TestCommentary.html file into a Python script with make_selenium:

python make_selenium.py -p TestCommentary.html

This command generates a file called TestCommentary.py, with the following contents:

S.open('/message/20050409174524.GA4854@highenergymagic.org')
S.dblclick('//blockquote')
S.waitForCondition('''var value = selenium.getText("//textarea[@name=\'comment\']");
value == ""
''', '10000')
S.pause('2000')
S.store('javascript{Math.round(1000*Math.random())}', 'var')
S.type('username', 'user${var}')
S.type('email', 'user${var}@mos.org')
S.type('comment', 'hello there from user${var}')
S.click('//form//button[1]')
S.waitForCondition('var value = selenium.getText("//div[@class=\'commentary-comment commentary-inline\']"); value.match(/hello there from user${var}/);', '10000')
S.pause('5000')
S.verifyText('//div[@class="commentary-comment commentary-inline"]', 'regexp:hello there from user${var}')
S.clickAndWait('//div/div[position()="1" and @style="font-size: 80%;"]/a[position()="2" and @href="/search"]')
S.type('q', 'user${var}')
S.clickAndWait('//input[@type=\'submit\' and @value=\'search\']')
S.verifyValue('q', 'user${var}')
S.assertTextPresent('Query: user${var}')
S.assertTextPresent('in Re: [socal-piggies] meeting Tues Apr 12th: confirmed')
S.open('/message/20050409174524.GA4854@highenergymagic.org')
S.assertTextPresent('hello there from user${var}')
S.assertTextPresent('delete')
S.click('link=delete')
S.waitForCondition('''
var allText = selenium.page().bodyText();
var unexpectedText = "hello there from user${var}"
allText.indexOf(unexpectedText) == -1;
''', '10000')
S.pause('5000')
S.assertTextNotPresent('hello there from user${var}')
S.assertTextNotPresent('delete')
S.clickAndWait('//div/div[position()="1" and @style="font-size: 80%;"]/a[position()="2" and @href="/search"]')
S.type('q', 'user${var}')
S.clickAndWait('//input[@type=\'submit\' and @value=\'search\']')
S.verifyValue('q', 'user${var}')
S.assertTextPresent('Query: user${var}')
S.assertTextPresent('no matches')

Now we can just add more commands using the special S.command(args) syntax, where command can be any Selenium command, and args are the arguments specific to that command. When we're done, we run make_selenium.py again, this time without the -p switch, in order to generate an HTML file. We can also specify a different name for the target HTML file:

python make_selenium.py TestCommentary.py TestCommentary2.html

But there's more to make_selenium than moving tests back and forth between HTML table format and Python syntax. In fact, Michał's initial goal in writing make_selenium was to offer Python programmers an easy way of creating Selenium tests by writing Python code which would then be translated into HTML.

With make_selenium, you can take full advantage of Python constructs such as for loops when you write your Selenium tests. Here's an example inspired by Michał's documentation. Assume you have a large form with various elements that you need to type in. Then you click a submit button, and you want to make sure that after submitting, the values of the elements are the same. Here's how you would do it in a make_selenium-aware Python script:

def type_values(mapping):
   for key, value in mapping.iteritems():
   S.type(key, value)

def verify_values(mapping):
   for key, value in mapping.iteritems():
   S.assertValue(key, value)

data = {
   'first_name': 'John',
   'last_name': 'Smith',
   'age': '25',
}

type_values(data)
S.clickAndWait('submit_data')
verify_values(data)

The only convention you need to follow is to use the special S object when you want to indicate a Selenium command. The functions type_values and verify_values are normal Python functions which repeatedly call S.type and S.assertValue on all the elements of the data dictionary. If you save this code in a file called test1.py and run make_selenium on it, you get a file called test1.html with the following contents:

Test1
type	first_name	John
type	last_name	Smith
type	age	25
clickAndWait	submit_data
assertValue	first_name	John
assertValue	last_name	Smith
assertValue	age	25

You can then run this file in a TestRunner-based Selenium test suite.

You may ask why do you need make_selenium when Selenium RC is available. I think they both have their place. Selenium RC is your friend if you need the full-blown power of Python in your Selenium scripts. But sometimes, if only for documentation purposes, it's nice to have equivalent HTML-based tests around. On the other hand, HTML is cumbersome to modify and extend. I think that make_selenium bridges the gap between the "scripted Selenium" world and the "HTML TestRunner" world. In fact, it may be possible to write scripts that work in Selenium RC mode, and modify them minimally so that make_selenium understands them and is able to turn them back and forth, to and from HTML. I know Michał was working on this integration, but I haven't had a chance to test it yet with the current version of make_selenium (maybe Michał can leave a comment with a working example ?) Assuming this integration is working, then you can truly say your code is DRY -- you have one Python script expressing a Selenium test which can be run against a Selenium RC server, or turned into HTML for TestRunner consumption.

The current version of make_selenium is 0.9.5. You can download it from here. I encourage you to read the documentation (which BTW is almost entirely automatically generated from docstrings, in an agile fashion :-) Congratulations to Michał for a fine piece of work!