Wednesday, May 09, 2007

Apache virtual hosting with Tomcat and mod_jk

In a previous post I talked about "Configuring Apache 2 and Tomcat 5.5 with mod_jk". I'll revisit some of the topics in there, but within a slightly different scenario.

Let's say you want to configure virtual hosts in Apache, with each virtual host talking to a different Tomcat instance via the mod_jk connector. Each virtual host serves up a separate application via an URL such as http://www.myapp.com. This URL needs to be directly mapped to a Tomcat application. This is a fairly important requirement, because you don't want to go to a URL such as http://www.myapp.com/somedirectory to see your application. This means that your application will need to be running in the ROOT of the Tomcat webapps directory.

You also want Apache to serve up some static content, such as images.

Running multiple instances of Tomcat has a couple of advantages: 1) you can start/stop your Tomcat applications independently of each other, and 2) if a Tomcat instance goes down in flames, it won't take with it the other ones.

Here is a recipe that worked for me. My setup is: CentOS 4.4, Apache 2.2 and Tomcat 5.5, with mod_jk tying Apache and Tomcat together (mod_jk2 has been deprecated).

Scenario: we want www.myapp1.com to go to a Tomcat instance running on port 8080, and www.myapp2.com to go to a Tomcat instance running on port 8081. Apache will serve up www.myapp1.com/images and www.myapp2.com/images.

1) Install Apache and mod_jk. CentOS has the amazingly useful yum utility (similar to apt-get for you Debian/Ubuntu fans), which makes installing packages a snap:

# yum install httpd
# yum install mod_jk-ap20

2) Get the tar.gz for Tomcat 5.5 -- you can download it from the Apache Tomcat download site. The latest 5.5 version as of now is apache-tomcat-5.5.23.tar.gz.

3) Unpack apache-tomcat-5.5.23.tar.gz under /usr/local. Rename apache-tomcat-5.5.23 to tomcat8080. Unpack the tar.gz one more time, rename it to tomcat8081.

4) Change the ports tomcat is listening on for the instance that will run on port 8081.

# cd /usr/local/tomcat8081/conf
- edit server.xml and change following ports:
8005 (shutdown port) -> 8006
8080 (non-SSL HTTP/1.1 connector) -> 8081
8009 (AJP 1.3 connector) -> 8010

There are other ports in server.xml, but I found that just changing the 3 ports above does the trick.

I won't go into the details of getting the 2 Tomcat instances to run. You need to create a tomcat user, make sure you have a Java JDK or JRE installed, etc., etc.

The startup/shutdown scripts for Tomcat are /usr/local/tomcat808X/bin/startup.sh|shutdown.sh.

I will assume that at this point you are able to start up the 2 Tomcat instances. The first one will listen on port 8080 and will have an AJP 1.3 connector (used by mod_jk) listening on port 8009. The second one will listen on port 8081 and will have the AJP 1.3 connector listening on port 8010.

5) Deploy your applications.

Let's say you have war files called app1.war for your first application and app2.war for your second application. As I mentioned in the beginning of this post, your goal is to serve up these applications directly under URLs such as http://www.myapp1.com, as opposed to http://www.myapp1.com/app1. One solution I found for this is to rename app1.war to ROOT.war and put it in /usr/local/tomcat8080/webapps. Same thing with app2.war: rename it to ROOT.war and put it in /usr/local/tomcat8081/webapps.

You may also need to add one line to the Tomcat server.xml file, which is located in /usr/local/tomcat808X/conf. The line in question is the one starting with Context, and you need to add it to the Host section similar to this one in server.xml. I say 'you may also need' because I've seen cases where it worked without it. But better safe than sorry. What the Context element does is it specifies ROOT as the docBase of your Web application (similar if you will to the Apache DocumentRoot directory).

<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true"
xmlValidation="false" xmlNamespaceAware="false">
<Context path="" docBase="ROOT" debug="0"/>
At this point, if you restart the 2 Tomcat instances, you should be able to go to http://www.myapp1.com:8080 and http://www.myapp2.com:8081 and see your 2 Web applications.

6) Create Apache virtual hosts for www.myapp1.com and www.myapp2.com and tie them to the 2 Tomcat instances via mod_jk.

Here is the general mod_jk section in httpd.conf -- note that it needs to be OUTSIDE of the virtual host sections:

#
# Mod_jk settings
#
# Load mod_jk module
LoadModule jk_module modules/mod_jk.so
# Where to find workers.properties
JkWorkersFile conf/workers.properties
# Where to put jk logs
JkLogFile logs/mod_jk.log
# Set the jk log level [debug/error/info]
JkLogLevel emerg
# Select the log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
# JkOptions indicate to send SSL KEY SIZE,
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories
# JkRequestLogFormat set the request format
JkRequestLogFormat "%w %V %T"

Note that the section above has an entry called JkWorkersFile, referring to a file called workers.properties, which I put in /etc/httpd/conf. This file contains information about so-called workers, which correspond to the Tomcat instances we're running on that server. Here are the contents of my workers.properties file:
#
# This file provides minimal jk configuration properties needed to
# connect to Tomcat.
#
# The workers that jk should create and work with
#

workers.tomcat_home=/usr/local/tomcat8080
workers.java_home=/usr/lib/jvm/java
ps=/
worker.list=app1, app2

worker.app1.port=8009
worker.app1.host=localhost
worker.app1.type=ajp13
worker.app1.lbfactor=1

worker.app2.port=8010
worker.app2.host=localhost
worker.app2.type=ajp13
worker.app2.lbfactor=1
The file declares 2 workers that I named app1 and app2. The first worker corresponds to the AJP 1.3 connector running on port 8009 (which is part of the Tomcat instance running on port 8080), and the second worker corresponds to the AJP 1.3 connector running on port 8010 (which is part of the Tomcat instance running on port 8081).

The way Apache ties into Tomcat is that each of the VirtualHost sections configured for www.app1.com and www.app2.com declares a specific worker. Here is the VirtualHost section I have in httpd.conf for www.app1.com:

<VirtualHost *:80>
ServerName www.app1.com
DocumentRoot "/usr/local/tomcat8080/webapps/ROOT"
<Directory "usr/local/tomcat8080/webapps/ROOT">
# Options Indexes FollowSymLinks MultiViews
Options None
AllowOverride None
Order allow,deny
allow from all
</Directory>
ErrorLog logs/app1-error.log
CustomLog logs/app1-access.log combined
# Send ROOT app. to worker named app1
JkMount /* app1
JkUnMount /images/* app1
RewriteEngine On
RewriteRule ^/(images/.+);jsessionid=\w+$ /$1
</VirtualHost>

The 2 important lines as far as the Apache/mod_jk/Tomcat configuration is concerned are:

JkMount /* app1
JkUnMount /images/* app1

The line "JkMount /* app1" tells Apache to send everything to the worker app1, which then ties into the Tomcat instance on port 8080.

The line "JkUnMount /images/* app1" tells Apache to handle everything under /images itself -- which was one of our goals.

At this point, you need to restart Apache, for example via 'sudo service httpd restart'. If everything went well, you should be able to go to http://www.myapp1.com and http://www.myapp2.com and see your 2 Web applications running merrily.

You may have noticed a RewriteRule in each of the 2 VirtualHost sections in httpd.conf. What happens with many Java-based Web application is that when a user first visits a page, the application does not know yet if the user has cookies enabled or not, so the application will use a session ID mechanism fondly known as jsessionid. If the user does have cookies enabled, the application will not use jsessionid the second time a page is loaded. If cookies are not enabled, the application (Tomcat in our example) will continue generating URLs such as

http://www.myapp1.com/images/myimage.gif;jsessionid=0E45D13A0815A172BD1DC1D985793D02

In our example, we told Apache to process all URLs that start with 'images'. But those URLs have already been polluted by Tomcat with jsessionid the very first time they were hit. As a result, Apache was trying to process them, and was failing miserably, so images didn't get displayed the first time a user hit a page. If the user refreshed the page, images would get displayed properly (if the user had cookies enabled).

The solution I found for this issue was to use a RewriteRule that would get rid of the jsessionid in every URL that starts with 'images'. This seemed to do the trick.

That's about it. I hope this helps somebody. It's the result of some very intense googling :-)

If you have comments or questions, please leave them here and I'll try to answer them.

Monday, May 07, 2007

JRuby buzz

I wish I could put "Jython buzz" as the title of my post, but unfortunately I can't seem to detect any Jython buzz anywhere. JRuby though seems to generate a lot of it, judging by this InfoQ article on Mingle, a commercial application based on JRuby and created by ThoughtWorks Studios.

One thing I found very interesting in the InfoQ article was that ThoughtWorks preferred to develop Mingle with JRuby (which is the JVM-based version of Ruby) over writing it on top of Ruby on Rails. They cite ease of deployment as a factor in favor of JRuby:

"In particular, the deployment story for Ruby on Rails applications is still significantly more complex than it should be. This is fine for a hosted application where the deployment platform is in full control of a single company, but Mingle isn't going to be just hosted. Not only is it going to need to scale ‘up’ to the sizes of Twitter (okay, that's wishful thinking and maybe it won't need to scale that much) but it's also going to need to scale ‘down’ to maybe a simple Windows XP machine with just a gig of RAM. On top of that, it's going to be installed by someone who doesn't understand anything about Ruby on Rails deployment and, well, possibly not much about deployment either."

They continue by saying that their large commercial customers wanted to be able to deploy Mingle by dropping a Java .war file under any of the popular Java application servers.

So, for all the talk about Ruby on Rails and the similarly hot Python frameworks, Java and J2EE are far from dead.

Here's wishing that Jython will start generating the same amount of buzz.

Tuesday, May 01, 2007

Michael Dell uses Ubuntu on his home laptop

Found this on Planet Ubuntu, which is buzzing with the news that Dell will be offering laptops preloaded with Feisty Fawn. I still use Edgy on my Inspiron 6000, but I'll probably upgrade to Feisty soon. Or maybe I'll just wait for a Gutsy Gibbon to burst on the scene :-)

Saturday, April 28, 2007

Thursday, April 26, 2007

PyCon07 Testing Tools Tutorial slides up

Spurred by a request from David Brochu, I put the PyCon 07 Testing Tools Tutorial slides up on agilistas.org. Titus's slides are named titus-tutorial-a.pdf through -e.pdf. My slides are only one PDF file, as my portion of the tutorial consisted mainly in Selenium and FitNesse demos. Enjoy!

Mounting local file systems using the 'bind' mount type

Sometimes paths are hardcoded in applications -- let's say you have the path to the Apache DocumentRoot directory hardcoded inside a web application to /home/apache/www.mysite.com. You can't change the code of the web app, but you want to migrate it. You don't want to use the same path on the new server, for reasons of standardization across servers. Let's say you want to set DocumentRoot to /var/www/www.mysite.com.

But /home is NFS-mounted, so that all users can have their home directory kept in one place. One not-so-optimal solution would be to create an apache directory under /home on the NFS server. At that point, you can create a symlink to /var/www/www.mysite.com inside /home/apache. This is suboptimal because the production servers will come to depend on the NFS-mounted directory. You would like to keep things related to your web application local to each server running that application.

A better solution (suggested by my colleague Chris) is to mount a local directory, let's call it /opt/apache_home, as /home/apache. Since the servers are already using automount, this is a question of simply adding this line as the first line in /etc/auto.home:

apache -fstype=bind :/opt/apache_home

/etc/auto.home was already referenced in /etc/auto.master via this line:

/home /etc/auto.home

Note that we're using the neat trick of mounting a local file system via the 'bind' mount type. This can be very handy in situations where symbolic links don't help, because you want to reference a real directory, not a file pointing to a directory. See also this blog post for other details and scenarios where this trick is helpful.

Now all applications that reference /home/apache will actually use /opt/apache_home.

For the specific case of the DocumentRoot scenario above, all we needed to do at this point was to create a symlink inside /opt/apache_home, pointing to the real DocumentRoot of /var/www/www.mysite.com.

Thursday, March 29, 2007

Dell to offer pre-installed Linux on desktops

I knew about the Dell survey which asked people whether they'd like to see Linux pre-installed on Dell desktops and laptops. Looks like more than 70% of the respondents said yes (see this BBC story) -- so Dell is going for it. Now the question is which flavor(s) of Linux will be offered. From what I remember, the survey mentioned Fedora Core, Ubuntu and OpenSuse. Regardless, this is a pretty big win for Linux.

Wednesday, March 28, 2007

OLPC and the Romanian politicians

Interesting blog post from Jani Monoses on how the Romanian parliament rejected the country's participation in the OLPC program. All the arguments centered around cost and lack of applications such as....MS Word! As Jani says -- cluelessness abounds.

Having seen Ivan Krstic's keynote on OLPC at PyCon this year, I realize that the One Laptop Per Child program is mainly about re-introducing kids to their intuitive ways of learning, through play, peer activities and free exploration, as opposed to the centralized, one-to-many teaching method that is used in schools everywhere. The laptop becomes in this case just a tool for facilitating the new ways of learning -- or I should say the old ways, since this is what kids do naturally. But this is one of those disruptive ideas that is hard to grasp by serious grown-up people, especially politicians...

Thursday, March 22, 2007

File sharing with Apache and WebDAV

If you want to share files from a Linux box to Windows clients, Samba is a popular solution. However, it can also be done with Apache and WebDAV. Here is a short HOWTO.

1) Let's say we want to share files in a directory named /usr/share/myfiles. I created a sub-directory called dav in that directory, and then I ran:
# chmod 775 dav
# chgrp apache dav
2) Make sure httpd.conf loads the mod_dav modules:
LoadModule dav_module modules/mod_dav.so
LoadModule dav_fs_module modules/mod_dav_fs.so
3) Create an Apache password file (if you want to use basic authentication) and a user -- let's call it webdav:
# htpasswd -c /etc/httpd/conf/.htpasswd webdav

4) Create a virtual host entry in httpd.conf, similar to this one:
<VirtualHost *>
ServerName share.mydomain.com
DocumentRoot "/usr/share/myfiles"
<Directory "/usr/share/myfiles">
Options Indexes FollowSymLinks MultiViews
AllowOverride AuthConfig
Order allow,deny
allow from all
</Directory>
ErrorLog share-error.log
CustomLog share-access.log combined

DavLockDB /tmp/DavLock

<Location /dav>
Dav On
AuthType Basic
AuthName DAV
AuthUserFile /etc/httpd/conf/.htpasswd
Require valid-user
</Location>
</VirtualHost>

5) Restart httpd, verify that if you go to http://share.mydomain.com/dav you are prompted for a user name and password, and that once you get past the security dialog you can see something like 'Index of /dav'.

Now it's time to configure your Windows client to see the shared WebDAV resource. On the Windows client, either:
  • go to "My network connections" and add a new connection, or
  • go to Windows Explorer->Tools->Map Network Drive, then click on "Signup for online storage or connect to a network server"
Either option will bring up the "Add Network Place Wizard".
  • Click Next, then select "Choose another network location", then click Next.
  • For "Internet or network address", set http://share.mydomain.com/dav. At this point you'll be prompted for a user name/password; specify the ones you defined above.
  • After mapping the resource, you should be able to read/write to it.
CAVEAT

Sometimes the Windows dialog asking for a user name and password will say "connecting to share.mydomain.com" and will keep asking you for the user name/password. The dialog is supposed to show the text you set in AuthName (DAV in my case). If it doesn't, click Cancel, then try again. You can also try to force HTTP basic authentication (as opposed to Windows authentication, which is what Windows tries to do) by specifying http://share.mydomain.com:80/dav as the URL. See also this entry on the WebDAV Wikipedia page.

Resources

Wednesday, March 21, 2007

Ubuntu "command not found" magic

Via Alan Pope's blog: Ubuntu Edgy and above includes a "command not found magic" -- a bash hook that intercepts 'command not found' errors and replaces them with more useful messages, such as what packages you need to install to get that command. I tried it on my Edgy laptop and what do you know, it actually works.

First you need to apt-get the command-not-found package:

$ sudo apt-get install command-not-found
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
command-not-found-data
The following NEW packages will be installed:
command-not-found command-not-found-data
0 upgraded, 2 newly installed, 0 to remove and 15 not upgraded.
Need to get 471kB/475kB of archives.
After unpacking 6263kB of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://us.archive.ubuntu.com edgy/universe command-not-found-data 0.1.0 [471kB]
Fetched 300kB in 2s (109kB/s)
Selecting previously deselected package command-not-found-data.
(Reading database ... 92284 files and directories currently installed.)
Unpacking command-not-found-data (from .../command-not-found-data_0.1.0_i386.deb) ...
Selecting previously deselected package command-not-found.
Unpacking command-not-found (from .../command-not-found_0.1.0_all.deb) ...
Setting up command-not-found-data (0.1.0) ...
Setting up command-not-found (0.1.0) ...

Then you need to open a new shell window, so that the hook gets installed. In that window, try running some commands which are part of packages that you don't yet have installed. For example:

$ nmap
The program 'nmap' is currently not installed, you can install it by typing:
sudo apt-get install nmap

$ snort
The program 'snort' can be found in the following packages:
* snort-pgsql
* snort-mysql
* snort
Try: sudo apt-get install

Pretty cool, huh. And of course the bash hook is written in Python.

Monday, March 19, 2007

Founder of Debian joins Sun

Interesting announcement from Ian Murdoch, the founder of Debian: he's joining Sun as "Chief Operating Platforms Officer" (now that's a mouthful.) I've been really skeptical so far about Sun's embrace of Open Source, but this seems like a major step in the right direction.

CheeseRater - voting in the CheeseShop

Found via Joe Gregorio's blog: CheeseRater, a slick-looking Django app that lets you vote on CheeseShop packages. Interesting idea. Since tagging/redditing/social networking/web2.0ing/etc. are all the rage these days, this might prove more popular than the automatic scoring that Michał is doing with the Cheesecake Service. Maybe we can combine the two....that would be interesting.

Sunday, March 18, 2007

Stone soup as a cure for broken windows

Stones are used to break windows, but in this insightful blog post, Dave Nicolette shows how making stone soup (i.e. getting everybody to contribute a bit of something) can help deal with the broken windows syndrome. Getting everybody to contribute is an art I've been trying to master myself, but like any art, it's far from easy...

Wednesday, March 14, 2007

A few good agile men

Kumar sent me a link to a hilarious blog post: A few good managers. It's a gem. Excerpt:

"Marketing: "Did you cut the automated, edit sync [insert favorite feature here] feature?"

Development: "I did the job I was hired to do."

Marketing: "Did you cut the automated, edit sync feature?"
Development: "I delivered the release on time."

Marketing: "Did you cut the automated, edit sync feature?"
Development: "You're g%$#@*& right I did!""

Thursday, March 08, 2007

Tuesday, February 27, 2007

testing-in-python mailing list

Titus created a mailing list dedicated to topics related to testing in Python. If you're interested, you can subscribe via its mailman interface here. We hope it will become a discussion forum for things such as:
  • testing tools that people have successfully used in their projects
  • testing techniques that help in certain situations (mocking for example)
  • real life scenarios where a specific type of testing (e.g. functional) helped more than another type of testing (e.g. unit)
  • etc. etc.
I think I will affectionately refer to this list as TIP from now on :-)

Update 02/28/07

What do you know, the name TIP struck a chord, so Titus created an alias for it. You can now send email to the list via tip at lists.idyll.org too.

William McVey's PyCon notes as mindmap

Via Elliot Murphy's blog, a very nice mindmap created by William McVey and showing his notes from PyCon07. It demonstrates the power of mindmaps: a lot of information condensed in one page, with links to more detailed information below.

Also from William, other PyCon07 notes.

Monday, February 26, 2007

Testing Tools Panel at PyCon

I'd like to thank the participants in the Testing Tools Panel at PyCon07 for sharing their insights into testing with the audience. Here they are, in alphabetical order of first name:
  • Benji York: zope.testbrowser
  • Brian Dorsey: py.test (representing Holger Krekel)
  • Chad Whitacre: testosterone (created) switched to nose
  • Ian Bicking: paste.test.fixture, minimock, FitLoader
  • Jeff Younker: PyMock
  • Kumar McMillan: fixture - module for loading and referencing test data
  • Martin Taylor: test framework within TI
  • Neal Norwitz: PyChecker
  • Tim Couper: WATSUP (Windows GUI Testing)
  • Titus Brown: twill, scotch, figleaf, pinocchio

Matt Harrison has a very good write-up on the discussions we had during the panel (actually I lifted the list above from his blog post, because he summarized it so well).

One thing that I think all the participants felt, and maybe the audience too, was that 45 minutes was totally not sufficient for this kind of panel. And I know I felt the same thing with the other two panels, for Python-dev and for the Web frameworks. So I'd like to ask people to leave some comments on this post, with ideas about turning these panels into discussions that would last longer, at least 1 hour and even more.

My gut feeling is that there would be a lot of interest in getting framework/library/tool creators together and having a discussion/Q&A with them in front of the audience, with audience participation of course. I'm not sure what the best format would be for this kind of thing -- maybe a round table? But if we get enough ideas, maybe we can fit something like this in next year's PyCon schedule, and allocate it a generous amount of time.

Testability

During the Testing Tools tutorial that Titus and I gave at PyCon, there was a short discussion on testability -- what makes software more testable? I mentioned a list put together by Michael Bolton, and summarized/enhanced by Adam Goucher in this blog post. Recommended reading, both for developers who want to add testing hooks into their software, and for testers who want to know what to ask for from developers so that their life gets easier (and if you're one of the unfortunate souls who have to deal with Java or .NET, this blog post by Roy Osherove talks about testability and pure OOP.)

Although our tutorial was focused on tools and techniques for implementing test automation, we also mentioned that you will never be able to get rid of manual testing. Even though the Google testing team says that 'Life is too short for manual testing' (and I couldn't agree more with them), they hasten to qualify this slogan by adding that automated testing frees you up to do more meaningful exploratory testing.

My experience as a tester shows that the nastiest bugs are often discovered by manual testing. But when you do discover them manually, the best strategy is to write automated tests for them, so that you'll check your application in that particular area from that moment on, via an automated test suite which runs in your continuous integration system.

You do have an automated test suite, right? And it does run periodically (daily or upon on every check-in) in a continuous integration system, right? And you have everything set up so that you're notified by email or RSS feeds when something fails, right? And you fix failures quickly so that everything turns back to green, because you know that too much red, too often, leads to broken windows and bit rot, right?

If you answered No to any of these questions, then you are not testing your application, period (but you already knew this if you took our tutorial -- it was on the last slide :-)

Friday, February 23, 2007

Photos from PyCon panels

Here are photos I took today at the python-dev panel and at the Web frameworks panel. Enjoy!

BTW, here are two very good write-ups on the Web framework panel: one from Matt Harrison, the other from James Bennett.

PyCon day 1

Gave the testing tutorial with Titus yesterday; went pretty well from the feedback we got. We'll publish the slides soon.

Just got out of the first keynote, Ivan Krstic's talk on the "One Laptop Per Child" project. Pretty interesting -- here are some tidbits I remember:
  • OLPC wants to change the way teaching and learning is done these days; they want to go back to the time when preschool kids interacted with each other by playing, and learned naturally peer-to-peer (as opposed to institutionalized teaching, which is one-to-many)
  • contrary to popular opinion, the laptop does not have a hand crank (it would wear down too fast if it had one); however, the laptop can be powered by a pull string that reacts to the puller's strength and powers the device accordingly
  • the 2 rabbit ears are used for wireless; the laptop can speak 802.11s, a new protocol that can be used for fully meshed networking; as soon as one laptop is connected to the internet, all the other ones in its mesh will be connected too
  • the CPU is an AMD Geode at 366 MHz (not 400 or 500, actually 366)
  • no hard drive, uses 512 MB of flash storage
  • OS is a stripped-down version of Fedora
  • runs Python wherever it can (including the init boot daemon); some exceptions are the X.org windowing system, the mDNS daemon, and the bus communication; pretty much all other user-level software, including the file system, is written in Python
  • the laptop has a 'show source' button which obviously shows Python source code that can be edited, etc.
  • no adult has ever been able to open the laptop in less than 2 minutes
  • no child has ever needed more than 30 seconds to open the laptop
  • two fortunate souls got an OLPC XO laptop today: Guido (as the creator of Python), and a guy who was able to recognize a very complicated formula that Ivan showed on a slide (the BBP formula for computing the n-th decimal place of pi); the guy needed approx. 1 minute to open the laptop; Guido's was already open, in a sign of respect I guess
  • OLPC needs good Python developers; if you're interested, check out dev.laptop.org

Saturday, February 17, 2007

Wikipatterns

Via Grady Booch's blog, a site which looks very promising: Wikipatterns. It identifies patterns and anti-patterns for wiki adoption. Here's an excerpt:

"Any grassroots, or bottom-up, strategy is the best place to start since the success of a wiki depends on building active, sustainable participation and this only happens when people see that the software is simple enough to immediately be useful, and meets their needs without requiring them to spend lots of extra time.

A good first step is to identify a group or department who would likely benefit the most from using a wiki, and whose people are open to trying new tools. If you're looking to expand wiki use in another group, look for the thought leader in the group - someone who is very forward thinking, respected by peers, and willing to Champion a new idea and get others around them involved."


Seems like a very good complement to a book I read recently: "Fearless Change: Patterns for Introducing New Ideas" by Mary Lynn Manns and Linda Rising.

Friday, February 16, 2007

Anybody doing LDom on Solaris Sparc?

Lazy Web-type question: has anybody tried out the brand new LDom functionality available in Solaris 10? From some googling around I've done, it seems Sun hasn't yet shipped the full-fledged LDom functionality in Solaris 10 version 11/06.

LDom seems like a cool way to partition some big Solaris Sparc boxes, if you have them. I wonder if the logical domains/virtual machines created with LDom can have Ubuntu installed on top of them (because Ubuntu supports Solaris Sparc).

The Buildbot project has a Trac instance as its home

Brian Warner just announced on the buildbot-devel mailing list that his Buildbot project has a brand new home in the form of a Trac instance. Check it out here. Glad to see the move, as Trac is so much nicer to work with than the clunky SourceForge interface.

Tuesday, February 13, 2007

Ubuntu not to activate proprietary drivers by default

This just in via Jonathan Carter's blog on Planet Ubuntu: the Ubuntu Technical Board has decided that Ubuntu will not activate proprietary drivers by default. Proprietary drivers will be provided for convenience, and users will of course be free to install them if they so choose. This should settle a controversy which has been raging for a while in the Ubuntu community.

Friday, February 09, 2007

Cheesecake Service launched

The Cheesecake Service is a result of Michał Kwiatkowski's hard work during the Google Summer of Code 2006. The Web interface for the service is based on web.py and it has been up and running since last August, but we're making it public now, to coincide with the release of Cheesecake 0.6.1.

The back-end of the Cheesecake Service talks directly to the PyPI repository using the PyPI API in order to find out about packages newly posted to PyPI. Then the service uses Cheesecake and tries to download, install, and score the package. If your package is not there, it might mean you haven't released a new version after August 10, 2006, date from which we started to score packages. Let us know and we can manually score it so that it appears in the list.

Michał just posted a blog entry on Cheesecake and its Service, so please read it and let us know how we can improve on the various things he describes. We are aware that scoring packages is controversial, and we've been called names before because of it, but as Michał also says, the Cheesecake score is meant to be used as a relative number by creators of Python packages who can try to improve it, and not as an absolute ranking among packages. Think of it as an Apgar score for your software.

Kudos to Michał for his continuing hard work on new Cheesecake features and improvements.

Thursday, February 08, 2007

Internal blogs as project tracking tools

I'm reading "The Corporate Blogging Book" by Debbie Weil. She talks about two different types of blogs -- internal (Intranet-type) and external (public access) -- and she mentions how some big name companies use internal blogs for all sorts of purposes: knowledge/information sharing, email replacement, and, the thing that caught my attention, project management. Apparently IBM is big on all these things when it comes to internal blogs.

So I had a mini-revelation: an internal blog is a very good tool for tracking time you spend on various projects. Take the example of a hosting company -- it can set up an internal blog and have categories corresponding to various projects/customers; employees can jot down a summary of what they worked on each day, and put it in the appropriate category. After a while, a timeline of work done on particular projects emerges. Because posts are automatically dated, it's easy to see what you were working on 3 weeks or 3 months ago. And each blog post can contain links to more detailed howtos that are kept in a wiki which serves as a knowledge base. Blogs and wikis make entering information a snap, as opposed to more complicated project management/tracking tools. Blogs and wikis are also searchable, so finding information is easy. To me, this is a lean/agile way of keeping track of your work.

Anyway, maybe this is an obvious use of blogs, but to me it's new, and of course I'm going to implement it :-)

Monday, February 05, 2007

New job

Today was my first day at RIS Technology, a Web hosting/managed services company which hosts grammy.com and missuniverse.com, among other sites. My responsibilities include selecting and helping implement various technologies that make up our offerings to our customers. Of course, if you follow my blog, you know that Python and automated testing are big in my book, so you can be sure that our offerings will include both :-) Stay tuned for more details.

Sunday, January 28, 2007

Connecting to people on LinkedIn

I wasn't such a big LinkedIn fan until a short time ago, when a post by Guy Kawasaki caught my attention. Then synchronicity kicked in and Tennessee Leeuwenburg sent a message to the python-advocacy mailing list, asking Python developers to connect to each other on LinkedIn; here's what Tennessee had to say:

"One way to help spread Python would be to have a strong presence of Python developers in various online networks. One that springs to mind is LinkedIn, a job related social networking site.

If we could encourage Python developers to start adding eachother to their LinkedIn network, then we shoud be able to create a well-connected developer network with business and industry contacts. This should benefit everyone -- both people looking for Python developers, and also people looking for work."

So in the past week or so I started to send LinkedIn invitations to people I know, either by having worked with them, or through the various forums, mailing lists and Open Source communities I have been part of. It's amazing how many people we all know, if we think about it.

LinkedIn has several nice features that can help when you're looking for people to hire, or when you're looking for a job. Perhaps the easiest way to find people is to click on 'Advanced search' (the small link next to the main search box) and type something in the Keywords field. Try it with 'python' for example -- you'll see that a lot of people whose blogs are aggregated on Planet Python have a LinkedIn profile. Your next step, if you are a Python developer yourself, is to send invitations to people you want to connect with. If enough of us Pythonistas do this, our networks will become more and more interconnected, to everybody's advantage. And you can replace 'Pythonistas' with 'agilistas', 'rubyistas' or whatever your interest is.

It's also interesting to see how LinkedIn displays the number of the degrees of separation between yourself and people you are searching for. Amazingly enough, that number is usually 2 or 3, if not 1. This makes me think of Malcolm Gladwell's theory about Connectors in 'The Tipping Point', namely that there is a small number of people that have a LOT of connections. If you are connected to one of these Connectors, then all of a sudden you have a huge number of people in your network, and you can potentially benefit by introducing yourself to them as someone only 2 or 3 degrees of separation apart from them. This is true in my own network, where I am only 2 degrees of separation away from Guy Kawasaki for example. Why? Because a long long time ago I accepted a LinkedIn invitation from one Paul Davis, who has 500+ LinkedIn connections.

If I made you curious about LinkedIn, I'd advise you one more time to read Guy Kawasaki's blog post on how to improve your LinkedIn profile.

Speaking of jobs and hiring, if you are a hardcore Python programmer looking for work, especially in the D.C. area, the Zope Corporation is hiring.

Saturday, January 20, 2007

Pybots updates

Since my last Pybots-related blog post, a lot has happened. We added 2 buildslaves, a Sparc Solaris 10 host running the Django unit test suite (courtesy of Matthew Flanagan) and a G5 OSX host running the SQLAlchemy test suite (courtesy of Skip Montanaro). So now we have 10 buildslaves altogether. I also added more test suites to my Ubuntu Breezy buildslave. In addition to the Cheesecake test suite, I'm now running the unit test suites for the py library, nose, twill and Testoob.

Email notifications finally started to work too, after I finally figured out I was passing the wrong builder names to the MailNotification class. And we also have RSS feeds available. If you want to be notified of failures from all builders, subscribe to:

http://www.python.org/dev/buildbot/community/all/rss or
http://www.python.org/dev/buildbot/community/all/atom

To be notified of failures from the trunk builders, subscribe to:

http://www.python.org/dev/buildbot/community/trunk/rss or
http://www.python.org/dev/buildbot/community/trunk/atom

To be notified of failures from the 2.5 branch builders, subscribe to:

http://www.python.org/dev/buildbot/community/2.5/rss or
http://www.python.org/dev/buildbot/community/2.5/atom

Matthew Flanagan also added functionality that allows you to subscribe to a feed for a particular builder. For example, to subscribe to the feed for the "x86 Ubuntu Breezy trunk" builder, use this URL:

http://www.python.org/dev/buildbot/community/all/rss?show=x86%20Ubuntu%20Breezy%20trunk

If you are interested in contributing a buildslave to the Pybots project, please send a message to the Pybots mailing list, or to me (grig at gheorghiu dot net), or leave a comment here.

Tuesday, January 09, 2007

Steve Rowe on "Letting test drive the process"

Just came across a blog post by Microsoft' Steve Rowe called "Letting test drive the process". Steve quotes an article by Richard Collins -- "Test, test and test again" -- and adds his own observations on the practices of involving testers early in the development process, and of building testable interfaces into the product instead of heavy UIs.

According to Steve Rowe, Microsoft's development and testing process follows these recommended practices. I quote Steve:

"Also, at Microsoft, testing begins from day one. Every product I've ever been involved with at Microsoft has had daily builds from very early on. Every product has also had what we call BVTs (build verfication tests) that are run every day right after the build completes. If any of their tests fail, the product is held until they can be fixed."

Hmmm...I would expect Microsoft to have less problems with their products in this case. But I think a couple of problems that plague Microsoft in particular are backwards compatibility and the sheer amount of hardware/OS/service pack combinations that they need to test.

Speaking of Microsoft and testing, I found The Braidy Tester's blog very informative.

Monday, January 08, 2007

Testing tutorial at PyCon07

Titus and I will present a tutorial on "Testing Tools in Python" at PyCon07. It is scheduled for the afternoon session on Thursday Feb.22. It will be an improved version of our "Agile Development and Testing in Python" tutorial from last year.

Here is the tutorial outline (courtesy of Titus). If you have any suggestions, please leave a comment.

Introduction
* Why test?
* What to test?
* Using testing to boost maintainability of code.

Setting up a project
* Source control management with Subversion.
* A brief introduction to using Trac for project
documentation and ticket management.
* Packaging with distutils
* Packaging with setuptools
* Registering your project with the Python Cheeseshop
* What else is out there? (distributed vs svn, roundup, ...)

Unit testing
* How to think about unit testing
* Using nose to run unit tests
* doctest-style unit tests
* What else is out there? (unittest, py.test, testosterone...)

Functional Web testing with twill
* Writing twill scripts
* Running twill scripts
* Using scotch to record actions
* Using wsgi_intercept to avoid network sockets
* What else is out there? (zope.testbrowser, mechanize, mechanoid)

Using code coverage in conjunction with unit/functional testing
* Basic code coverage with figleaf
* Monitoring code coverage in remote servers
* Combining figleaf code coverage analyses
* What else is out there (coverage)

** BREAK **

Acceptance testing with FitNesse/PyFit
* How FitNesse works
* Writing fixtures
* Running Python fixtures

Web application testing with Selenium
* How Selenium works
* Writing and recording Selenium tests
* Scripting Selenium tests remotely with SeleniumRC
* What else is out there? (Sahi, Watir)

Continuous integration with buildbot
* Introduction to buildbot
* Discussion of concepts, demonstration.
* Integrating tests into buildbot.
* GUI testing in buildbot.
* Using pybots to test your open source project

Conclusion
* Why test, revisited
* Maintainability and testing

Testing Tools Panel at PyCon07: questions needed

I will be moderating the Testing Tools Panel at PyCon07, currently scheduled from 11:40 AM to 12:25 PM on Sat. Feb. 24th (and immediately followed by Guido's keynote). I put together a Wiki page for the panel, with questions and topics that I thought would be interesting for the audience (and with some input from Ian Bicking.)

I'd be very grateful if people who plan on attending the panel could add more questions or topics of interest either by directly editing the Wiki page, or by leaving a comment here, or by sending me an email at grig at gheorghiu.net. Thanks in advance!

Sunday, January 07, 2007

New Year's resolution

Here's one New Year resolution I'm trying to keep: each day, read the corresponding page for that date from "The Daily Drucker". I blogged about this book before, and I continue to be amazed at the insight and wisdom that Drucker manages to pack in almost every sentence he writes. Although Drucker writes about general practices of management and leadership, many of his ideas can be easily applied to software development in general, and testing in particular.

Here are some fragments from January 4th, on "Organizational inertia", which can be applied just as well to any software project ("bitrot" and "goldplating" come to mind):

"All organizations need to know that virtually no program or activity will perform effectively for a long time without modifications and redesign. Eventually every activity becomes obsolete."

"Businessmen are just as sentimental about yesterday as bureaucrats. They are just as likely to respond to the failure of a product or program by doubling the efforts invested in it. But they are, fortunately, unable to indulge freely in their predilections. They stand under an objective discipline, the discipline of the market. They have an objective outside measurement, profitability.And so they are forced to slough off the unsuccessful and unproductive sooner or later."

And how do you measure the efficiency of an organization? By testing, testing, testing:

"All organizations must be capable of change. We need concepts and measurements that give to other kinds of organizations what the market test and profitability yardstick give to business. Those tests and yardsticks will be quite different."

Friday, January 05, 2007

Cheesecake now including PEP8 checks

The inclusion actually happened a couple of weeks ago. I saw Johann Rocholl's message on conp.lang.python.announce where he talked about his pep8.py module -- a tool which checks Python modules against some of the style conventions in PEP8.

Here's a sample output of running pep8.py against one of the modules in the Cheesecake project. By default, pep8 reports only the first occurrence of the error or warning. The numbers after the file name represent the line and column where the error/warning occurred:

$ python pep8.py logger.py
logger.py:1:11: E401 multiple imports on one line
logger.py:7:23: W291 trailing whitespace
logger.py:8:5: E301 expected 1 blank line, found 0
logger.py:40:33: W602 deprecated form of raising exception
logger.py:60:1: E302 expected 2 blank lines, found 1
logger.py:114:80: E501 line too long (85 characters)
If you want to see all occurrences, use the --repeat flag.

If you just want to see how many lines in a given file have PEP8-related errors/warnings, use the --statistics flag, along with -qq, which quiets the default output:

$ python pep8.py logger.py --statistics -qq
3 E301 expected 1 blank line, found 0
4 E302 expected 2 blank lines, found 1
1 E401 multiple imports on one line
1 E501 line too long (85 characters)
40 W291 trailing whitespace
1 W602 deprecated form of raising exception
You can also pass multiple file and directory names to pep8.py, and it will give you an overall line count when you use the --statistics flag.

So now cheesecake_index.py includes a check for PEP8 compatibility as part of the 'code kwalitee' index. To compute the PEP8 score, it only looks at types of errors and warnings, not at the line count for each type. It subtracts 1 from the code kwalitee score for each warning type reported by pep8, and 2 for each error type reported. Johann told me he'll try to come up with a scoring scheme within the pep8 module, so when that's ready I'll just use it instead of my ad-hoc one. Kudos to Johann for creating a very useful module.

Tuesday, January 02, 2007

Martin Fowler's "Mocks Aren't Stubs" -- updated version

As the title says, Martin Fowler just announced a significant update -- indeed, a rewrite -- of his classic "Mocks Aren't Stubs". Most of the terminology he uses in the new version of the article is borrowed from Gerard Meszaros' xUnit Patterns jargon. Highly recommended. I'm not so vain as to believe that my post on mock testing influenced Martin's rewrite, so I'm just going to invoke synchronicity here :-)

Friday, December 15, 2006

Mock testing examples and resources

Mock testing is a very controversial topic in the area of unit testing. Some people swear by it, others swear at it. As always, the truth is somewhere in the middle. But first of all, let's ask Wikipedia about mock objects. Here's what it says:

"Mock objects are simulated objects that mimic the behavior of real objects in controlled ways. A computer programmer typically creates a mock object to test the behavior of some other object, in much the same way that an automobile designer uses a crash test dummy to test the behavior of an automobile during an accident."

This is interesting, because it talks about accidents, which in software development speak would be errors and exceptions. And indeed, I think one of the main uses of mock objects is to simulate errors and exceptions that would otherwise be very hard to reproduce.

Let's get some terminology clarified: when people say they use mock objects in their testing, in most cases they actually mean stubs, not mocks. The difference is expanded upon with his usual brilliance by Martin Fowler in his article "Mocks aren't stubs". I'll let you read that article and draw your own conclusions. Here are some of mine: stubs are used to return canned data to your methods or functions under test, so that you can make some assertions on how your program reacts to that data (here, I use "program" as shorthand for "method or function under test", not for executable or binary.) Mocks, on the other hand, are used to specify certain expectations about how the methods of the mocked object are called by your program: how many times, with how many arguments, etc.

In my experience, stubs are more useful than mocks when it comes to unit testing. You should still use a mock library or framework even when you want to use stubs, because these libraries make it very easy to instantiate and work with stubs -- as we'll see in some of the examples I'll present.

I said that mock testing is a controversial topic. If you care to follow the exchange of comments I had with Bruce Leggett on this topic, you'll see that his objections to mocking are very valid. His main point is that if you mock an object and the interface or behavior of that object changes, your unit tests which use the mock will pass happily, when in fact your application will fail.

I thought some more about Bruce's objections, and I think I can come up with a better rule of thumb now than I could when I replied to him. Here it is: use mocking at the I/O boundaries of your application and mock the interactions of your application with external resources that are not always under your control.

When I say "I/O boundaries", I mean mostly databases and network resources such as Web servers, XML-RPC servers, etc. The data that these resources produce is consumed by your application, and it often contains some randomness that makes it very hard for your unit tests to assert things about it. In this case, you can use a stub instead of the real external resource and you can return canned data from the stub. This gives you some control over the data that is consumed by your program and allows you to make more meaningful assertions about how your program reacts to that data.

These external resources are also often unreachable due to various error conditions which again are not always under your control, and which are usually hard to reproduce. In this case, you can mock the external resource and simulate any errors or exceptions you want, and see how your program reacts to them in your unit tests. This relates to the "crash test dummy" concept from the Wikipedia article.

In most cases, the external resources that your application needs are accessed via stable 3rd party libraries or APIs whose interfaces change rarely. For example, in Python you can use standard library modules such as urllib or xmlrpclib to interact with Web servers or XML-RPC servers, or 3rd party modules such as cxOracle or MySQLdb to interact with various databases. These modules, either part of the Python stdlib or 3rd party, have well defined interfaces that rarely if ever change. So you have a fairly high degree of confidence that their behavior won't change under you at short notice, and this makes them good candidates for mocking.

I agree with Bruce that you shouldn't go overboard with mocking objects that you create in your own application. There's a good chance the behavior/interface of those objects will change, and you'll have the situation where the unit tests which use mock versions of these objects will pass, when in fact the application as a whole will fail. This is also a good example of why unit tests are not sufficient; you need to exercise your application as a whole via functional/integration/system testing (here's a good concrete example why). In fact, even the most enthusiastic proponents of mock testing do not fail to mention the need for testing at higher levels than unit testing.

Enough theory, let's see some examples. All of them use Dave Kirby's python-mock module. There are many other mock libraries and modules for Python, with the newest addition being Ian Bicking's minimock module, which you should definitely check out if you use doctest in your unit tests.

The first example is courtesy of Michał, who recently added some mock testing to the Cheesecake unit tests. This is how cheesecake_index.py uses urllib.urlretrieve to retrieve a package in order to investigate it:

try:
downloaded_filename, headers = urlretrieve(self.url, self.sandbox_pkg_file)
except IOError, e:
self.log.error("Error downloading package %s from URL %s" % (self.package, self.url))
self.raise_exception(str(e))
if headers.gettype() in ["text/html"]:
f = open(downloaded_filename)
if re.search("404 Not Found", "".join(f.readlines())):
f.close()
self.raise_exception("Got '404 Not Found' error while trying to download package ... exiting")
f.close()

To test this functionality, we used to have a unit test that actually grabbed a tar.gz file from a Web server. This was obviously sub-optimal, because it required the Web server to be up and running, and it couldn't reproduce certain errors/exceptions to see if we handle them correctly in our code. Michał wrote a mocked version of urlretrieve:

def mocked_urlretrieve(url, filename):
if url in VALID_URLS:
shutil.copy(os.path.join(DATA_PATH, "nose-0.8.3.tar.gz"), filename)
headers = Mock({'gettype': 'application/x-gzip'})
elif url == 'connection_refused':
raise IOError("[Errno socket error] (111, 'Connection refused')")
else:
response_content = '''
HTML_INCLUDING_404_NOT_FOUND_ERROR
''''
dump_str_to_file(response_content, filename)
headers = Mock({'gettype': 'text/html'})

return filename, headers
(see the _helper_cheesecake.py module for the exact HTML string returned, since Blogger refuses to include it because of its tags)

The Mock class from python-mock is used here to instantiate and mock the headers object returned by urlretrieve. When you do:
headers = Mock({'gettype': 'text/html'})
you get an object which has all its methods stubbed out and returning None, with the exception of the one method you specified, gettype, which in this case will return the string 'text/html'.

This is the big advantage of using a library such as python-mock: you don't have to manually stub out all the methods of the object you want to mock; instead, you simply instantiate that object via the Mock class, and let the library handle everything for you. If you don't specify anything in the Mock constructor, all the methods of the mocked object will return None. In our case, since cheesecake_index.py only calls header.gettype(), we were only interested in this method, so we specified it in the dictionary passed to the Mock class, along with its return value.

The mocked_urlretrieve function inspects its first argument, url, and, based on its value, either copies a tar.gz file into a target location (indicated by filename) for further inspection, or raises an IOError exception, or returns an HTML document with a '404 Not Found' error. This illustrates the usefulness of mocking: it avoids going to an external resource (a Web server in this case) to retrieve a file, and instead it copies it from the file system to another location on the file system; it simulates an exception that would otherwise be hard to reproduce consistently; and it returns an error which also would be hard to reproduce. Now all that remains is to exercise this mocking functionality in some unit tests, and this is exactly what test_index_url_download.py does, by exercising 3 test cases: valid URL, invalid URL (404 error) and unreachable server. Just to exemplify, here's how the "Connection refused" exception is tested:

try:
self.cheesecake = Cheesecake(url='connection_refused',
sandbox=default_temp_directory, logfile=logfile)
assert False, "Should throw a CheesecakeError."
except CheesecakeError, e:
print str(e)
msg = "Error: [Errno socket error] (111, 'Connection refused')\n"
msg += "Detailed info available in log file %s" % logfile
assert str(e) == msg

You might have a question at this point: how did we make our application aware of the mocked version of urlretrieve? In Java, where the mock object techniques originated, this is usually done by what is called "dependency injection". This simply means that the mocked object is passed to the object under test (OUT) either via the OUT's constructor, or via a setter method of the OUT's. In Python, this is absolutely unnecessary, because of one honking great idea called namespaces. Here's how Michał did it:
import cheesecake.cheesecake_index as cheesecake_index
from _helper_cheesecake import mocked_urlretrieve
cheesecake_index.urlretrieve = mocked_urlretrieve
What happens here is that the urlretrieve name used inside the cheesecake_index module is simply reassigned and pointed to the mocked_urlretrieve function. Very simple and elegant. This way, the OUT, in our case the cheesecake_index module, is completely unchanged and blissfully unaware of any mocked version of urlretrieve. It is only in the unit tests that we reassign urlretrieve to its mocked version. Further proof, if you needed one, of Python's vast superiority over Java :-)

The second example is courtesy of Karen Mishler from ARINC. She used the python-mock module to mock an interaction with an external XML-RPC server that produces avionics data. In this case, the module that gets mocked is xmlrpclib (I changed around some names of servers and methods and I got rid of some information which is not important for this example):

fakeResults = {
"Request":('|returncode|0|/returncode|',
'|machineid|fakeServer:81:4080|/machineid|'),
"Results":('|returncode|0|/returncode|',
'|origin|ABC|/origin|\n|destination|DEF|/destination|\n'),
}
mockServer = Mock(fakeResults)
xmlrpclib = Mock({"Server":mockServer})

(I replaced the XML tag brackets with | because Blogger had issues with the tags....Beta software indeed)

Karen mocked the Server object used by xmlrpclib to return a handle to the XML-RPC server. When the application calls xmlrpclib.Server, it will get back the mockServer object. When the application then calls the Request or Results methods on this object, it will get back the canned data specified in the fakeResults dictionary. This completely avoids the network traffic to and from the real XML-RPC server, and allows the application to consume specific data about which the unit tests can make more meaningful assertions.

The third example doesn't use mocking per se, but instead illustrates a pattern sometimes called "Fake Object"; that is, replacing an object that your application depends on with a more lightweight and faster version to be used during testing. A good example is using an in-memory database instead of a file system-based database. This is usually done to speed up the unit tests and thus have more frequent continuous integration runs.

The MailOnnaStick application that Titus and I presented at our PyCon06 tutorial uses Durus as the back-end for storing mail message indexes. In the normal functionality of the application, we store the data on the file system using the FileStorage functionality in Durus (see the db.py module). However, Durus also provides MemoryStorage, which we decided to use for our unit tests via the mockdb.py module. In this case, mockdb is actually a misnomer, since we're not actually mocking or stubbing out methods of the FileStorage version, but instead we're reimplementing that functionality using the faster MemoryStorage. You can see how we use mockdb in our unit tests by looking at the test_index.py unit test module. Python namespaces come to the rescue again, since we don't have to make index.py, the consumer of the database functionality, aware of any mocking-related changes, except inside the unit test. In the test_index.py unit test, we reassign the index.db name to mockdb:
from mos import index, mockdb
index.db = mockdb
Speaking of patterns, I found very thorough explanations of unit testing patterns at the xUnit Patterns Web site. Sometimes the explanations are too thorough, if I may say so -- too much hair splitting going on -- but overall it's a good resource if you're interested in the more subtle nuances of Test Stubs, Test Doubles, Mock Objects, Test Spies, etc.

Mock testing is being used pretty heavily in Behavior-Driven Development (BDD), which I keep hearing about lately. I haven't looked too much into BDD so far, but from the little I've read about it, it seems to me that it's "just" syntactic sugar on top of the normal Test-Driven Development process. They do emphasize good naming for the unit tests, which if done to the letter turns the list of unit tests into a specification for the behavior of the application under test (hence the B in BDD). I think this can be achieved by properly naming your unit test, without necessarily resorting to tools such as RSpec. But I may be wrong, and maybe BDD is a pretty radical departure from TDD -- I don't know yet. It's worth checking it out in any case.

I'll finish by listing some Web sites and articles related to mock testing. Enjoy!

Mind maps and testing

Jonathan Kohl, whose blog posts are always very insightful, writes about using mind maps to visualize software testing mnemonics (FCC CUTS VIDS; each letter represents an area of functionality within a product where testing efforts can be applied.) He finds that a mind map goes beyond the linearity of a list of mnemonics and gives testers a home base from which they can venture out into the product and explore/test new areas. Jonathan's findings match my experiences in using mind maps.

Thursday, December 14, 2006

"The Problem with JUnit" article

Simon Peter Chappell posted a blog entry on "The Problem with JUnit". The title is a bit misleading, since Simon doesn't really have a problem with JUnit per se. His concern is that this tool/framework is so ubiquitous in the Java world, that people new to unit testing think that by simply using it, they're done, they're "agile", they're practicing TDD.

Simon's point is that JUnit is just a tool, and as such it cannot magically make you write good unit tests. This matches my experience: writing unit tests is hard. It's less important what tool or framework you use; what matters is that you cover as many scenarios as possible in your unit tests. What's more, unit tests are definitely necessary, but also definitely not sufficient for a sound testing strategy. You also need comprehensive automated functional and integration tests, and even (gasp) GUI tests. Just keep in mind Jason Huggins's FDA-approved testing pyramid.

Simon talks about how JUnit beginners are comfortable with "happy path" scenarios, but are often clueless about testing exceptions and other "sad path" conditions. This might partly be due to the different mindset that developers and testers have. When you write tests, you need to put your tester hat on and try breaking your software, as well as making sure it does what it's supposed to do.

In the Python testing world, we are fortunate to have a multitude of unit test tools, from the standard library unittest and doctest to tools and frameworks such as py.test, nose, Testoob, testosterone, and many others (see the Unit Testing Tools section of the PTTT for more details). There is no tool that rules them all, such as JUnit in the Java world, and I think this is a good thing, since it allows people to look at different ways to write their unit tests, each with their own strengths and weaknesses. But tools are not enough, as Simon points out, and what we need are more articles/tutorials/howtos on techniques and strategies for writing good tests, be they unit, functional, etc. I'm personally looking forward to read Roy Osherove's book "The Art of Unit Testing" when it will be ready. You may also be interested in some of my articles on testing and other topics. And the MailOnnaStick tutorial wiki might give you some ideas too.

Switched to Blogger Beta

I apologize if your RSS feed reader is suddenly swamped with posts from my blog. It's hopefully a one-time thing due to my having switched my blog to Blogger Beta.

Wednesday, December 13, 2006

Hungry for cheesecake?

If you are, search for "cheesecake" using Google Code Search. If you do, you'll get a unit test from the Cheesecake project as the very first result. Clearly, Google have their act together! :-)

Tuesday, December 05, 2006

"Scrum and XP From the Trenches" report

This just in via the InfoQ blog: a report (PDF) written by Henrik Kniberg with the intriguing title "Scrum and XP From the Trenches". Haven't read all of it yet, but the quote from the report included at the end of the InfoQ blog post caught my attention:

"I've probably given you the impression that we have testers in all Scrum teams, that we have a huge acceptance test team for each product, that we release after each sprint, etc., etc. Well, we don't. We've sometimes managed to do this stuff, and we've seen that it works when we do. But we are still far from an acceptable quality assurance process, and we still have a lot to learn there."

Testing is hard. But testing can also be fun!

Friday, December 01, 2006

"Performance Testing with JUnitPerf" article

Andrew Glover, who has been publishing a series of articles related to code quality on IBM developerWorks, talks about "Peformance Testing with JUnitPerf". The idea is to decorate your unit tests with timing constraints, so that they also become performance tests. If you want to do the same in Python, I happen to know about pyUnitPerf, the Python port of JUnitPerf. Here is a blog post/tutorial I wrote a while ago on pyUnitPerf.

PyCon news

I was very glad to see that the 3 proposals I submitted to PyCon07 were accepted: a "Testing Tools in Python" tutorial presented jointly with Titus, a "Testing Tools Panel" that I will moderate, and a talk on the Pybots project. The complete list of accepted talks and panels is here.

Here are the brief description and the outline for the Testing Tools tutorial that Titus and I will present. We will cover much more than just testing tools actually -- we'll talk about test and development techniques and strategies. It should be as good or better than the one we gave last year, which attracted a lot of people.

The Testing Tools Panel has a Wiki page. If you're interested in attending, please consider adding questions or topics of interest to you. If there is enough interest, I'm thinking about also organizing a BoF session on Testing Tools and Techniques, since the panel's duration will be only 45 minutes.

Finally, my Pybots talk will consist of an overview of the Pybots project: I will talk about the setup of the Pybots buildbot farm, about the issues that the Pybots farm has helped uncover,
and also about lessons learned in building, sustaining and growing an open-source community project.

The program for PyCon07 looks very solid, with a lot of interesting talks and tutorials. I'm very much looking forward to the 4 days I'll spend in beautiful Addison, TX :-)

Tuesday, November 21, 2006

Good Unix-related blog

Vladimir Melnikoff brought his blog to my attention: "Nothing but Unix". Good resource for Unix enthusiasts, mostly composed of industry-related news.

Python Fuzz Testing Tools

Ian Bicking suggested I create a new category in the Python Testing Tools Taxonomy: Fuzz Testing or Fuzzing. Done. If you're not familiar with the term, see the Wikipedia article which talks about this type of testing. Here's an excerpt: "The basic idea is to attach the inputs of a program to a source of random data ("fuzz"). If the program fails (for example, by crashing, or by failing built-in code assertions), then there are defects to correct. The great advantage of fuzz testing is that the test design is extremely simple, and free of preconceptions about system behavior."

Ian told me about the Peach Fuzzer Framework. I was familiar with Pester (the home page talks about a Java tool called Jester, and it has links to the Python version called Pester); I also googled some more and found other Python fuzzing tools such as antiparser and Taof, which are both geared towards fuzzing network protocols. In fact, many fuzzing tools are used in security testing because they can aid in attacking software via random inputs. See this Hacksafe article on "Fuzz testing tools and techniques" and this PacketStorm list of fuzzing tools. Another good overview is Elliotte Harold's developerWorks article on fuzz testing. Very interesting stuff. If the "Python Testing Tools" tutorial Titus and I proposed for PyCon gets accepted, expect to see some fuzz testing included in our arsenal :-)

I also added Ian's minimock tool to the PTTT page. Very cool minimal approach to mock testing, achieved by embedding mocking constructs in doctests.

In other testing-related blog posts, Titus talks about the difficulty of retrofitting testing to an existing application (even when you wrote the testing tools!), and Max Ischenko presents some uber-cool plugins which integrate nose into vim.

Thursday, November 02, 2006

"Swap space management" article at IBM developerWorks

From IBM developerWorks, a very nice summary of the issues involved in setting up and maintaining swap space on *nix systems: "Swap space management and tricks".

Wednesday, November 01, 2006

Daniel Read on software and Apgar scores

Daniel Read blogs on the topic: "Does software need an Apgar score?". He mentions the fact that a simple metric (the Apgar score for newborns) revolutionized the childbirth process, "through standardization of techniques, training, and regulation of who exactly was allowed to perform certain procedures (based on whether they had the training and experience)". He then talks about how a similar simple score might help the quality of software development, by assessing its "health". Hmmm... all this sounds strangely familiar to me -- Cheesecake anybody? Of course, Daniel accepts that this idea is highly controversial and maybe a bit simplistic. However, I for one am convinced that it would help with improving, if not the quality, then at least the kwalitee of the software packages we see in the wild today.

Thursday, October 26, 2006

Got Edge?

I mean Edgy. I mean Edgy Eft. Get it.

Update

I followed the EdgyUpgrades document from the Ubuntu Wiki and all it took to upgrade my Dell laptop from Dapper to Edgy was one command:

gksu "update-manager -c" 

I call this painless. Everything seems to be working just fine after the upgrade. Haven't had time to play with it at all -- in fact, I don't even know how long the upgrade process took, since I left after I started it.

Proposal for Testing Tools Panel at PyCon07

Following Titus's example with his Web Frameworks Panel proposal for PyCon07, I proposed a Testing Tools Panel. And yes, I expect the author of twill to participate and take questions :-)

I created a TestingToolsPanel page on the PyCon07 wiki. Please feel free to add your own testing-related topics of interest and/or questions for the authors. If you are a testing tool author, please consider participating in the panel. You can leave a comment here or send me an email (grig at gheorghiu.net) and let me know if you're interested in participating.

Here's what I have so far on the Wiki page:

I maintain a "Python Testing Tools Taxonomy" (PTTT) Wiki page.

Here are some of the tools listed on the PTTT page:
  • unit testing tools (unittest/doctest/py.test/nose/Testoob/testosterone)
  • mock/stub testing tools (python-mock/pmock/stubble)
  • Web testing tools (twill/webunit/zope.testbrowser/Pamie/paste.test.fixture)
  • acceptance testing tools (PyFit/texttest/FitLoader)
  • GUI testing tools (pywinauto/WATSUP/winGuiAuto/guitest)
  • source code checking tools (pylint/pychecker/pyflakes)
  • code coverage tools (coverage/figleaf/Pester)
  • other miscellaneous testing tools (pysizer/pymetrics/svnmock/testtools)

I propose to have a panel where authors of some of these tools would discuss and take questions on topics such as:
  • what need prompted the creation of the testing tool
  • what type of testing does the tool belong to (unit, functional, acceptance, system, performance)
  • what specific features does the tool offer that other tools lack
  • what are the most common testing scenarios you have seen in your user base
  • are there any platform- or OS-specific gotchas related to the tool
  • how extensible is the tool (plugins etc.)
  • how easy to learn is the tool
  • how well tested is the tool
  • how well documented is the tool

Thursday, October 19, 2006

"Agile in action" photostream

From a blog I read with pleasure, Simon Baker's "Agile in action", here's a link to a Flickr photostream that shows in my opinion what agile is all about: collaboration, camaraderie, storytelling....in short, having great fun and producing great software in the process. Stevey, I can tell you've never been part of an agile team in your life -- otherwise why would you be so bitter and cranky about it?

Monday, October 09, 2006

The 90-9-1 rule and building an open source community

Jakob Nielsen talks about the 90-9-1 rule in his latest Alertbox newsletter: "Participation inequality: encouraging more users to contribute". Simply put, the rule states that in a typical online community, 90% of the users are lurkers, 9% are occasional contributors, and only 1% are active contributors. This should be interesting for people trying to build and grow open source projects. Nielsen has some suggestions to offer on how to overcome this "participation inequality". Read the article for his suggestions.

Here are some of my own observations and lessons learned from various open source efforts I've been part of (many of them are things I've tried to do on the Pybots project):

How to build an open source community

* Blog, blog, blog
* Send messages to mailing lists related to the area of your project
* Write extensive documentation, make it easy for people to join
* Create a project repository (Google Code)
* Get help from early adopters, involve them in the project

How to sustain and grow an open source community

* Blog, blog, blog
* Send messages to individuals who might be interested in contributing
* Acknowledge contributions
* Respond quickly to issues on mailing list
* Demonstrate usefulness of the project, both to contributors, and to any organizations involved (e.g. the PSF)
* Market/promote/evangelize the project tirelessly
* Recommended reading: "Fearless change: Patterns for introducing new ideas" by Mary Lynn Manns and Linda Rising

Comments about your own experience in building an open source community are much appreciated.

Thursday, October 05, 2006

Let's celebrate Roundup by turning it into the official Python bug tracker

Richard Jones just posted a note about Roundup turning 5. What better birthday gift than turning it into the official Python bug/issue tracker. Readers of Planet Python certainly know by now that there are 2 issue trackers in contention: JIRA and Roundup. Unless people step up to volunteer as admins for maintaining a Roundup-based Python issue tracker, the PSF will choose JIRA. I walked the walk and volunteered myself. I know there must be other people out there who would like to see a Python-based project be selected over a Java project. Any takers? Send an email by Oct. 16th to infrastructure at python.org stating your intention to volunteer. All it takes is 8 to 10 people.

Pybots news

I'm happy to report that the Pybots project continues to gain momentum. In raw numbers, we have 8 buildslaves running the automated tests for 17 projects, and also testing the installation of 18 other packages. Pretty impressive, if I may say so myself. This table is copied from the main pybots.org page and shows the current setup:

Builder name Project(s) tested Pre-requisites installed Owner
x86 Red Hat 9 Twisted setuptools, zope.interface, pycrypto, pyOpenSSL Grig Gheorghiu
x86 Debian Unstable docutils, roundup N/A Seo Sanghyeon
x86 Ubuntu Dapper parsedatetime setuptools Mike Taylor (Bear)
x86 OSX vobject, zanshin setuptools, zope.interface, Twisted Mike Taylor (Bear)
x86 Gentoo pysqlite, Genshi, Trac, feedvalidator clearsilver, pysqlite Manuzhai
amd64 Ubuntu Dapper MySQLdb, CherryPy N/A Elliot Murphy
x86 Windows 2003 lxml (dev, stable) Bazaar (dev, stable) libxml2, libxslt, zlib, iconv Sidnei da Silva
x86 Ubuntu Breezy Cheesecake setuptools, nose, logilab-astng, pylint Grig Gheorghiu

Some more projects and buildslaves are in the pipeline, so I hope to be able to announce them soon. I'd like to thank all the contributors so far, in chronological order of their contributions: Seo Sanghyeon, Mike Taylor (Bear), Manuzhai, Elliot Murphy and Sidnei da Silva.

People interested in this project -- whether they'd like their project to be tested on an existing buildslave, or they'd like to contribute a buildslave -- are encouraged to peruse the documentation on the Pybots page, then send a message to the Pybots mailing list.

Friday, September 22, 2006

Notepad++ rocks

I heard about Notepad++ from Michael Carter, who showcased SQLAlchemy at our last SoCal Piggies meeting and used Notepad++ to edit his Python files. I downloaded it this week and all I can say is that it rocks! For Windows users, it's one of the best editors I've ever seen, and of course it's completely free. Notepad++ is based on the Scintilla editor, and it does syntax coloring for a gazillion languages (Python included of course), it includes a huge number of plugins, such as a Hex Editor, etc., etc. Highly recommended!

Tuesday, September 19, 2006

Buildbot used for continuous integration in the Gnome project

José Dapena Paz has a nice write-up on how the Gnome Build Brigade is using buildbot for continuous integration of all the projects under the Gnome umbrella. Buildbot scores again!

Monday, September 18, 2006

Any projects that need Pybots buildslaves?

Know of any projects that need Pybots buildslaves?

This is a question that's been asked twice already on the Pybots mailing list, or in emails addressed directly to me. Every time I answered that I don't really know any, and I advised the people asking the question to go through the list of their favorite Python projects, and pick some that they'd like to test. Or go through the list of major Web frameworks and pick some.

Anyway -- maybe a better way is to ask all the readers of this blog: do you have a project that you'd like to test in the Pybots buildbot farm, but maybe you don't have the hardware to run the buildslave on? Then let me know, or even better, let the Pybots mailing list know. There are people such as Jeff McNiel and Skip Montanaro who have hardware and resources, and are looking for projects to test.

BTW, the Pybots farm now has 6 buildslaves (the 6th was courtesy of Elliot Murphy, who contributed an AMD-64 Ubuntu Dapper box running the MySQLdb tests), with 3 or 4 buildslaves on the way.

Thanks a lot to all the people who have contributed or will contribute soon. It's very rewarding to see the Python community responding in an enthusiastic manner to a call to do better testing of Python projects.

Titus has a new blog

If you've been reading Titus's advogato blog, you'll be interested in knowing that he has a new blog: "Daily Life in an Ivory Basement". Will Guaraldi will be happy to know the blog is based on pyblosxom.

Tuesday, September 12, 2006

Pybots project keeps rolling

Some updates on the Pybots project:
  • Mike Taylor aka Bear from OSAF contributed 2 buildslaves: an Ubuntu Dapper box for testing his parsedatetime package, and an Intel Mac OSX box for testing two libraries used by OSAF -- vobject and zanshin
  • Manuzhai contributed a Gentoo box for testing pysqlite and Trac
  • Elliot Murphy from mysql.com will contribute an AMD-64 Ubuntu Dapper box for testing MySQLdb
  • In summary, we're up to 5 (soon to be 6) buildslaves, with more on the way
  • Seo Sanghyeon created a Google Code project for pybots; he and I are the current admins for this project; you can browse the Subversion repository for various scripts and buildbot config. files
  • As a cool side note, Sanghyeon rewrote the home page for pybots.org in reST; the page is kept in subversion and the server hosting pybots.org is doing a svn update every hour, followed by a rest2html call
As I said in a previous post, the Pybots setup already proved its usefulness by uncovering issues with new keywords such as 'with' and 'as'. Some of the projects affected by these new keywords are zope.interface, roundup and zanshin. According to the Zope developers, the issue had been fixed a while ago in the svn repository, but no release has happened since. The roundup developers already fixed the issue -- nice to see this.

Also, the Trac unit tests, when running against the latest Python trunk, are failing with an ugly backtrace. If somebody can shed some light, please do.

Again, if you're interested in running a Pybots buildslave, take a look at the various pieces of documentation at pybots.org and send a message to the Pybots mailing list.

Friday, September 08, 2006

Pay attention to the new 'with' and 'as' keywords

As of Sept. 6th (revision 51767), Python 2.6 has two new keywords: with and as. Python code that uses either one of these words as a variable name will be in trouble. How do I know that? Because the Twisted unit tests have been failing in the Twisted Pybots buildslave ever since. Actually the issue is not with Twisted code, but with zope.interface, which is one of the Twisted pre-requisites. Here's the offending code:

Traceback (most recent call last):
File "/tmp/Twisted/bin/trial", line 23, in
from twisted.scripts.trial import run
File "/tmp/Twisted/twisted/scripts/trial.py", line 10, in
from twisted.application import app
File "/tmp/Twisted/twisted/application/app.py", line 10, in
from twisted.application import service
File "/tmp/Twisted/twisted/application/service.py", line 20, in
from twisted.python import components
File "/tmp/Twisted/twisted/python/components.py", line 37, in
from zope.interface.adapter import AdapterRegistry
File "/tmp/python-buildbot/local/lib/python2.6/site-packages/zope/interface/adapter.py", line 201
for with, objects in v.iteritems():
^
SyntaxError: invalid syntax

It would be great if the Zope folks fixed their code so that the Twisted tests will start passing again. This issue actually impacts all packages that depend on zope.interface -- for example zanshin, which also fails in the Pybots buildslave for the OSAF libraries (graciously set up by Bear from OSAF).

If you're interested in following such issues as they arise in the Pybots buildbot farm, I encourage you to subscribe to the Pybots mailing list.

Update

I mentioned the 'with' keyword already causing problems. As it turns out, Seo Sanghyeon's buildslave, which is running tests for docutils and roundup, uncovered an issue in roundup, related to the 'as' keyword:

Traceback (most recent call last):
File "run_tests.py", line 889, in
process_args()
File "run_tests.py", line 879, in process_args
bad = main(module_filter, test_filter, libdir)
File "run_tests.py", line 671, in main
runner(files, test_filter, debug)
File "run_tests.py", line 585, in runner
s = get_suite(file)
File "run_tests.py", line 497, in get_suite
mod = package_import(modname)
File "run_tests.py", line 489, in package_import
mod = __import__(modname)
File "/home/buildslave/pybots/roundup/./test/test_actions.py", line 6, in
from roundup import hyperdb
File "/home/buildslave/pybots/roundup/roundup/hyperdb.py", line 29, in
import date, password
File "/home/buildslave/pybots/roundup/roundup/date.py", line 735
as = a[0]
^
SyntaxError: invalid syntax

Wednesday, September 06, 2006

Pybots update

I'm happy to report that the Pybots project got its first user other than yours truly. Seo Sanghyeon set up a buildslave running on Debian Unstable which is running the docutils unit tests every time a checkin is made into Python trunk or in the 2.5-maint branch.

Marc-Andre Lemburg also offered to run a buildslave for the egenix-mx-base tests, while Manuzhai offered to run a buildslave for the Trac tests. Skip Montanaro also expressed interest in running a buildslave, but he hasn't decided on a project yet.

You can see the current pybots buildslaves here. Expect to see more active buildslaves in the next few days.

Marc-Andre suggested I write a Pybots FAQ and put some info on the Python wiki, so here they are:

Modifying EC2 security groups via AWS Lambda functions

One task that comes up again and again is adding, removing or updating source CIDR blocks in various security groups in an EC2 infrastructur...