Posts

Showing posts from 2007

The power of checklists (especially when automated)

Just stumbled on this post at InfoQ on the power of checklists. It talks about a low-tech approach to improving care in hospitals, by writing down the steps needed in various medical procedures and putting together a checklist for each case. I've seen the power of this approach at my own company -- until we put together checklists with things we have to do when setting up various servers or applications, we were guaranteed to skip one or more small but important steps.

I'd like to take this approach up a notch though: if you're in the software business, you actually need to AUTOMATE your checklists. Otherwise it's still very easy for a human being to skip a step. Scripts don't usually make that mistake. Yes, a human being still needs to run the script and to make intelligent decisions about the overall outcome of its execution. If you do take this approach, make sure your scripts also have checks and balances embedded in them -- also known as tests. For example, if …

GHOP students ROCK!

I've been involved in the GHOP project for the last couple of weeks (although not as much as I'd have liked, due to time constraints) and I've been constantly amazed by the high quality of the work produced by the GHOP participants, who, let's not forget, are all still in high-school! I think all mentors were surprised at the speed with which the tasks were claimed, and at the level of proficiency showed by the students.

This bodes very well for Open Source in general, and for the Python community in particular. I hope that the students will continue to contribute to existing Python projects and start their own.

Here are some examples from tasks that I've been involved with:
Michael Kremer profiledeffbot's widefinder implementations and discussed the results superbly
Eren Turkay wrote unit tests for the pydigg module, obtaining almost 90% code coverage; he's currently tackling a 2nd task, writing unit tests for SimpleXMLRPCiammisc (not sure what the real name i…

Interview with Jerry Weinberg at Citerus

Read it here. As usual, Jerry Weinberg has many thought-provoking things to say. My favorite:

"Q: If you're the J.K Rowling of software development, who's Harry P then?A: Well, first of all, I'm not a billionaire, so it's probably not correct to say I'm the J.K. Rowling of software development. But if I were, I suspect my Harry Potter would be a test manager, expected to do magic but discounted by software developers because "he's only a tester." As for Voldemort, I think he's any project manager who can't say "no" or hear what Harry is telling him."Testers are finally redeemed :-)

Vonnegut's last interview

Via Tim Ferriss's blog, an inspiring interview with Kurt Vonnegut. BTW, if you haven't read Tim's '4-hour workweek' book, I highly recommend it. It will make you green with envy, but it will also offer you some good ideas about organizing your work and your life a bit differently.

PyCon'08 Testing Tutorial proposal

It's that time of the year, when the PyCon organizers are asking for talk, panel, and tutorial proposals. Titus and I are thinking about doing a three-peat of our Testing tutorial, but this time....with a twist. Read about it on Titus's blog; then send us your code/application that you'd like to test, or problems you have related to testing. Should be lots of fun.

Pybots updates

Time for the periodical update on the Pybots project. Since my last post in July, John Hampton added a buildslave running Gentoo on x86 and testing Trac and SQLAlchemy. A belated thank you to John.

I also had to disable the tests for bzr dev on my RH 9 buildslave, because for some reason they were leaving a lot of orphaned/zombie processes around.

With help from Jean-Paul Calderone from the Twisted team, we managed to get the Twisted buildslave (running RH 9) past some annoying multicast-related failures. Jean-Paul had me add an explicit iptables rule to allow multicast traffic. The rule is:
iptables -A INPUT -j ACCEPT -d 225.0.0.0/24

This seemed to have done the trick. There are some Twisted unit tests that still fail -- some of them are apparently due to the fact that raising string exceptions is now illegal in the Python trunk (2.6). Jean-Paul will investigate and I'll report on the findings -- after all, this type of issues is exactly why we set up the Pybots farm in the first pl…

Compiling mod_python on RHEL 64 bit

I just went through the fairly painful exercise of compiling mod_python 3.3.1 on a 64-bit RHEL 5 server. RHEL 5 ships with Python 2.4.3 and mod_python 3.2.8. I needed mod_python to be compiled against Python 2.5.1. I had already compiled and installed Python 2.5.1 from source into /usr/local/bin/python2.5. The version of Apache on that server is 2.2.3.

I first tried this:

# tar xvfz mod_python-3.3.1.tar.gz
# cd mod_python-3.3.1
# ./configure --with-apxs==/usr/sbin/apxs --with-python=/usr/local/bin/python2.5
# make

...at which point I got this ugly error:

/usr/lib64/apr-1/build/libtool --silent --mode=link gcc -o mod_python.la \
-rpath /usr/lib64/httpd/modules -module -avoid-version finfoobject.lo \
hlistobject.lo hlist.lo filterobject.lo connobject.lo serverobject.lo util.lo \
tableobject.lo requestobject.lo _apachemodule.lo mod_python.lo\
-L/usr/local/lib/python2.5/config -Xlinker -export-dynamic -lm\
-lpython2.5 -lpthread -ldl -lutil -lm
/usr/bin/ld: /usr/local/lib/python2.5/config/libpython2…

What's more important: TDD or acceptance testing?

The answer, as far as I'm concerned, is 'BOTH'. Read these entertaining blog posts to see why: Roy Osherove's JAOO conference writeup (his take on Martin Fowler's accent cracked me up), Martin Jul's take on the pull-no-punches discussions on TDD between Roy O. and Jim Coplien, and also Martin Jul's other blog post on why acceptance tests are important.

As I said before, holistic testing is the way to go.

Roy Osherove book on "The art of unit testing"

Just found out from Roy Osherove's blog that his book on "The Art of Unit Testing" is available for purchasing online -- well, the first 5 chapters are, but then you get the next as they're being published. Roy uses NUnit to illustrate unit testing concepts and techniques, but that shouldn't deter you from buying the book, because the principles are pretty much the same in all languages. I'm a long time reader of Roy's blog and I can say this is good stuff, judging by his past posts on unit testing and mock testing techniques.

Beware of timings in your tests

Finally I get to write a post about testing. Here's the scenario I had to troubleshoot yesterday: a client of ours has a Web app that uses a java applet for FTP transfers to a back-end server. The java applet presents a nice GUI to end-users, allowing them to drag and drop files from their local workstation to the server.

The problem was that some file transfers were failing in a mysterious way. We obviously looked at the network connectivity between the user reporting the problem initially and our data center, then we looked at the size of the files he was trying to transfer (he thought files over 10 MB were the culprit). We also looked at the number of files transferred, both multiple files in one operation and single files in consecutive operations. We tried transferring files using both a normal FTP client, and the java applet. Everything seemed to point in the direction of 'works for me' -- a stance well-known to testers around the world. All of a sudden, around an hou…

Barack Obama is now a connection

That's the message I see on my LinkedIn home page. How could this be possible, you ask? Well, yesterday I checked out my home page, and I noticed the 'featured question of the day' asked by Barack Obama himself (of course, the question was "how can the next president better help small businesses and entrepreneurs thrive".) A co-worker decided to send a LinkedIn invite to Barack. A little while later, he got the acceptance in his inbox. I followed his example, just for fun, and what do you know, I got back the acceptance in a matter of SECONDS, not even minutes! It seems that B.O. has set his LinkedIn account to accept each and every invite he gets. I guess when you're running for president, every little statistic counts. He already has 500+ connections, and I'm sure the time will come when he'll brag to the other candidates that his LinkedIn account is bigger than theirs.

The bottom line is that YOU TOO can have Barack as your connection, if only to br…

I must be bored out of my mind

Image
...otherwise why would I have taken the time to get my nerd score?





Now YOU do it!

Security testing book review on Dr. Dobbs site

I wrote a review for "The Art of Security Testing" a while ago for Dr. Dobbs. I found out only now that it's online at the Dr. Dobbs's Portal site. Read it here.

Weinberg on Agile

A short but sweet PM Boulevard interview with Jerry Weinberg on Agile management/methods. Of course, he says we need to drop the A and actually drop 'agile' altogether at some point, and just talk about "normal, sensible, professional methods of developing software." Count me in.

Jakob Nielsen on fancy formatting and fancy words

Just received the latest Alertbox newsletter from Jakob Nielsen. The topic is "Fancy Formatting, Fancy Words = Ignored". I'd have put 2 equal signs in there, but anyway....The 'ignored' in question is your web site, if you're trying to draw attention to important facts/figures by using red bold letters and pompous language. Nielsen's case study in the article is the U.S. Census Bureau's homepage, which displayed the current population of the US in big red bold letters, and called it "Population clock". As a result, users were confused as to the meaning of that number, and what's more, they didn't bother to even read the full number, because they thought it's an ad of some sort. Interesting stuff.

Some notes from the August SoCal Piggies meeting

Read them here.

Put your Noonhat on

You may have seen this already, but here's another short blurb from me: Brian Dorsey, a familiar face to those of you who have been at the last 2 or 3 PyCon conferences, has launched a Django-based Web site he called Noonhat. The tagline says it all: "Have a great lunch conversation". It's a simple but original, fun and hopefully viral idea: you specify your location on a map, then you indicate your availability for lunch, and Noonhat puts you in touch with other users who have signed up and are up for lunch in your area at that time.

This has potential not only for single people trying to find a date, but also for anybody who's unafraid of stepping out of their comfort zone and strike interesting conversations over lunch. Brian and his site have already been featured on a variety of blogs and even in mainstream media around Seattle. Check out the Noonhat blog for more details.

Well done, Brian, and may your site prosper (of course, the ultimate in prosperity is be…

Fuzzing in Python

I just bought "Fuzzing: Brute Force Vulnerability Discovery" and skimmed it a bit. I was pleasantly surprised to see that Python is the language of choice for many fuzzing tools, and clearly the favorite language of the authors, since they implemented many of their tools in Python. See the fuzzing.org site/blog also, especially the Fuzzing software page. Sulley in particular seems a very powerful fuzzing framework. I need to look more into it (so much cool stuff, so little time.)

Update: got through the first 5-6 chapters of the book. Highly entertaining and educational so far.

Werner Vogels talk at QCon

Werner Vogels is the CTO of Amazon. You can watch a talk he gave at the QCon conference on the topics of Availability and Consistency. The bottom line is that, as systems scale (and for amazon.com that means hundreds of thousands of systems), you have to pick 2 of the following 3: Consistency, Availability, Partitioning (actually the full name of the third one is "Tolerance to network partitioning.) This is called the CAP theorem, and Eric Brewer from Inktomi first came up with it.

Vogels pretty much equated partitioning with failure. Failure is inevitable, so you have to choose it out of those 3 properties. You're left with a choice between consistency and availability, or between ACID and BASE. According to Vogels, it turns out there's also a middle-of-the-road approach, where you choose a specific approach based on the needs of a particular service. He gave the example of the checkout process on amazon.com. When customers want to add items to their shopping cart, you AL…

Automating tasks with pexpect

I started to use pexpect for some of the automation needs I have, especially for tasks that involve logging into a remote device and running commands there. I found the module extremely easy to use, and the documentation on the module's home page is very good. Basically, if you follow the recipe shown there for logging into an FTP server, you're set.

A couple of caveats I discovered so far:
make sure you specify correctly the text you expect back; even an extra space can be costly, and make your script wait forever; you can add '.*' to the beginning or to the end of the text you're expecting to make sure you're catching unexpected charactersif you want to print the output from the other side of the connection, use child.before (where child is the process spawned by pexpect)Here's a complete script for logging into a load balancer and showing information about a load balanced server and its real servers:

#!/usr/bin/env python

import pexpect

def show_virtual(child…

Think twice before working from a Starbucks

Here's an eye-opening article talking about a tool called Hamster that sniffs wireless traffic and reveals plain-text cookies which can then be used to impersonate users. The guy running the tool was able to log in into some poor soul's Gmail account during a BlackHat presentation.

Pretty scary, and it makes me think twice before firing up my laptop in a public wireless hotspot. The people who wrote Hamster, from Errata Security, already released another tool called Ferret, which intercepts juicy bits of information -- they call it 'information seepage'. You can see a presentation on Ferret here. They're supposed to release Hamster into the wild any day now.

Update: If the above wasn't enough to scare you, here's another set of wireless hacking tools called Karma (see the presentation appropriately called "All your layers are belong to us".)

That's what I call system testing

According to news.com, the IT systems for the 2008 Olympics in Beijing will be put through rigorous testing which will take more than 1 year! The people at Atos Origin, the company in charge of setting up the IT for the 2008 Olympics, clearly know what they are doing.

It's also interesting that the article mentions insiders as a security threat -- namely, that insiders will try to print their own accreditation badges, or do it for their friends, etc. As always, the human factor is the hardest to deal with. They say they resort to extensive background checks for the 2,500 or so IT volunteers, but I somehow doubt that will be enough.

For your summer reading list: book on Continuous Integration

I found out about this book from the InfoQ blog -- the book is called Continuous Integration: Improving Software Quality and Reducing Risk and it is written by 3 guys from Stelligent, who also blog regularly on testearly.com. Seems like a very interesting and timely read for people interested in automated testing and obviously in continuous integration (which to me are the 2 first stepping stones on the path to 'agile testing'). You can also read a chapter from the book in PDF format: "Continuous testing".

Notes from the SoCal Piggies meeting

Just published the notes from the SoCal Piggies meeting we had last week on the "Happenings in Python Usergroups" blog. Keywords: jabber, xmpppy,orbited, comet.

Dilbert, the PHB, and automated tests

Today's Dilbert cartoon shows that even the PHB can think "agile". He tells Dilbert to go write his own automated test software, instead of buying off-the-shelf. That's got to be agile, with small "a" :-) Of course, it's not recommended to call your team members "big babies" during the stand-up meeting.

Your purpose is the Python group

Image
At our last SoCal Piggies meeting 2 days ago, Diane Trout showed us some Jabber bots, one of them based on PyAIML, an Eliza/AI kind of bot. When Diane asked this awfully intelligent little bot to smile for the Python group, this is what it replied:



How did it guess??? I used to not be a big believer in AI, but now I'm sold.

Pybots updates

After a long hibernation period, the Pybots project shows some signs of life -- I should probably say mixed with signs of death. Elliot Murphy from Canonical added the Storm ORM project to his AMD64 Ubuntu Gutsy buildslave, while Manuzhai and Jeff McNeil had to drop their buildslaves out of the mix, hopefully only temporarily. In Manuzhai's case though, the project he was testing -- Trac -- proved to have maintainers that were not interested in fixing their failing tests. In this case, there is no point in testing that project in Pybots. Hopefully Manuzhai will find a different, more test-infected project to run in Pybots.

Speaking of test-infected projects, it was nice to see unit testing topping the list of topics in the Django tutorial given at OSCON by Jeremy Dunck, Jacob Kaplan-Moss and Simon Willison. In fact, Titus is quoted too on this slide, which seems to be destined to be a classic (and I'm proud he uttered those words during the Testing Tools Panel that I moderated …

Another Django success story

Pownce is yet another social networking site, but with the added twist that the creator of digg is one of its founders. Read about the technologies used to build it (Django included) here.

Interested in a book on automated Web app testing?

Want to know more about twill and Selenium? In this case, I happen to know a very good book just published in the O'Reilly Short Cuts series :-) It's cheap too, $9.99, so please go buy it!

Brian Marick on "Four implementation styles for workflow tests"

Brian Marick just posted a very insightful article on various types of Web application tests that he calls 'workflow tests'. If you're in the Web application testing business, this should help you decide which type -- or better which TYPES (plural) -- of tests to use in your specific application scenario.

Consulting opportunities

If you're a Django or Selenium expert and live in the Los Angeles area (or even if you live in a different area, but can meet periodically with clients in Los Angeles), please send me an email at grig at gheorghiu.net. I know of a couple of great consulting gigs.

Dilbertian Information Dashboard

Do NOT do this :-)

Eliminating dependencies with regenerative build tools

This just in from Michael Feathers of "Working Effectively with Legacy Code" fame: a blog post on regenerative build tools. In the post, Michael describes an eye-opening practice related to continuous integration. Some smart people had the idea of running a script as part of a continuous build system that would comment out #include lines, one at a time, and then run the build. If the build succeeded, it meant that the include line in question was superfluous and thus could be deleted. Very interesting idea.

I bet this idea could be easily applied to Python projects, where you would comment out import statements and see if your unit test suite still passes. Of course, you can combine it with snakefood, a very interesting dependency graphing tool just released by Martin Blais. And you can also combine it with fault injection tools (aka fuzzers) such as Pester -- which belongs to the Jester family of tools also mentioned by Michael Feathers in his blog post.

Brian Marick has a new blog

If you're serious about testing, you need to read Brian Marick's blog, which he recently moved to exampler.com from the old testing.com URL. Brian says:

" This is the another step in my multi-decade switch from testing.com to exampler.com. "Exampling", though not a verb, is a better description of what I do now. It includes testing, but has a larger scope."

And by the way, the style of technical writing that Brian describes in his latest post bugs me no end too...

Resetting MySQL account passwords

I recently needed to reset the MySQL root account password. Here are the steps, for future reference:
1) Stop mysqld, for example via 'sudo /etc/init.d/mysqld stop'
2) Create text file /tmp/mysql-init with the following contents (note that the file needs to be in a location that is readable by user mysql, since it will be read by the mysqld process running as that user): SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpassword');
3) Start mysqld_safe with following option, which will set the password to whatever was specified in /tmp/mysql-init: $ sudo /usr/bin/mysqld_safe --init-file=/tmp/mysql-init &
4) Test connection to mysqld: $ sudo mysql -uroot -pnewpassword
5) If connection is OK, restart mysqld server: $ sudo /etc/init.d/mysqld restart

Also for future reference, here's how to reset a normal user account password in MySQL:

Connect to mysqld as root (I assume you know the root password):

$ mysql -uroot -prootpassword

Use the SET PASSWORD comman…

Apache virtual hosting with Tomcat and mod_jk

In a previous post I talked about "Configuring Apache 2 and Tomcat 5.5 with mod_jk". I'll revisit some of the topics in there, but within a slightly different scenario.

Let's say you want to configure virtual hosts in Apache, with each virtual host talking to a different Tomcat instance via the mod_jk connector. Each virtual host serves up a separate application via an URL such as http://www.myapp.com. This URL needs to be directly mapped to a Tomcat application. This is a fairly important requirement, because you don't want to go to a URL such as http://www.myapp.com/somedirectory to see your application. This means that your application will need to be running in the ROOT of the Tomcat webapps directory.

You also want Apache to serve up some static content, such as images.

Running multiple instances of Tomcat has a couple of advantages: 1) you can start/stop your Tomcat applications independently of each other, and 2) if a Tomcat instance goes down in flames, it w…

JRuby buzz

I wish I could put "Jython buzz" as the title of my post, but unfortunately I can't seem to detect any Jython buzz anywhere. JRuby though seems to generate a lot of it, judging by this InfoQ article on Mingle, a commercial application based on JRuby and created by ThoughtWorks Studios.

One thing I found very interesting in the InfoQ article was that ThoughtWorks preferred to develop Mingle with JRuby (which is the JVM-based version of Ruby) over writing it on top of Ruby on Rails. They cite ease of deployment as a factor in favor of JRuby:

"In particular, the deployment story for Ruby on Rails applications is still significantly more complex than it should be. This is fine for a hosted application where the deployment platform is in full control of a single company, but Mingle isn't going to be just hosted. Not only is it going to need to scale ‘up’ to the sizes of Twitter (okay, that's wishful thinking and maybe it won't need to scale that much) but it…

Michael Dell uses Ubuntu on his home laptop

Found this on Planet Ubuntu, which is buzzing with the news that Dell will be offering laptops preloaded with Feisty Fawn. I still use Edgy on my Inspiron 6000, but I'll probably upgrade to Feisty soon. Or maybe I'll just wait for a Gutsy Gibbon to burst on the scene :-)

What programming language are *you*?

Image
Not sure what this really says about me, but I am PHP according to the survey below...(which I stumbled upon via Brian Marick's blog)


Which Programming Language are You?

PyCon07 Testing Tools Tutorial slides up

Spurred by a request from David Brochu, I put the PyCon 07 Testing Tools Tutorial slides up on agilistas.org. Titus's slides are named titus-tutorial-a.pdf through -e.pdf. My slides are only one PDF file, as my portion of the tutorial consisted mainly in Selenium and FitNesse demos. Enjoy!

Mounting local file systems using the 'bind' mount type

Sometimes paths are hardcoded in applications -- let's say you have the path to the Apache DocumentRoot directory hardcoded inside a web application to /home/apache/www.mysite.com. You can't change the code of the web app, but you want to migrate it. You don't want to use the same path on the new server, for reasons of standardization across servers. Let's say you want to set DocumentRoot to /var/www/www.mysite.com. But /home is NFS-mounted, so that all users can have their home directory kept in one place. One not-so-optimal solution would be to create an apache directory under /home on the NFS server. At that point, you can create a symlink to /var/www/www.mysite.com inside /home/apache. This is suboptimal because the production servers will come to depend on the NFS-mounted directory. You would like to keep things related to your web application local to each server running that application.
A better solution (suggested by my colleague Chris) is to mount a local dire…

Dell to offer pre-installed Linux on desktops

I knew about the Dell survey which asked people whether they'd like to see Linux pre-installed on Dell desktops and laptops. Looks like more than 70% of the respondents said yes (see this BBC story) -- so Dell is going for it. Now the question is which flavor(s) of Linux will be offered. From what I remember, the survey mentioned Fedora Core, Ubuntu and OpenSuse. Regardless, this is a pretty big win for Linux.

OLPC and the Romanian politicians

Interesting blog post from Jani Monoses on how the Romanian parliament rejected the country's participation in the OLPC program. All the arguments centered around cost and lack of applications such as....MS Word! As Jani says -- cluelessness abounds.

Having seen Ivan Krstic's keynote on OLPC at PyCon this year, I realize that the One Laptop Per Child program is mainly about re-introducing kids to their intuitive ways of learning, through play, peer activities and free exploration, as opposed to the centralized, one-to-many teaching method that is used in schools everywhere. The laptop becomes in this case just a tool for facilitating the new ways of learning -- or I should say the old ways, since this is what kids do naturally. But this is one of those disruptive ideas that is hard to grasp by serious grown-up people, especially politicians...

File sharing with Apache and WebDAV

If you want to share files from a Linux box to Windows clients, Samba is a popular solution. However, it can also be done with Apache and WebDAV. Here is a short HOWTO.

1) Let's say we want to share files in a directory named /usr/share/myfiles. I created a sub-directory called dav in that directory, and then I ran:
# chmod 775 dav
# chgrp apache dav2) Make sure httpd.conf loads the mod_dav modules:
LoadModule dav_module modules/mod_dav.so
LoadModule dav_fs_module modules/mod_dav_fs.so3) Create an Apache password file (if you want to use basic authentication) and a user -- let's call it webdav:
# htpasswd -c /etc/httpd/conf/.htpasswd webdav
4) Create a virtual host entry in httpd.conf, similar to this one:
<VirtualHost *>
ServerName share.mydomain.com
DocumentRoot "/usr/share/myfiles"
<Directory "/usr/share/myfiles">
Options Indexes FollowSymLinks MultiViews
AllowOverride AuthConfig
Order allow,deny
allow from all
</Directory>
ErrorLog shar…

Ubuntu "command not found" magic

Via Alan Pope's blog: Ubuntu Edgy and above includes a "command not found magic" -- a bash hook that intercepts 'command not found' errors and replaces them with more useful messages, such as what packages you need to install to get that command. I tried it on my Edgy laptop and what do you know, it actually works.

First you need to apt-get the command-not-found package:

$ sudo apt-get install command-not-found
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
command-not-found-data
The following NEW packages will be installed:
command-not-found command-not-found-data
0 upgraded, 2 newly installed, 0 to remove and 15 not upgraded.
Need to get 471kB/475kB of archives.
After unpacking 6263kB of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://us.archive.ubuntu.com edgy/universe command-not-found-data 0.1.0 [471kB]
Fetched 300kB in 2s (109kB/s)
Se…

Founder of Debian joins Sun

Interesting announcement from Ian Murdoch, the founder of Debian: he's joining Sun as "Chief Operating Platforms Officer" (now that's a mouthful.) I've been really skeptical so far about Sun's embrace of Open Source, but this seems like a major step in the right direction.

CheeseRater - voting in the CheeseShop

Found via Joe Gregorio's blog: CheeseRater, a slick-looking Django app that lets you vote on CheeseShop packages. Interesting idea. Since tagging/redditing/social networking/web2.0ing/etc. are all the rage these days, this might prove more popular than the automatic scoring that Michał is doing with the Cheesecake Service. Maybe we can combine the two....that would be interesting.

Stone soup as a cure for broken windows

Stones are used to break windows, but in this insightful blog post, Dave Nicolette shows how making stone soup (i.e. getting everybody to contribute a bit of something) can help deal with the broken windows syndrome. Getting everybody to contribute is an art I've been trying to master myself, but like any art, it's far from easy...

A few good agile men

Kumar sent me a link to a hilarious blog post: A few good managers. It's a gem. Excerpt:

"Marketing: "Did you cut the automated, edit sync [insert favorite feature here] feature?"
Development: "I did the job I was hired to do."Marketing: "Did you cut the automated, edit sync feature?"
Development: "I delivered the release on time."Marketing: "Did you cut the automated, edit sync feature?"
Development: "You're g%$#@*& right I did!""

I'm Ubuntu counted!

I just registered as Ubuntu user #12343 at Ubuntu Counter. Pretty cool number!

Who said love is simple?

Pay attention when you say 'I love you'.

testing-in-python mailing list

Titus created a mailing list dedicated to topics related to testing in Python. If you're interested, you can subscribe via its mailman interface here. We hope it will become a discussion forum for things such as:
testing tools that people have successfully used in their projectstesting techniques that help in certain situations (mocking for example)real life scenarios where a specific type of testing (e.g. functional) helped more than another type of testing (e.g. unit)etc. etc.
I think I will affectionately refer to this list as TIP from now on :-)

Update 02/28/07

What do you know, the name TIP struck a chord, so Titus created an alias for it. You can now send email to the list via tip at lists.idyll.org too.

William McVey's PyCon notes as mindmap

Via Elliot Murphy's blog, a very nice mindmap created by William McVey and showing his notes from PyCon07. It demonstrates the power of mindmaps: a lot of information condensed in one page, with links to more detailed information below.

Also from William, other PyCon07 notes.

Testing Tools Panel at PyCon

I'd like to thank the participants in the Testing Tools Panel at PyCon07 for sharing their insights into testing with the audience. Here they are, in alphabetical order of first name:
Benji York: zope.testbrowserBrian Dorsey: py.test (representing Holger Krekel)Chad Whitacre: testosterone (created) switched to noseIan Bicking: paste.test.fixture, minimock, FitLoaderJeff Younker: PyMockKumar McMillan: fixture - module for loading and referencing test dataMartin Taylor: test framework within TINeal Norwitz: PyCheckerTim Couper: WATSUP (Windows GUI Testing)Titus Brown: twill, scotch, figleaf, pinocchio
Matt Harrison has a very good write-up on the discussions we had during the panel (actually I lifted the list above from his blog post, because he summarized it so well).

One thing that I think all the participants felt, and maybe the audience too, was that 45 minutes was totally not sufficient for this kind of panel. And I know I felt the same thing with the other two panels, for Python-…

Testability

During the Testing Tools tutorial that Titus and I gave at PyCon, there was a short discussion on testability -- what makes software more testable? I mentioned a list put together by Michael Bolton, and summarized/enhanced by Adam Goucher in this blog post. Recommended reading, both for developers who want to add testing hooks into their software, and for testers who want to know what to ask for from developers so that their life gets easier (and if you're one of the unfortunate souls who have to deal with Java or .NET, this blog post by Roy Osherove talks about testability and pure OOP.)

Although our tutorial was focused on tools and techniques for implementing test automation, we also mentioned that you will never be able to get rid of manual testing. Even though the Google testing team says that 'Life is too short for manual testing' (and I couldn't agree more with them), they hasten to qualify this slogan by adding that automated testing frees you up to do more mean…

Photos from PyCon panels

Here are photos I took today at the python-dev panel and at the Web frameworks panel. Enjoy!

BTW, here are two very good write-ups on the Web framework panel: one from Matt Harrison, the other from James Bennett.

PyCon day 1

Gave the testing tutorial with Titus yesterday; went pretty well from the feedback we got. We'll publish the slides soon.

Just got out of the first keynote, Ivan Krstic's talk on the "One Laptop Per Child" project. Pretty interesting -- here are some tidbits I remember:
OLPC wants to change the way teaching and learning is done these days; they want to go back to the time when preschool kids interacted with each other by playing, and learned naturally peer-to-peer (as opposed to institutionalized teaching, which is one-to-many)
contrary to popular opinion, the laptop does not have a hand crank (it would wear down too fast if it had one); however, the laptop can be powered by a pull string that reacts to the puller's strength and powers the device accordingly
the 2 rabbit ears are used for wireless; the laptop can speak 802.11s, a new protocol that can be used for fully meshed networking; as soon as one laptop is connected to the internet, all the other ones in its me…

Wikipatterns

Via Grady Booch's blog, a site which looks very promising: Wikipatterns. It identifies patterns and anti-patterns for wiki adoption. Here's an excerpt:

"Any grassroots, or bottom-up, strategy is the best place to start since the success of a wiki depends on building active, sustainable participation and this only happens when people see that the software is simple enough to immediately be useful, and meets their needs without requiring them to spend lots of extra time. A good first step is to identify a group or department who would likely benefit the most from using a wiki, and whose people are open to trying new tools. If you're looking to expand wiki use in another group, look for the thought leader in the group - someone who is very forward thinking, respected by peers, and willing to Champion a new idea and get others around them involved."
Seems like a very good complement to a book I read recently: "Fearless Change: Patterns for Introducing New Ideas&qu…

Anybody doing LDom on Solaris Sparc?

Lazy Web-type question: has anybody tried out the brand new LDom functionality available in Solaris 10? From some googling around I've done, it seems Sun hasn't yet shipped the full-fledged LDom functionality in Solaris 10 version 11/06.

LDom seems like a cool way to partition some big Solaris Sparc boxes, if you have them. I wonder if the logical domains/virtual machines created with LDom can have Ubuntu installed on top of them (because Ubuntu supports Solaris Sparc).

The Buildbot project has a Trac instance as its home

Brian Warner just announced on the buildbot-devel mailing list that his Buildbot project has a brand new home in the form of a Trac instance. Check it out here. Glad to see the move, as Trac is so much nicer to work with than the clunky SourceForge interface.

Ubuntu not to activate proprietary drivers by default

This just in via Jonathan Carter's blog on Planet Ubuntu: the Ubuntu Technical Board has decided that Ubuntu will not activate proprietary drivers by default. Proprietary drivers will be provided for convenience, and users will of course be free to install them if they so choose. This should settle a controversy which has been raging for a while in the Ubuntu community.

Cheesecake Service launched

The Cheesecake Service is a result of Michał Kwiatkowski's hard work during the Google Summer of Code 2006. The Web interface for the service is based on web.py and it has been up and running since last August, but we're making it public now, to coincide with the release of Cheesecake 0.6.1.

The back-end of the Cheesecake Service talks directly to the PyPI repository using the PyPI API in order to find out about packages newly posted to PyPI. Then the service uses Cheesecake and tries to download, install, and score the package. If your package is not there, it might mean you haven't released a new version after August 10, 2006, date from which we started to score packages. Let us know and we can manually score it so that it appears in the list.

Michał just posted a blog entry on Cheesecake and its Service, so please read it and let us know how we can improve on the various things he describes. We are aware that scoring packages is controversial, and we've been called na…

Internal blogs as project tracking tools

I'm reading "The Corporate Blogging Book" by Debbie Weil. She talks about two different types of blogs -- internal (Intranet-type) and external (public access) -- and she mentions how some big name companies use internal blogs for all sorts of purposes: knowledge/information sharing, email replacement, and, the thing that caught my attention, project management. Apparently IBM is big on all these things when it comes to internal blogs.

So I had a mini-revelation: an internal blog is a very good tool for tracking time you spend on various projects. Take the example of a hosting company -- it can set up an internal blog and have categories corresponding to various projects/customers; employees can jot down a summary of what they worked on each day, and put it in the appropriate category. After a while, a timeline of work done on particular projects emerges. Because posts are automatically dated, it's easy to see what you were working on 3 weeks or 3 months ago. And each…

New job

Today was my first day at RIS Technology, a Web hosting/managed services company which hosts grammy.com and missuniverse.com, among other sites. My responsibilities include selecting and helping implement various technologies that make up our offerings to our customers. Of course, if you follow my blog, you know that Python and automated testing are big in my book, so you can be sure that our offerings will include both :-) Stay tuned for more details.

Connecting to people on LinkedIn

I wasn't such a big LinkedIn fan until a short time ago, when a post by Guy Kawasaki caught my attention. Then synchronicity kicked in and Tennessee Leeuwenburg sent a message to the python-advocacy mailing list, asking Python developers to connect to each other on LinkedIn; here's what Tennessee had to say:

"One way to help spread Python would be to have a strong presence of Python developers in various online networks. One that springs to mind is LinkedIn, a job related social networking site.

If we could encourage Python developers to start adding eachother to their LinkedIn network, then we shoud be able to create a well-connected developer network with business and industry contacts. This should benefit everyone -- both people looking for Python developers, and also people looking for work."

So in the past week or so I started to send LinkedIn invitations to people I know, either by having worked with them, or through the various forums, mailing lists and Open Sou…

Pybots updates

Since my last Pybots-related blog post, a lot has happened. We added 2 buildslaves, a Sparc Solaris 10 host running the Django unit test suite (courtesy of Matthew Flanagan) and a G5 OSX host running the SQLAlchemy test suite (courtesy of Skip Montanaro). So now we have 10 buildslaves altogether. I also added more test suites to my Ubuntu Breezy buildslave. In addition to the Cheesecake test suite, I'm now running the unit test suites for the py library, nose, twill and Testoob.

Email notifications finally started to work too, after I finally figured out I was passing the wrong builder names to the MailNotification class. And we also have RSS feeds available. If you want to be notified of failures from all builders, subscribe to:

http://www.python.org/dev/buildbot/community/all/rss or
http://www.python.org/dev/buildbot/community/all/atom

To be notified of failures from the trunk builders, subscribe to:

http://www.python.org/dev/buildbot/community/trunk/rss or
http://www.python.org/dev/b…

Steve Rowe on "Letting test drive the process"

Just came across a blog post by Microsoft' Steve Rowe called "Letting test drive the process". Steve quotes an article by Richard Collins -- "Test, test and test again" -- and adds his own observations on the practices of involving testers early in the development process, and of building testable interfaces into the product instead of heavy UIs.

According to Steve Rowe, Microsoft's development and testing process follows these recommended practices. I quote Steve:

"Also, at Microsoft, testing begins from day one. Every product I've ever been involved with at Microsoft has had daily builds from very early on. Every product has also had what we call BVTs (build verfication tests) that are run every day right after the build completes. If any of their tests fail, the product is held until they can be fixed."

Hmmm...I would expect Microsoft to have less problems with their products in this case. But I think a couple of problems that plague Micros…

Testing tutorial at PyCon07

Titus and I will present a tutorial on "Testing Tools in Python" at PyCon07. It is scheduled for the afternoon session on Thursday Feb.22. It will be an improved version of our "Agile Development and Testing in Python" tutorial from last year.

Here is the tutorial outline (courtesy of Titus). If you have any suggestions, please leave a comment.

Introduction
* Why test?
* What to test?
* Using testing to boost maintainability of code.

Setting up a project
* Source control management with Subversion.
* A brief introduction to using Trac for project
documentation and ticket management.
* Packaging with distutils
* Packaging with setuptools
* Registering your project with the Python Cheeseshop
* What else is out there? (distributed vs svn, roundup, ...)

Unit testing
* How to think about unit testing
* Using nose to run unit tests
* doctest-style unit tests
* What else is out there? (unittest, py.test, testosterone...)

Functional Web testing with twill

Testing Tools Panel at PyCon07: questions needed

I will be moderating the Testing Tools Panel at PyCon07, currently scheduled from 11:40 AM to 12:25 PM on Sat. Feb. 24th (and immediately followed by Guido's keynote). I put together a Wiki page for the panel, with questions and topics that I thought would be interesting for the audience (and with some input from Ian Bicking.)

I'd be very grateful if people who plan on attending the panel could add more questions or topics of interest either by directly editing the Wiki page, or by leaving a comment here, or by sending me an email at grig at gheorghiu.net. Thanks in advance!

New Year's resolution

Here's one New Year resolution I'm trying to keep: each day, read the corresponding page for that date from "The Daily Drucker". I blogged about this book before, and I continue to be amazed at the insight and wisdom that Drucker manages to pack in almost every sentence he writes. Although Drucker writes about general practices of management and leadership, many of his ideas can be easily applied to software development in general, and testing in particular.

Here are some fragments from January 4th, on "Organizational inertia", which can be applied just as well to any software project ("bitrot" and "goldplating" come to mind):

"All organizations need to know that virtually no program or activity will perform effectively for a long time without modifications and redesign. Eventually every activity becomes obsolete."

"Businessmen are just as sentimental about yesterday as bureaucrats. They are just as likely to respond to the f…

Cheesecake now including PEP8 checks

The inclusion actually happened a couple of weeks ago. I saw Johann Rocholl's message on conp.lang.python.announce where he talked about his pep8.py module -- a tool which checks Python modules against some of the style conventions in PEP8.

Here's a sample output of running pep8.py against one of the modules in the Cheesecake project. By default, pep8 reports only the first occurrence of the error or warning. The numbers after the file name represent the line and column where the error/warning occurred:

$ python pep8.py logger.py
logger.py:1:11: E401 multiple imports on one line
logger.py:7:23: W291 trailing whitespace
logger.py:8:5: E301 expected 1 blank line, found 0
logger.py:40:33: W602 deprecated form of raising exception
logger.py:60:1: E302 expected 2 blank lines, found 1
logger.py:114:80: E501 line too long (85 characters)
If you want to see all occurrences, use the --repeat flag.

If you just want to see how many lines in a given file have PEP8-related errors/warnings, use the -…

Nice summary of virtualization techniques

Martin Fowler's "Mocks Aren't Stubs" -- updated version

As the title says, Martin Fowler just announced a significant update -- indeed, a rewrite -- of his classic "Mocks Aren't Stubs". Most of the terminology he uses in the new version of the article is borrowed from Gerard Meszaros' xUnit Patterns jargon. Highly recommended. I'm not so vain as to believe that my post on mock testing influenced Martin's rewrite, so I'm just going to invoke synchronicity here :-)