Dynamically updating buildbot status text

Let's assume you want to update the build step status text displayed in the buildbot HTML status page, based on some information that only the build slave knows -- such as a version number that is computed by the slave during the build step for example.

Note that if you just want to customize the build step status with some text that is known in advance by the master (e.g. "client install" or "twill functional tests"), all you need to do is to subclass from ShellCommand and set the descriptionDone class variable to the desired custom text. See this post for more details on how to do this.

For dynamically updating the status text, the solution I found was to override some of the methods in the ShellCommand class.

My particular scenario is this: the build slave installs some package and identifies its version number. I want to be able to display that version number in the status for that build step.

I defined the following subclass of ShellCommand:

class ClientInstall(ShellCommand):
name = "client install"
description = ["running %s" % name]
descriptionDone = [name]

def __init__(self, **kwargs):
    ShellCommand.__init__(self, **kwargs)
    self.version = None

def createSummary(self, log):
    log_text = log.getText()
    s = re.search("--version=(.*)", log_text)
    if s:
        self.version = s.group(1)

def getText(self, cmd, results):
    text = self.describe(True)[:]
    if results == WARNINGS:
        text.append("warnings")
    if results == FAILURE:
        text.append("failed")
    if self.version:
        text.append("version=" + self.version)
    return text

The two most important methods in this case are createSummary and getText. I chose createSummary for overriding because it has access to the slave's log. In my case, that log contained the version number computed by the slave, so I just introduced a new variable, self.version, and set it to the result of a regular expression search for "--version=(.*)".

The getText method is called by ShellCommand inside the setStatus method, like this (the ShellCommand class lives in the process/step.py file installed under the buildbot root installation directory, in my case /usr/local/lib/python2.4/site-packages/buildbot):

def setStatus(self, cmd, results):
    # this is good enough for most steps, but it can be overridden to
    # get more control over the displayed text
    self.step_status.setColor(self.getColor(cmd, results))
    self.step_status.setText(self.getText(cmd, results))
    self.step_status.setText2(self.maybeGetText2(cmd, results))

My overridden version of getText (shown above) checks to see if self.version is non-empty, and if this is the case, it appends it to the variable text, which is a copy of the list returned by self.describe(True). Copying the list into a variable instead of modifying it in place is very important. Initially, I did something like:

text = self.describe(True)
if results == WARNINGS:
    text += ["warnings"]

The net effect of this was that each of my build slaves was updating this particular build step status with version information from all the other build slaves. It felt like a global or class-wide variable was being trampled under foot by all the build slaves, and indeed this was the case, as I found out when I sent a message to the buildbot-devel list and Brian Warned explained what was going on: the list of strings returned by self.describe() is a class-wide value that's not supposed to be mutated. Brian suggested modifying the above snippet of code to:

text = self.describe(True)
if results == WARNINGS:
    text = text + ["warnings"]

Neal Norwitz suggested the solution I finally adopted, which is to first make a copy of the list returned by self.describe, then append to it. This is more efficient, because it only allocates the list one time, then resizes it if necessary:

text = self.describe(True)[:]
if results == WARNINGS:
    text.append("warnings")

Once again, buildbot proved to be very flexible and customizable -- but not without jumping through some hoops in this particular scenario. In any case, I hope this post will be useful for buildbot users out there who want to display more customized information in their build steps.