Friday, April 28, 2006

Selenium test creation and maintenance with make_selenium.py

Michał Kwiatkowski is the author of make_selenium.py, a useful Python module that can help you create and maintain Selenium tests. With make_selenium, you can go back and forth between Selenium tests written in HTML table format and the same tests written in Python.

When they start using Selenium, most people are drawn into using the Selenium IDE, which simplifies considerably the task of writing tests in HTML table format-- especially writing Selenium "action"-type commands such as clicking on links, typing text, selecting drop-down items, submitting forms, etc. However, maintaining the tests in HTML format can be cumbersome. Enter make_selenium, which can turn a Selenium test table into a Python script.

Here's an example of an HTML-based Selenium test that deals with Ajax functionality (see this post for more details on this kind of testing):

TestCommentary
open /message/20050409174524.GA4854@highenergymagic.org
dblclick //blockquote
waitForCondition var value = selenium.getText("//textarea[@name='comment']"); value == "" 10000
store javascript{Math.round(1000*Math.random())} var
type username user${var}
type email user${var}@mos.org
type comment hello there from user${var}
click //form//button[1]
waitForCondition var value = selenium.getText("//div[@class='commentary-comment commentary-inline']"); value.match(/hello there from user${var}/); 10000
verifyText //div[@class="commentary-comment commentary-inline"] regexp:hello there from user${var}
clickAndWait //div/div[position()="1" and @style="font-size: 80%;"]/a[position()="2" and @href="/search"]
type q user${var}
clickAndWait //input[@type='submit' and @value='search']
verifyValue q user${var}
assertTextPresent Query: user${var}
assertTextPresent in Re: [socal-piggies] meeting Tues Apr 12th: confirmed
open /message/20050409174524.GA4854@highenergymagic.org
assertTextPresent hello there from user${var}
assertTextPresent delete
click link=delete
waitForCondition var allText = selenium.page().bodyText(); var unexpectedText = "hello there from user${var}" allText.indexOf(unexpectedText) == -1; 10000
assertTextNotPresent hello there from user${var}
assertTextNotPresent delete
clickAndWait //div/div[position()="1" and @style="font-size: 80%;"]/a[position()="2" and @href="/search"]
type q user${var}
clickAndWait //input[@type='submit' and @value='search']
verifyValue q user${var}
assertTextPresent Query: user${var}
assertTextPresent no matches


Let's assume we want to add more assertion commands to this table. Let's turn the TestCommentary.html file into a Python script with make_selenium:

python make_selenium.py -p TestCommentary.html

This command generates a file called TestCommentary.py, with the following contents:

S.open('/message/20050409174524.GA4854@highenergymagic.org')
S.dblclick('//blockquote')
S.waitForCondition('''var value = selenium.getText("//textarea[@name=\'comment\']");
value == ""
''', '10000')
S.pause('2000')
S.store('javascript{Math.round(1000*Math.random())}', 'var')
S.type('username', 'user${var}')
S.type('email', 'user${var}@mos.org')
S.type('comment', 'hello there from user${var}')
S.click('//form//button[1]')
S.waitForCondition('var value = selenium.getText("//div[@class=\'commentary-comment commentary-inline\']"); value.match(/hello there from user${var}/);', '10000')
S.pause('5000')
S.verifyText('//div[@class="commentary-comment commentary-inline"]', 'regexp:hello there from user${var}')
S.clickAndWait('//div/div[position()="1" and @style="font-size: 80%;"]/a[position()="2" and @href="/search"]')
S.type('q', 'user${var}')
S.clickAndWait('//input[@type=\'submit\' and @value=\'search\']')
S.verifyValue('q', 'user${var}')
S.assertTextPresent('Query: user${var}')
S.assertTextPresent('in Re: [socal-piggies] meeting Tues Apr 12th: confirmed')
S.open('/message/20050409174524.GA4854@highenergymagic.org')
S.assertTextPresent('hello there from user${var}')
S.assertTextPresent('delete')
S.click('link=delete')
S.waitForCondition('''
var allText = selenium.page().bodyText();
var unexpectedText = "hello there from user${var}"
allText.indexOf(unexpectedText) == -1;
''', '10000')
S.pause('5000')
S.assertTextNotPresent('hello there from user${var}')
S.assertTextNotPresent('delete')
S.clickAndWait('//div/div[position()="1" and @style="font-size: 80%;"]/a[position()="2" and @href="/search"]')
S.type('q', 'user${var}')
S.clickAndWait('//input[@type=\'submit\' and @value=\'search\']')
S.verifyValue('q', 'user${var}')
S.assertTextPresent('Query: user${var}')
S.assertTextPresent('no matches')

Now we can just add more commands using the special S.command(args) syntax, where command can be any Selenium command, and args are the arguments specific to that command. When we're done, we run make_selenium.py again, this time without the -p switch, in order to generate an HTML file. We can also specify a different name for the target HTML file:

python make_selenium.py TestCommentary.py TestCommentary2.html

But there's more to make_selenium than moving tests back and forth between HTML table format and Python syntax. In fact, Michał's initial goal in writing make_selenium was to offer Python programmers an easy way of creating Selenium tests by writing Python code which would then be translated into HTML.

With make_selenium, you can take full advantage of Python constructs such as for loops when you write your Selenium tests. Here's an example inspired by Michał's documentation. Assume you have a large form with various elements that you need to type in. Then you click a submit button, and you want to make sure that after submitting, the values of the elements are the same. Here's how you would do it in a make_selenium-aware Python script:

def type_values(mapping):
for key, value in mapping.iteritems():
S.type(key, value)

def verify_values(mapping):
for key, value in mapping.iteritems():
S.assertValue(key, value)

data = {
'first_name': 'John',
'last_name': 'Smith',
'age': '25',
}

type_values(data)
S.clickAndWait('submit_data')
verify_values(data)

The only convention you need to follow is to use the special S object when you want to indicate a Selenium command. The functions type_values and verify_values are normal Python functions which repeatedly call S.type and S.assertValue on all the elements of the data dictionary. If you save this code in a file called test1.py and run make_selenium on it, you get a file called test1.html with the following contents:

Test1
type first_name John
type last_name Smith
type age 25
clickAndWait submit_data
assertValue first_name John
assertValue last_name Smith
assertValue age 25

You can then run this file in a TestRunner-based Selenium test suite.

You may ask why do you need make_selenium when Selenium RC is available. I think they both have their place. Selenium RC is your friend if you need the full-blown power of Python in your Selenium scripts. But sometimes, if only for documentation purposes, it's nice to have equivalent HTML-based tests around. On the other hand, HTML is cumbersome to modify and extend. I think that make_selenium bridges the gap between the "scripted Selenium" world and the "HTML TestRunner" world. In fact, it may be possible to write scripts that work in Selenium RC mode, and modify them minimally so that make_selenium understands them and is able to turn them back and forth, to and from HTML. I know Michał was working on this integration, but I haven't had a chance to test it yet with the current version of make_selenium (maybe Michał can leave a comment with a working example ?) Assuming this integration is working, then you can truly say your code is DRY -- you have one Python script expressing a Selenium test which can be run against a Selenium RC server, or turned into HTML for TestRunner consumption.

The current version of make_selenium is 0.9.5. You can download it from here. I encourage you to read the documentation (which BTW is almost entirely automatically generated from docstrings, in an agile fashion :-) Congratulations to Michał for a fine piece of work!

4 comments:

Martin Blais said...

Now what would be truly awesome would be a python library that implements all the Selenium commands via mechanize...

VISHAL SINGH SACHAN said...

Hi Grig,
I am new with selenium .
I want to automate my testing for one of my web project based in PHP.
I am using Linux and succesfully installed IDE and CORE to my local web.

Now i want to write my own test for my project.
Can you please help me.

I would like to keep in touch with you.

so can you please share your contact info with me.

here is my
vishal.sachan@gmail.com
vishal.sachan@tekritisoftware.com

Joe He said...

Thanks for sharing. It is very helpful.

Biglee said...

hi i am unable to get the verifyHtmlSource functionality. Can you help me out with it?

thanks a lot,
Biglee