Ajax testing with Selenium using waitForCondition

An often-asked question on the selenium-users mailing list is how to test Ajax-specific functionality with Selenium. The problem with Ajax testing is that the HTML page under test is modified asynchronously, so a plain Selenium assert or verify command might very well fail because the element being tested has not been created yet by the Ajax call. A quick-and-dirty solution is to put a pause command before the assert, but this is error-prone, since the pause might be not sufficient on a slow machine, while being unnecessarily slow on a faster one.

A better solution is to use Dan Fabulich's waitForCondition extension. But first, a word about Selenium extensions.

If you've never installed a Selenium extension, it's actually pretty easy. You should have a file called user-extensions.js.sample in the same directory where you installed the other core Selenium files (such as TestRunner.html and selenium-api.js). You need to rename that file as user-extensions.js, so that it will be automatically picked up by Selenium the next time you run a test. To install a specific extension such as waitForCondition, you need to download and unpack the extension's zip file, then add the contents of the user-extensions.js.waitForCondition file to user-extensions.js. That's all there is to it.

Now back to testing Ajax functionality. For the MailOnnaStick application, Titus and I used Ian Bicking's Commentary application as an example of Ajax-specific functionality that we wanted to test with Selenium. See this post of mine for details on how Commentary works and how we wrote our initial tests. The approach we took initially was the one I mentioned in the beginning, namely putting pause commands before the Ajax-specific asserts. Interestingly enough, this was the only Selenium test that was breaking consistently in our buildbot setup, precisely because of speed differences between the machines that were running buildbot. So I rewrote the tests using waitForCondition.

What does waitForCondition buy you? It allows you to include arbitrary Javascript code in your commands and assert that a condition (written in Javascript) is true. The test will not advance until the condition becomes true (hence the wait prefix). Or, to put it in the words of Dan Fabulich:

waitForCondition: Waits for any arbitrary condition, by running a JavaScript snippet of your choosing. When the snippet evaluates to "true", we stop waiting.

Here's a quick example of a Selenium test table row that uses waitForCondition (note that the last value in the 3rd cell is a timeout value, in milliseconds):

waitForCondition

var value = selenium.getText("//textarea[@name='comment']"); value == ""

10000

What I'm doing here is asserting that a certain HTML element is present in the page under test. For the Commentary functionality, the element I chose is the text area of the form that pops up when you double-click on the page. This element did not exist before the double-click event, so by asserting that its value is empty, I make sure that it exists, which means that the asynchronous Ajax call has completed. If the element is not there after the timeout has expired (10 seconds in my case), the assertion is marked as failed.

To get to the element, I used the special variable selenium, which is available for use in Javascript commands that you want to embed in your Selenium tables. The methods that you can call on this variable are the same methods that start with Selenium.prototype in the file selenium-api.js. In this case, I called getText, which is defined as follows in selenium-api.js:

Selenium.prototype.getText = function(locator) {
var element = this.page().findElement(locator);
return getText(element).trim();
};

This function gets a locator as its only argument. In the example above, I used the XPath-style locator "//textarea[@name='comment']" -- which means "give me the HTML element identified by the tag textarea, and whose attribute name has the value 'comment'". The value of this HTML element is empty, so this is exactly what I'm asserting in the test table: value == "".

You might wonder how I figured out which element to use in the assertion. Easy: I inspected the HTML source of the page under test before and after I double-clicked on the page, and I identified an element which was present only after the double-click event.

The other scenario I had to test was that the Commentary post-it note is not present anymore after deleting the commentary. Again, I looked at the HTML page under test before and after clicking on the Delete link, and I identified an element which was present before, and not present after the deletion. Here is the waitForCondition assertion I came up with:

waitForCondition

var allText = selenium.page().bodyText(); var unexpectedText = "hello there from user${var}" allText.indexOf(unexpectedText) == -1;

10000

Here I used selenium.page() to get to the HTML page under test, then bodyText() to get to the text of the body tag. I then searched for the text that I was NOT expecting to find anymore, and I asserted that the Javascript indexOf() method returned -1 (i.e. the text was indeed not found.)

Here is the test table for the Commentary functionality in its entirety:

TestCommentary
open	/message/20050409174524.GA4854@highenergymagic.org
dblclick	//blockquote
waitForCondition	var value = selenium.getText("//textarea[@name='comment']"); value == ""	10000
store	javascript{Math.round(1000*Math.random())}	var
type	username	user${var}
type	email	user${var}@mos.org
type	comment	hello there from user${var}
click	//form//button[1]
waitForCondition	var value = selenium.getText("//div[@class='commentary-comment commentary-inline']"); value.match(/hello there from user${var}/);	10000
verifyText	//div[@class="commentary-comment commentary-inline"]	regexp:hello there from user${var}
clickAndWait	//div/div[position()="1" and @style="font-size: 80%;"]/a[position()="2" and @href="/search"]
type	q	user${var}
clickAndWait	//input[@type='submit' and @value='search']
verifyValue	q	user${var}
assertTextPresent	Query: user${var}
assertTextPresent	in Re: [socal-piggies] meeting Tues Apr 12th: confirmed
open	/message/20050409174524.GA4854@highenergymagic.org
assertTextPresent	hello there from user${var}
assertTextPresent	delete
click	link=delete
waitForCondition	var allText = selenium.page().bodyText(); var unexpectedText = "hello there from user${var}" allText.indexOf(unexpectedText) == -1;	10000
assertTextNotPresent	hello there from user${var}
assertTextNotPresent	delete
clickAndWait	//div/div[position()="1" and @style="font-size: 80%;"]/a[position()="2" and @href="/search"]
type	q	user${var}
clickAndWait	//input[@type='submit' and @value='search']
verifyValue	q	user${var}
assertTextPresent	Query: user${var}
assertTextPresent	no matches

For more details on how to identify HTML elements that you want to test using Selenium test tables, see this post on useful Selenium tools, and this post on using the Selenium IDE.

Update: Since images are worth thousands and thousands of words, here are 2 screencasts (no sound) of running the Commentary test with Selenium: one in Windows AVI format, and the other one in Quicktime MOV format (you might be better off saving the files to your local disk before viewing them.)