Tuesday, March 21, 2006

Ajax testing with Selenium using waitForCondition

An often-asked question on the selenium-users mailing list is how to test Ajax-specific functionality with Selenium. The problem with Ajax testing is that the HTML page under test is modified asynchronously, so a plain Selenium assert or verify command might very well fail because the element being tested has not been created yet by the Ajax call. A quick-and-dirty solution is to put a pause command before the assert, but this is error-prone, since the pause might be not sufficient on a slow machine, while being unnecessarily slow on a faster one.

A better solution is to use Dan Fabulich's waitForCondition extension. But first, a word about Selenium extensions.

If you've never installed a Selenium extension, it's actually pretty easy. You should have a file called user-extensions.js.sample in the same directory where you installed the other core Selenium files (such as TestRunner.html and selenium-api.js). You need to rename that file as user-extensions.js, so that it will be automatically picked up by Selenium the next time you run a test. To install a specific extension such as waitForCondition, you need to download and unpack the extension's zip file, then add the contents of the user-extensions.js.waitForCondition file to user-extensions.js. That's all there is to it.

Now back to testing Ajax functionality. For the MailOnnaStick application, Titus and I used Ian Bicking's Commentary application as an example of Ajax-specific functionality that we wanted to test with Selenium. See this post of mine for details on how Commentary works and how we wrote our initial tests. The approach we took initially was the one I mentioned in the beginning, namely putting pause commands before the Ajax-specific asserts. Interestingly enough, this was the only Selenium test that was breaking consistently in our buildbot setup, precisely because of speed differences between the machines that were running buildbot. So I rewrote the tests using waitForCondition.

What does waitForCondition buy you? It allows you to include arbitrary Javascript code in your commands and assert that a condition (written in Javascript) is true. The test will not advance until the condition becomes true (hence the wait prefix). Or, to put it in the words of Dan Fabulich:

waitForCondition: Waits for any arbitrary condition, by running a JavaScript snippet of your choosing. When the snippet evaluates to "true", we stop waiting.

Here's a quick example of a Selenium test table row that uses waitForCondition (note that the last value in the 3rd cell is a timeout value, in milliseconds):

waitForCondition var value = selenium.getText("//textarea[@name='comment']"); value == "" 10000

What I'm doing here is asserting that a certain HTML element is present in the page under test. For the Commentary functionality, the element I chose is the text area of the form that pops up when you double-click on the page. This element did not exist before the double-click event, so by asserting that its value is empty, I make sure that it exists, which means that the asynchronous Ajax call has completed. If the element is not there after the timeout has expired (10 seconds in my case), the assertion is marked as failed.

To get to the element, I used the special variable selenium, which is available for use in Javascript commands that you want to embed in your Selenium tables. The methods that you can call on this variable are the same methods that start with Selenium.prototype in the file selenium-api.js. In this case, I called getText, which is defined as follows in selenium-api.js:

Selenium.prototype.getText = function(locator) {
var element = this.page().findElement(locator);
return getText(element).trim();
};

This function gets a locator as its only argument. In the example above, I used the XPath-style locator "//textarea[@name='comment']" -- which means "give me the HTML element identified by the tag textarea, and whose attribute name has the value 'comment'". The value of this HTML element is empty, so this is exactly what I'm asserting in the test table: value == "".

You might wonder how I figured out which element to use in the assertion. Easy: I inspected the HTML source of the page under test before and after I double-clicked on the page, and I identified an element which was present only after the double-click event.

The other scenario I had to test was that the Commentary post-it note is not present anymore after deleting the commentary. Again, I looked at the HTML page under test before and after clicking on the Delete link, and I identified an element which was present before, and not present after the deletion. Here is the waitForCondition assertion I came up with:

waitForCondition var allText = selenium.page().bodyText(); var unexpectedText = "hello there from user${var}" allText.indexOf(unexpectedText) == -1; 10000

Here I used selenium.page() to get to the HTML page under test, then bodyText() to get to the text of the body tag. I then searched for the text that I was NOT expecting to find anymore, and I asserted that the Javascript indexOf() method returned -1 (i.e. the text was indeed not found.)

Here is the test table for the Commentary functionality in its entirety:

TestCommentary
open /message/20050409174524.GA4854@highenergymagic.org
dblclick //blockquote
waitForCondition var value = selenium.getText("//textarea[@name='comment']"); value == "" 10000
store javascript{Math.round(1000*Math.random())} var
type username user${var}
type email user${var}@mos.org
type comment hello there from user${var}
click //form//button[1]
waitForCondition var value = selenium.getText("//div[@class='commentary-comment commentary-inline']"); value.match(/hello there from user${var}/); 10000
verifyText //div[@class="commentary-comment commentary-inline"] regexp:hello there from user${var}
clickAndWait //div/div[position()="1" and @style="font-size: 80%;"]/a[position()="2" and @href="/search"]
type q user${var}
clickAndWait //input[@type='submit' and @value='search']
verifyValue q user${var}
assertTextPresent Query: user${var}
assertTextPresent in Re: [socal-piggies] meeting Tues Apr 12th: confirmed
open /message/20050409174524.GA4854@highenergymagic.org
assertTextPresent hello there from user${var}
assertTextPresent delete
click link=delete
waitForCondition var allText = selenium.page().bodyText(); var unexpectedText = "hello there from user${var}" allText.indexOf(unexpectedText) == -1; 10000
assertTextNotPresent hello there from user${var}
assertTextNotPresent delete
clickAndWait //div/div[position()="1" and @style="font-size: 80%;"]/a[position()="2" and @href="/search"]
type q user${var}
clickAndWait //input[@type='submit' and @value='search']
verifyValue q user${var}
assertTextPresent Query: user${var}
assertTextPresent no matches


For more details on how to identify HTML elements that you want to test using Selenium test tables, see this post on useful Selenium tools, and this post on using the Selenium IDE.

Update: Since images are worth thousands and thousands of words, here are 2 screencasts (no sound) of running the Commentary test with Selenium: one in Windows AVI format, and the other one in Quicktime MOV format (you might be better off saving the files to your local disk before viewing them.)

40 comments:

Jason Huggins said...

As usual, great work, Grig! :-)

The screencasts truly are worth a thousand words. :-) The syntax of the test (especially the waitForCondition command) is starting to look scary, though. Very powerful, but scary looking-- like PHP or Perl. :-) I'm kinda craving a mini-port of Ruby for the JavaScript "platform" so we can get some of its DSL and "anonymous code block" goodness.

Grig Gheorghiu said...

Jason,

Thanks for the comments. Yeah, it is kind of scary to see all that Javascript code in there, but on the other hand it does offer you tremendous flexibility (so you can shoot yourself in the foot at your ease :-)

Ruby-in-Javascript sounds pretty enticing to me :-)

Anonymous said...

On the first example you need to place a ';' at the end of the first example or it doesn't work.

var value = selenium.getText("//textarea[@name='comment']"); value == "";

Hari said...

Fascination blog..u made my life simpler..
I have a requirement to test the AJAX pages in the web app.

Scenario :

1)Enter a value in a edit box . (Hello)
2) Click on a button which does a partial page refresh or a partial post back.
3)The value entered in the edit box will show up in the page on step (2).
4)Need to check or read this value.(Check for Hello)

sel.waitForCondition("String str=sel.getBodyText();str.indexOf(Hello)!=-1;","30000");
error on running:
com.thoughtworks.selenium.SeleniumException: missing ; before statement

Whts to be done..??

Nagendra said...

In my next project I have to test an ajax application. I am trying to find out the challenges posed by ajax applications w.r.t testing. Will the normal automation tools work or are there any specific tools to test ajax apps? I am also trying to findout why these specific tools are required? Why the already existing tools cant do the job?
Any useful links in this regard will be helpful.

Anonymous said...

So waitForCondition works by polling for a condition, but could some method of hooking in to the AJAX success callback or the general AJAX response handler be feasible?

-Trey

Paul Ingles said...

At my previous company we were using Selenium to test an AJAX-y app.

In the end we put in some code that extended the existing xxxAndWait commands to check that any outstanding requests had been serviced (a la COMs ->AddReference model).

It worked pretty well, and didn't need us to re-implement too much code. Although it was pretty ugly code :)

Dr. Java said...

Thanks, this is good, I was able to use the WaitForCondition command right away after reading your post..


Dr. Java

Anonymous said...

Hi,

In my version of selenium (0.8.1) the function call getText(..) will fail with an error.

I browsed a little through the selenium-browserbot.js and noticed that the function PageBot.findElement() will throw a SeleniumError if the element is not found. Probably this is a change in the newer versions of Selenium.

My quick and dirty fix was to surround the getText() and findElement() calls with try..catch blocks.

Thank you for this usefull post,
Stefan Hornea

Anonymous said...

Check out WebAii automation at www.artoftest.com. Seems like a new Ajax framework for anyone doing .NET testing. Looks like the beta is out and the final release will be sometime mid year...

Vitaliy Shevchuk said...

Wow, people love to make thinks more and more complicated :)

An simpler solution would be to pass your XMLHTTPRequest
from async to sync mode for in-browser testing session. Of course the ajax framework needs to support this some.

Anonymous said...

Very useful, Grig!. I am having trouble with mouseover in selenium.
The page that I am testing has a AJAX call only when you go to the bottom of the page. I get that with mouseover I can do that in selenium, but havent been able to successfully do it.

Any pointers?

alex_f_il said...

The AJAX creates problems for all testing tools, because the calls to XMLHTTPRequest do not generate Browser events - there are no “complete” events for the AJAX and there is no way to get real browser ready status. Without the "complete" events it is hard to create reliable automation solution for AJAX.
To my knowledge only SWExplorerAutomation (SWEA) from Webius(http://webiussoft.com) monitors the Browser network activity to get true browser ready status.

Anonymous said...

Where is selenium-api.js located? I'm using Selenium (installed Selenium+IDE.swf), searched my C:\ for selenium-api.js, nowhere to be found.

Anonymous said...

I am trying to see if Selenium is good for testing my applicaton or not. Am using the Selenium Ide. I want to know where the core files are installed along with the IDE? I need to change the sample files where?

Thanks

Kieran said...

In addition to the problem with findElement failing by throwing an exception, the current version of waitForCondition is out of sync with the current version of Selenium: The "testLoop" object has been renamed to "TestLoop" (with a capital "T"). It's a simple find and replace job to change the waitForCondition code, but it won't work until you make this change.

Kieran said...

Well that was a lot of wasted work: Selenium has since implemented a dedicated waitForElementPresent command, which does exactly what we need waitForCondition to do.

waitForCondition appears to be deprecated.

Anonymous said...

Hi, i am a.Net developer and one of my projects contain a page with two drop down boxes. When a vale is selected in upper drop down the lower drop down is enabled and populated with values corresponding to the selection in upper drop down. This is accomplished using AJAX. But when my selenium test case tries to select the value from the lower drop down box. It throws error that the object/Field (i.e Drop down selection option) not found. Truly so the source code of the page also does not have the Options in it for the lower Drop down so the selenium statement fails...What i tried in this scenario is to use the selenium.Submit(formLocator) function; which too failed coz the options never appeared in lower drop down box

selenium.Type("ctl00_BodyPlaceHolder_txtPostCode_field", "2000"); // Enter 2000 as post code value.

selenium.WaitForCondition("var elements = selenium.getSelectOptions('ctl00_BodyPlaceHolder_ddlSuburb_field'); elements.length > 2", "30000");
// Wait for the drop down to be populated with city names.
selenium.Select("ctl00_BodyPlaceHolder_ddlSuburb_field", "label=SYDNEY");
Select Option with label "Sydney"
selenium.Click("//option[@value='SYDNEY']");

This throws error that optio SYDNEY does not exist.. Rightly so as it is not present in the page source but I can view it on the page (coz of Ajax)..
Could you give your thoughts on this.

Anonymous said...

Hi All,

I am php developer. I was trying to work around for same problem. when i was getting option MUMBAI not found. I am using Selenium IDE. I added a waitForCondition code between 2 calls; i.e. after selecting state Wait till drop down values are populated and then select related city.
Code snippet :


<tr>
<td>select</td>
<td>regions</td>
<td>label=Maharashtra</td>
</tr>
<tr>
<td>waitForCondition</td>
<td>selenium.browserbot.getCurrentWindow().document.getElementById('cities').options.length >= 2</td>
<td>30000</td>
</tr>
<tr>
<td>select</td>
<td>cities</td>
<td>label=Mumbai</td>
</tr>


Hope this helps!

Cheers!

K.T.

chandan said...

hi everybody...
I need an urgent help to autiomate a test case using Selenum(using HTML).I want to declare a variable in "user-extension.js" file, which I will use in automated test cases.
Can anybody suggest how this will be possible?

Vanderhoven Nick said...

@Hari:

use "var str=selenium.getBodyText();str.indexOf('{0}')==-1;"; instead and use String.format to include your text

Anonymous said...

Hi Grigm,
Regarding the 'WaitForCondition', I have one question, would like to help me?

I use Selenium RC(C#) to develop the test script.And the AUT is a ajax web application.
There is a scenario, after the user click one button, saying button1, a drop down listbox will be filled in, it is ajax feature, which means the page will not be re-loaded totally,only the drop down listbox will be filled in.
And the button clicking tragger a javascript funtion to fill in the drop down listbox.
So I write the test script.
1. selenium.Click("Button1");
2, selenium.WaitForCondition(,10000);
I do not know what should be written as the first parameter of the WaitForCondition.
I know it should be javascript.
But I don't know how to invoke the javascript in the page's html.

Can you help me with it?
Please send the answer to my mail
Grrison.W.Wang@gmail.com

Thanks in Advance.

Anonymous said...

hi,

need advice on using selenium.getText with xpath location in C# via Nunit. It does not seem to work. any ideas?

Anonymous said...

hello!!

we nid help in using selenium..
we installed selenium ide..
how can we make a value generic?
coz it nids to be edited the value manually so that the case will be successful. It creates error when we dont change or put the specific value..
Do we nid variable?
if yes.. how?
pls help us..

thank's
cath, har & joy

Daniel Abella - abellad@gmail.com said...

This post is wonderful! Congratulations ;)

falcy said...

first of all let me thank you for all your selenium intel! it helped getting me started with the whole thing. i have a question though: your selenium posts are not the most recent and i was wondering if selenium is still the tool of your choice or if you're using sth. else now?

falcy said...

uh, me again! :)
is waitForCondition still the way to go with AJaX or is waitForElementPresent alternative by now?

Grig Gheorghiu said...

Hi falcy,

I haven't used Selenium for a while, so I'm a bit out of touch with the latest developments. From what I know though, it is being used heavily at Google, so that tells you Selenium is doing *something* right ;-)

As far as waitForCondition vs. waitForElement, I think it's better to use waitForElement, since it doesn't force you to use that much custom JavaScript in your tests.

Grig

falcy said...

hey grig,

thank you for responding! :)

I think it's better to use waitForElement, since it doesn't force you to use that much custom JavaScript in your tests.

i was thinking along exact those lines! :D

- f -

NaTeia said...

When I recorded an script with alerts the Selenium IDE save the command assertAlert with the message
of that alert in target.

When I play this script this command fails and the button ok isn't pressed?

How I can replace assertAlert with another command that works?

Thanks so much

Bruno

Prashant Dathwal said...

I have a simple login page written in extension JS in following way:
Ext.onReady(function()
{
Ext.QuickTips.init();
Ext.form.Field.prototype.msgTarget = 'side';
new Ext.Button({
renderTo:'login',
text: 'Login',
width: 800
}).on('click',function()
{
document.forms[0].submit();

});
});

Now when I use selenium testing via browser I get a randomly generated id which when hard coded can very well be used in following way:
selenium.click("ext-gen26");

My concern is how to get reference of this generated id, so my selenium test case in JUnit is not dependent on randomly generated ids from Extension JS?

Your suggestions appreciated.

Regards,
Prashant

Anonymous said...

selenium is a piece of shit. the IDE exports tests that are completely useless in Java. I get timeout errors all the time and there is no documentation. Google needs to fire the lead developer on this. waitForElementPresent apparently is part of the api but somehow it's not listed in the intellisense that eclipse has. Stupid software = I won't use it.

Romulus said...

This post is going in my file of things to send people who insist that Selenium is "broken" whenever this scenario occurs. Thank you for this.

Banty said...

A good post.

Banty said...

A very good post.

Dave747 said...

Hi there,

There are loads of useful information but my question is why a timeout must be set when using the waitForCondition command ?? If we get back to the initial problem when using a pause command to wait for some action requiring AJAX to take place, then isn't it the same approach when using the waitForCondition??

Rahul said...

I had a problem with gotoIF statement.i created atest in IDE and its working correctly there.
if i now export the code in C# RC server format, then the gotoIf line is commented out and also is label commented out.

I had installed the plugin for Selenium IDE Flow Control
Extension.from: https://github.com/darrenderidder/sideflow
and restarted the firefor version5.
then when i started IDE the gotoIF command was available for use. I
did used it also. but when I exported the Code as c#(Remotecontrol)
format.and opened it in Microsft visual studio 2010. All the gotoIF
statements are not recognized by the Microsft visual studio 2010. all
statements are commented out.
and
code looks like this when opened in studio:

selenium.Click("css=input[type=submit]");
Boolean TP = selenium.IsTextPresent("User name already exists");
System.Threading.Thread.Sleep(sleeptime);
// selenium.GotoIf("${TP}==false", "target1");
System.Threading.Thread.Sleep(sleeptime);
// selenium.Label("target1", "UseName");
System.Threading.Thread.Sleep(sleeptime);

How can i fix this issue?If i remove the comments// then it says: you
are missing a directive or assembly reference.

And if i try to add reference to the sideflow.js file it is not
detected.because it is not a .DLL file.
Anyone knows how to fix this?

Thanks in Advance!! :)

Rahul.

Anonymous said...

I download the user-extension and put it in right place, while executing a testing script including 'waitForCondition', throws an exception :"The error message is :'testLoop' is undefined"

Anonymous said...

Hi All,
I download the user-extension and put it in right place, while executing a testing script including 'waitForCondition', throws an exception :"The error message is :'testLoop' is undefined"

Thanks in ahead

Marcio Santos said...

Someone who is using RichFaces and component can wait for this condition:

String condition = "var display = selenium.browserbot.getCurrentWindow()";
condition += ".document.getElementById('ajaxLoadingModalBoxContainer').style.display;";
condition += "(display == 'none');";
selenium.waitForCondition(condition, 10000);