Tuesday, January 24, 2006

Useful tools for writing Selenium tests

I've been writing a lot of Selenium tests lately and I've been using some tools that I find extremely useful for composing table-style tests. Let me start by saying that writing GUI-based tests for Web apps is no fun, no matter what your testing tool is. You need to navigate through pages, fill and submit forms, and verify that certain elements are present on the pages. Doing all this manually can quickly become tedious and kill whatever joy you may find in testing.

If you're writing Selenium tests, these activities are, if not fun, at least tolerable due to the existence of the Selenium Recorder ("the Selenium Recorder -- can't imagine life as a Web app tester without it" seems like a good line for a commercial :-)

The Selenium Recorder (which I'll refer to as SelRec from now on) is a Firefox extension that you launch via the Tools menu. It intercepts Web browsing actions such as opening a URL, clicking on a link, entering text in a field form, selecting a drop-down menu item, submitting a form, etc., and it records them in HTML format, ready to be copied and pasted into your Selenium test files. What I find most remarkable about this is that I don't need to worry about identifying HTML elements in order to find a suitable Selenium locator such as an XPath expression, or a DOM expression. For example, SelRec is smart enough to identify links using the simplest locator expression that uniquely identifies a link. If the name of the link is unique, SelRec will use it. If several links have the same name, SelRec will automatically use an XPath expression that uniquely identifies each link.

SelRec can also help with certain Selenium assertion commands. For example, if you highlight the text on a Web page and right-click, you'll see a menu item called "Append Selenium command" which allows you to add one of the following 2 commands to your test table:
  • verifyTextPresent|highlighted text|
  • verifyTitle|title of the current Web page|
You can also append "open|current URL|" to your test table via the "Append Selenium command" menu.

I would guess that most of the testing you're doing with Selenium consists in opening URLs, clicking on various page elements, submitting forms and asserting that some text is present on a Web page or that a Web page has a certain title. Since all these actions and verifications are supported by SelRec out of the box, your life as a tester suddenly becomes a bit easier.

One problem with the "verifyTextPresent" command is that it doesn't support regular expressions. Many times you do need to verify that a specific chunk of text on your page conforms to a regular expression. For this, you need the "verifyText" command, which takes as its first argument the locator of the HTML element whose text you need to verify. This is when things become a bit hairy: you somehow need to identify that element. You can always try to do it in your head by eye-balling the HTML source of the page and trying to come up with the right XPath expression. But fortunately there's an easier way.

At this point, I'm glad to be able to point you to another sanity-saving tool: XPath Checker, which is also a Firefox extension. After installing it, you highlight any text on a Web page, right-click and choose "View XPath". A Firefox window will pop up with an entry field containing an XPath expression, as identified by the tool, and with the text matching that expression. This window allows you to interactively change the XPath expression and see the text that it matches. It's great for playing with and learning XPath (BTW, here is a great XPath tutorial).

Together, the Selenium Recorder and the XPath Checker tools will allow you to easily write the vast majority of your Selenium tests. For more advanced functionality, I refer you to the many Selenium user extensions contributed by people active in the Selenium community. For some nifty Ajax testing (if I may say so myself), see also this blog post of mine.

Note that XPath expressions don't work that well in Internet Explorer. If browser compatibility is high on your list, then you probably need to identify HTML elements via their DOM locations. For that, you can use the DOM Locator extension which ships with Firefox, but you can also peek at the XPath expression generated by the XPath Checker and try to come up with the equivalent DOM expression. Camino, a Mac-specific browser, doesn't seem to have this problem, so you can very easily write your Selenium tests using Firefox with its extensions, then run them in Camino (Titus and I did just that when testing our PyCon tutorial app).

To wrap this thing up, here's an example of using the SelRec and the XPath Checker. Let's assume you want to test some of the search functionality of amazon.com. Note that you can't do this with table-mode Selenium unless you work for amazon.com, because you need to deploy your tests on the amazon.com Web app servers. But for the sake of this example, I just want to illustrate how to use the tools when testing a complex Web site.

Here's a test table that was generated with the Selenium Recorder and that I edited with Mozilla Composer in order to add my own verifyText commands, for which I used XPath expressions that were generated with the XPath Checker:

Amazon Test
open http://www.amazon.com
verifyTitle Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more
type field-keywords python
clickAndWait //input[@type='image' and @alt='Go']
verifyTitle Amazon.com: All Products Search Results: python
verifyText //*[@id='Td:0']/table/tbody/tr/td/table/tbody/tr/td[2]/table/tbody/tr[1] Learning Python, Second Edition by Mark Lutz and David Ascher (Paperback - Dec 2003)
click link=Learning Python, Second Edition
verifyTitle Amazon.com: Learning Python, Second Edition: Books: Mark Lutz,David Ascher
verifyText /html/body/table[2]/tbody/tr/td[2]/div[2] regexp:Buy this book with .* by Alex Martelli today!
verifyText /html/body/table[2]/tbody/tr/td[2]/table[2]/tbody/tr/td/div/ul[1]/li[4] ISBN: 0596002815

The Selenium Recorder can help even when adding your own verifyText commands. Here's how I did it:
  • I highlighted the text I wanted to verify on the Web page under test, then I right-clicked and I chose "Append Selenium command" -> verifyTextPresent "highlighted text"
  • I saved the file generated by SelRec, then I opened it in Mozilla Composer
  • The last row in the file was something like |verifyTextPresent|some text||
  • I edited the row and I replaced "verifyTextPresent" with "verifyText", then I did a cut and paste of "some text" from the second to the third cell
  • I highlighted again the text on the Web page under test, I right-clicked and I chose "View XPath", then I copied the XPath expression generated by the XPath Checker in the second cell of the table row
  • The row now looks something like |verifyText|//some/xpath[2]/expression|some text|
  • At this point, if you need to make sure that "some text" matches a regular expression, you can start the third cell with "regexp:" followed by the regular expression (for example, in the table above, I used the somewhat-contrived "regexp:Buy this book with .* by Alex Martelli today!")
Hope this all helps. I'll soon add post another entry on dealing with frames and pop-up windows in Selenium.

Updates

  • Patrick Lightbody, the maintainer of openqa.org, where Selenium is hosted, reminded me that the Selenium Recorder project is now called the Selenium IDE and is hosted at OpenQA. There is no release of the Selenium IDE yet, but you'll find out when it will be available, I promise :-)
  • Speaking of the upcoming release for the Selenium Recorder/IDE, one feature I'd like to see added is the capability of adding verifyText commands, with the XPath expression corresponding to the highlighted text -- an unification of the XPath Checker functionality with the current verifyText functionality in SelRec
  • Luke Closs points out that he wrote a tool that allows him to write Selenium tests in simple wiki-like tables, then it will convert them into HTML and auto-create a TestSuite.html for him; you can find more info about Luke's Perl-based tool at awesnob.com/selutils.
  • An anonymous commenter points out XPather as another Firefox extension that's superior in his/her opinion to the XPath Checker extension

52 comments:

Luke Closs said...

When doing test driven development, I often write my selenium tests by hand. I don't like writing HTML tables, so I wrote a little tool to allow me to write my selenium tests in simple wiki-like tables, and it will convert them into HTML and auto-create a TestSuite.html for me.

The tool is called selenium-regen, and it is part of the perl package WWW-Selenium-Utils.

More info can be found at awesnob.com/selutils.

Anonymous said...

XPather is a superior XPath extension for Firefox.

Anonymous said...

Nicely done, Grig! Thanks a bunch!

Personally, since I'm a developer, I try to make sure that each element that I want to test has a unique "id" attribute. But I've heard complaints from Selenium users saying that isn't a realistic possibility for many testers (especially when they have no access to edit or influence how the code is written)... So in those cases, XPath (or DOM) expressions to find elements to is the fail-safe option for those users. The downside to XPath is that it isn't always easy to write those XPath expressions... But you're right, with "SelRec" and XPath Checker, the situation is much better! Now we just need to find a tool that can write all those tests automatically!

Anonymous said...

I can't find the DOM Locator for Firefox that you mention...

Grig Gheorghiu said...

Simon -- you need to choose "Custom mode" when installing Firefox; then you need to make sure you check the "Developer tools" box. I think the DOM Inspector doesn't get installed by default. Thanks for bringing this up.

Here's also an article I found on "Debugging CSS with the DOM Inspector".

Anonymous said...

Thanks, Grig, that did the trick.

Anonymous said...

My name is David, and I am doing testing for a web application where I am constantly creating new registrations. At the end of a registration, my application id is printed in text on the page, surrounded by "You have been assigned the following Registration Number: XXXXXXXXXX. Please print a copy of this for your records."

Is there a way to save that text using a selenium IDE command, and then print said text variable to a file?

-David
david.whitesell@gmail.com

Grig Gheorghiu said...

David: you need to identify the element that contains the text you're interested in. You can identify it by HTML id, or by name, or by XPath expression. You can use a Firefox extension such as XPather or XPath Checker to identify it by XPath.

Then you do something like this in your table to store the text in a variable my_text:
|storeText|your_element_id|my_text|

Then you need to write a simple CGI-type script that takes an argument and writes it to a file. You then open the URL corresponding to your CGI script -- something like this:

|open|http://myapp.com/write_to_file.cgi?text=${my_text}||

Hope this helps.

Grig

Anonymous said...

Hi,
Thanks for the post. It's very informative and I've bookmarked it for repeated reference (Along with the other posts that you have published :) )

I noticed you mentioned that for text matching, Selenium does not allow regex.
But on checking the Selenium reference, they do list out 3 different types of pattern matching capabilities for matching string values.
The reference is availaible at http://www.openqa.org/selenium/seleniumReference.html

Am I mistaken or has this functionality been recently added to Selenium?

Grig Gheorghiu said...

aJ,

I said that "verifyTextPresent" doesn't support regular expressions, but "verifyText" does.

Grig

Anonymous said...

Hi,I work with dotnet and I use datalist and data grid in my page that auto generate tables and rows how can I test such this thing that I don't have their ID,I asumme its a posibility to use xpath could you please send me a sample code for that. I want to write my code for this test case.and not using the recorder.

Thanks

Editor said...

I have a feeling that this is a very interesting and useful blog. I'm not a developer and I'd love to understand a bit more... my question is what is a "selenium test" ?

Thanks,
Nicolas
http://blog-dir.blogspot.com/

Anonymous said...

I am new to selenium and I want to use "verifyAllLinks" command to check one page that contains a lot of links. However, I could not make it work. In my case, what is the parttens and values?

Unknown said...

Hi All,

I am facing a problem in Popup window.I want to verify the text present in window and then click on ok of popup window.I tried it by using

assertAlertPresent
Id should be Unique


But it gives result false in this case and unable to click on ok button of popup window.
Also if I use asserAlertNotPresent then it returns true but in this case also it was unable to click on ok button of popup window(message)

Anonymous said...

I am so thankful for your blog entry. I had no idea we could use XPATH in Selenium. Using the XPath tool I was able to solve my problem: I needed to somehow manipulate a nonstandard slider that was not addressable by ID. I struggled for hours to write a user-extension but could not make it work. Whew and thanks!

Anonymous said...

is there any way to check whether an object is a link or not.

Anonymous said...

When using verifyTextPresent of assertTextPresent, once it returns FALSE (highlights red in the table), it halts my test. Is there a way to tell selenium IDE to continue the test if a condition fails? I'd like to be able to test a bunch of pages then look through my table results and see which highlighted red/green, etc. Thanks!

Grig Gheorghiu said...

Denise -- verify commands are supposed to not halt your test (as opposed to assert commands, which do halt it). I don't know why verify behaves the same way as assert in your case. Sounds like a bug in Selenium to me :-)

Grig

Anonymous said...

Ah, thanks Grig! It does work as expected. I have no clue why the other day when I tried using verify___ it was halting though :/

Anonymous said...

selenium IDE 1.0 Beta is available:
http://selenium-ide.openqa.org/
Dick H., an observer

Lindsay Kay said...

Thanks for this page, Grig.

One limitation I've run into with Selenium is that I can't script a right-click with it.

So I'm currently looking for a way around that...any ideas?

cheers,
Lindsay

Lindsay Kay said...

Hi, I was wondering if you had any work around for Seleniums lack of right-click support?

cheers,
Lindsay

Grig Gheorghiu said...

Lindsay -- see http://jira.openqa.org/browse/SEL-381 for more info and links to other discussions on this topic.

Grig

Maxi Ng said...

Hello Grig,
if I do have a DOM locater and know where that element is, what is the format of the expression should I put in the selenium file (table cell).
Because I need to test web apps that have dynamic generated element id.

Lindsay Kay said...

Maxi Ng,

you need to find the element with an XPath that is general enough to resolve to it based on things like its attribute values, index within the set of children of its parent etc.

cheers,
LK

vishal singh sachan said...

Hi I am new with selenium.
I want to automate my registration process .
How can i do this using IDE and CORE

Jyothsna said...

Hi,
Selenium is not identifying the popup.i used
selenium.selectwindow("name=windowid");
where windowid is the id of the popup.In this case the windowid of the popup is dynamically generating which is equal to the sessionid.i tried but its still not identifying the popup and i am not able to performs the actions on those popup controls.i really need it soon please provide me some solution.i will be thankfull to u.

Regards,
Jyothsna

Anonymous said...

Hi,
Great Blog - Thanks!
Is there anyway to use regexp when verifying the title of a page?

Thanks!

Anonymous said...

I want to do parameterization in selenium is there any option or command or any code available to it?
I have a application and I want to add multiple projects in it but every time I have to give different project name and Project Id.
How would i do this in a singe script?
Is there any option to do parameterization/looping in selenium??

Anonymous said...

Hi My name is Nteen:
I want to do parameterization in selenium is there any option or command or any code available to it?
I have a application and I want to add multiple projects in it but every time I have to give different project name and Project Id.
How would i do this in a singe script?
Is there any option to do parameterization/looping in selenium??

Anonymous said...

Useful article to start with Xpath in Selenium.

The only thing to be corrected is - while using the VerifyText command.

1st Column = VerifyText
2nd Column = XPath,
Where XPath should start with // (double back slash).
for example:
verifyText , //html/body/table[2]/tbody/tr/td[2]/a[6] , sample text

Single slash will not work.

Anonymous said...

Hi

Can someone help me with automating web pages navigation when it involves alerts and dialog boxes.
I guess Selenium does not support it.

Please suggest an alternative.

Unknown said...

Hi
I am new to selenium can any one tell me is synchronization possible in selenium if so please let me know how

Lindsay Kay said...

In Java (or whatever language you are coding your tests in if testing programatically) you can wait in a loop (perhaps calling waituntilElementExists) until the target XPath appears, then send your event at it. So you can wrap the selenium proxy (DefaultSelenium) with one that inserts this wait before calls to the wrapped methods. Cheers

Anonymous said...

Hi,

Can I use selenium for automating the testing of GUI of developed forms which should be as per the given mock-up. Testing invloves lots of comparison with the given mock-up.

Thanks,
Girish

Anonymous said...

I have many questions on selenium functionality.. Can u ple send me ur email id on funguyfunguy103@gmail.com

Anonymous said...

I am not finding any useful material from internet. Would you mind to provide book or any training material for Selenium. By which step by step instruction can be found for Selenium.

Grig Gheorghiu said...

Ahmed,

Excuse me for blowing my own horn, but the only Selenium book I know about it one I co-wrote with Titus Brown and Jason Huggins:

"An Introduction to Testing Web Applications with twill and Selenium"

http://oreilly.com/catalog/9780596527808/

HTH

Grig

Lindsay Kay said...

I just wrote an article on how we've been testing an ExtJS GUI in Java at:

http://www.neocoders.com/portal/articles/selenium-and-extjs

Primarily, it describes how we got around having to assign hand-crafted IDs to all the Ext components.

cheers,
Lindsay

Anonymous said...

Hi Grig!!Thanx alot for such a wonderful article...I am new to selenium and exploring it.I have a problem testing with selenium and I hope you could help me out in it.I am testing an application with selenium ide.This application dynamically generates ids and i dont want to write down xpaths manually each time.
I get following id by clicking some button and it changes each time.
e.g. - click gwt-uid-172

I want some kind of expression or may be some way by which i can capture these ids and wont need to change the xpaths each time.How should i go about this?
I hope you could help me out.
Rashmi.

Anand Silodia said...

Hi Grig,

I need some help with Selenium. I need to simulate Control key pressing events. I know there are controlKeyUp() and controlKeyDown() actions in selenium, but they seem not to be working.

What I really want to do is select multiple elements from a grid, for which I need the control press. I tried using Selenium IDE, but even that doesn't capture the ctrl key event.

Do you know what can be done?

vishnuvardan said...

hi grig,

while testing, i am receiving GWT error.
eg, a close button GWT id changes everytime when it is opened.

I need to post properly. please help me!

Anonymous said...

Parameterization requires Hybrid Framework around Selenium there are few available in market, you can try them...

Anonymous said...

hi,
when i am unable to play back the recorded action of clicking on one of the drop down hover menu item.Please give me some solution. the menu used is of ul li tags..unordered list

Anonymous said...

Is anybody know, how it is possible to use Selenium with XC Engine (http://xcengine.googlecode.com/files/xcengine%28minotaur%29-v0.2.zip) test automation framework? I'm looking for some examples, how I can do this.

will said...

I'm using firebug and its extension 'FireXPath' - it's vwery useful to get correct xpath-string.

Unknown said...

Grig,

I am in confusion, i have implemented the selenium automation using perl script. But i come across that no body implemented the automation using perl script,they have used java lot, this is making me worry.I wanted to know which is better one.

Unknown said...

Naveen Says: Your comment is awaiting moderation.
December 3rd, 2009 at 2:13 am

ERROR SERVER EXCEPTION :SESSION ID IS NOT NULL

hi
i am new to selenium .Recently i runned the selenium rc with Nunit i got this exception each time for different projects
can u help me

Thanks in advance
Naveen

Unknown said...

Can "seleninium" be used for automating windows based GUI tool?

Currently we are using RFT. But we are looking out for other options as well. We have developed Our scripts using Java.

Mahwish Khan said...

HI, its mahwish here and i m new at selenium, i m trying to add richtext field to my script but unable to add it.
please help.

i m scripting using html but uinable..plz plz plz help!!!1

Anonymous said...

Hi All,
In my application i am having some links arranged in an Unorderd list.


Cars


Bikes

.
.
.

Is there any way to move the mouse pointer over the Link "Cars".
Note:
mouseOver ("link=Cars"); not working.
Please reply me to judit_raj@rediffmail.com

Ashish Thakur said...

I have over 5 years of experience in selenium testing.
I teach selenium framework with java online.

- First class is demo class
- Java will be taught first and then selenium.
- Syllabus will include java, IDE, RC and GRID

Teachings will be done using skype and WEBEX.

Contact- itsthakur@gmail.com

Modifying EC2 security groups via AWS Lambda functions

One task that comes up again and again is adding, removing or updating source CIDR blocks in various security groups in an EC2 infrastructur...