Using Selenium to test a Plone site (part 1)

I will give an example of using Selenium to test a Plone site. I will use a default, out-of-the-box installation of Plone, with no customizations. The functional test I'll present is written as an HTML table. In this post, I'm using Plone only as an example of a Web application. The test table I present uses generally-available "Selenese" commands that are not specific to Plone, so this example can be used as a mini-tutorial for writing HTML table-based tests in Selenium. If you want to see how Selenium can be used in conjunction with a stand-alone Twisted-based server, read this post.

Update: I just found out that Jason Huggins checked in a lot of Plone-specific Selenium code today, so obviously I haven't had the chance to experiment with that yet. I will follow up this post with another one, more targeted to specific features available in the Plone product version of Selenium (setUp, tearDown methods, and postResults for summarizing the test run).

Selenium is under very active development and new and important features are added to the tool on a daily basis. For this post, I checked out via subversion the latest version of the code as of March 8th.

Installing Selenium as a Plone product

One of the implementations of Selenium is a Plone product. I'll give a short procedure for installing Selenium inside an existing Plone site. In my case, I'm running Plone2-2.0.3-2 installed as an RPM on a Red Hat 9 server. My main Plone site is under /var/lib/plone2/main/.

1. Check out Selenium via subversion from svn://selenium.codehaus.org/selenium/scm/trunk
(I'll call SELENIUM_ROOT the directory where you checked out the code. In my case, it is /usr/local/selenium)

2. Copy SELENIUM_ROOT/code/python/Zope/Selenium to the Products directory of your Plone installation. In my case, I ran:

cp -r /usr/local/selenium/code/python/Zope/Selenium /var/lib/plone2/main/Products

3. Copy all the files and directories under SELENIUM_ROOT/code/javascript to the Products/Selenium/skins/selenium_javascript directory. In my case, I ran:

cp -r /usr/local/selenium/code/javascript/* /var/lib/plone2/main/Products/Selenium/skins/selenium_javascript

4. Go to the Products/Selenium/skins/selenium_javascript directory of your Plone site and rename all the .html, .js and .css files to their original name plus .dtml. A small Python script is provided in one of the readme files. Here it is (I called it rename_files.py):

import os
from os.path import join

# We need to append ".dtml" to all html, js, and css files so the "original" filename can be
# called in the browser.
# For example:
# By default, in Zope, TestRunner.html would be available from a URL as "http://localhost/TestRunner" with
# no ".html" attached. To preserve the original file name, we append ".dtml" to the file name.

for root, dirs, files  in os.walk(os.getcwd()):
    if '.svn' in dirs or 'CVS' in dirs:
        dirs.remove('.svn') # don't visit Subversion or CVS directories

    for file in files:
        if file.endswith('.html') or file.endswith('.js') or file.endswith('.css'):
            old_file = join(root, file)
            new_file = old_file + '.dtml'
            os.rename(old_file,new_file)

In my case, I cd-ed into /var/lib/plone2/main/Products/Selenium/skins/selenium_javascript and ran python rename_files.py.

5. Restart your Plone instance (this will need to be done every time you change anything on the file system, unless you start up Zope in debug mode). In my case, I ran:

/etc/rc.d/init.d/plone2 restart

6. Use the Plone QuickInstaller to install the Plone product.

• Login to Plone as the administrator user -- not into the Zope Management Interface (ZMI), but into the Plone interface
• Click on the "My Preferences" link in the navigation bar
• Click on "Add/Remove products" in the "Plone setup" portlet
  
• Select Selenium from the products available for installation and click Install
  

7. Point your browser to http://PLONE_ROOT_URL/TestRunner.html. In my case, PLONE_ROOT_URL is www.example.com:8080/Plone, so I pointed my browser to http://www.example.com:8080/Plone/TestRunner.html (example.com is of course not the real domain name for my Web site)

You should now see the default TestSuite shipped with Selenium. Feel free to click on the tests in the upper-left frame, then click on the green "Selected test" button to run the test. Playing with the tests in the default TestSuite is a good way to learn what kind of actions and checks are available in Selenium.

Creating a custom Test Suite and adding a test table

A non-documented feature of Selenium is that you can run test suites other than the default TestSuite.html by passing an argument to TestRunner.html. In my case, I created an HTML file called CustomTestSuite.html.dtml in the tests subdirectory of selenium_javascript (the full path to this directory is in my case /var/lib/plone2/main/Products/Selenium/skins/selenium_javascript/tests). Note that although CustomTestSuite is an HTML file, you need to end its name with .dtml, otherwise Plone will strip the .html portion from its name and interfere with the internal Selenium logic.

My CustomTestSuite.html.dtml file contains a single table:

Custom Test Suite

TestNewUser

TestNewUser is a link to tests/TestNewUser.html.dtml.

I then created a file called TestNewUser.html.dtml in the same tests directory. This file contains a table with only one row:

Test New User

To have Selenium load my custom test suite, I restarted Plone, then pointed my browser to:

http://www.example.com:8080/Plone/TestRunner.html?test=./tests/CustomTestSuite.html

This loaded the table from CustomTestSuite.html.dtml in the top left frame of the browser.

I then started to fill the table in TestNewUser.html.dtml by adding "Selenese" commands as rows. I'll show here the final version of the table:

Test New User
setVariable	base_url	'http://www.example.com:8080/Plone'
setVariable	logout_url	'${base_url}/logout'
setVariable	join_url	'${base_url}/join_form'
open	${logout_url}
open	${base_url}
verifyTextPresent	Welcome to Plone
click	//a[@href='${join_url}']
verifyTitle	Portal - Please sign in
verifyLocation	join_form
verifyTextPresent	Registration Form
verifyValue	fullname
type	fullname	Test User Full Name
verifyValue	username
setVariable	random_user	'user'+(new Date()).getTime()
type	username	${random_user}
verifyValue	email
type	email	${random_user}@example.com
verifyValue	password
type	password	testUserPassword
verifyValue	confirm
type	confirm	testUserPassword
click	form.button.Register
verifyTextPresent	You have been registered as a member
click	//input[@value='Log in']
verifyTextPresent	You are now logged in
click	link:set up your Preferences
verifyLocation	/plone_memberprefs_panel
verifyTextPresent	My Preferences
click	//img[@src='user.gif']
verifyLocation	/personalize_form
verifyTextPresent	Personal Preferences
select	wysiwyg_editor	Epoz
click	listed
click	form.button.Save
verifyTextPresent	Your personal settings have been saved
verifyValue	wysiwyg_editor	Epoz
verifyValue	listed	off

This is a pretty long test that involves navigating through 6 or 7 pages. In a real-life testing situation this table should probably be split into several smaller tables, each one testing a specific piece of functionality. For the purpose of this tutorial I wanted as many various Selenese commands as I could fit into a still reasonably-sized table.

As it stands, the TestNewUser table tests the following functionality of a default, out-of-the-box Plone installation:

• log out the existing user, if any
• click the "New User?" button
• fill in the registration form and save it
• click the "log in" button
• go to the My Preferences page
• go to the Personal Preferences page
  
• edit some of the preferences and save the form
  
• check that the edited preferences were correctly saved

Instead of discussing the test table row by row, I'll discuss types of commands such as clicking on links, entering text, selecting value, clicking buttons, validating elements, etc. and I'll refer to the rows which use these commands.

Using variables

A brand-new feature of Selenium is the ability to use variables directly in the test tables. In the official documentation, the only way to deal with variables is via separate HTML pages where these variables are set. In the TesNewUser table, I'm using the setVariable command to set 3 variables: base_url, join_url and logout_url. The syntax for setting a variable is:

setVariable

var_name

var_value

(if the value is a string, it needs to be enclosed in quotes)

The syntax for getting the value of a variable is: ${var_name}

Note that join_url and logout_url use the value of base_url via interpolation, by enclosing the expression containing the variable in quotes:

setVariable

logout_url

'${base_url}/logout'

An important use of a variable, especially when testing Web sites that provide log in functionality, is to set a random user ID that will be used in the test. TestNewUser does this via:

setVariable

random_user

'user'+(new Date()).getTime()

Note that in this case the value for the random_user variable is obtained by concatenating a string ('user') with the value returned by a JavaScript function (the getTime() method for a Date object). So you can use JavaScript code in your variable assignments.

Opening Web pages by URL

Web pages can be opened by their URL via the open command. An example is:

open

${logout_url}

This particular "log out" command is the first real action in the TestNewUser table. It is necessary because after creating a user and logging in, that user will still be logged in when running the test table again. In Plone, the home page for a logged-in user is different from the home page of a non-logged in user, and I wanted to test the former situation.

Clicking on links

This is one of the trickiest aspects of writing tests in Selenium. In general, HTML elements on the Web pages you're trying to test can be referred to in Selenium commands via "element locators", which can be one of the following:

• identifiers: the id or name attribute of the element
• DOM traversal syntax: document.forms['myForm'].myDropdown
• XPath syntax: //img[@alt='The image alt text']

Life is easy when HTML elements such as links have id attributes such as id="the_link_id". In this case, the command you need to use for clicking on the link is simply:

click

the_link_id

Life is also pretty easy when links have text that is on the same line with the starting a tag and the closing /a tag. In this case, you can use the following XPath syntax (a good XPath tutorial recommended by Ian Bicking is here):

click

//a[text() = "the link text]

A recently introduced Selenium command for accessing links by their text is the link: command:

click

link:the link text

Note that you need to follow link by a colon, then immediately by the text of the link with no quotes. I used this command in the TestNewUser table like this:

click

link:set up your Preferences

In other cases, especially when the link text is on a line by itself or spans multiple lines, the text method will not work and you will need to identify the link by some other attributes. Here is an example from TestNewUser:

click

//a[@href='${join_url}']

This represents the command for clicking on the "New user?" link at the bottom of the "log in" portlet on the Plone home page. I initially tried to identify the link by the "New user?" text, but that method didn't work, because the starting a tag, the link text and the closing /a tag were on different lines. The only solution I found was to identify the link by its href tag. The XPath syntax I used was //a[@href='url'] where url is identified by the value of the variable ${join_url}.

Here is another example from TestNewUser:

click

//img[@src='user.gif']

This represents the command for clicking on the Personal Preferences link on the "My Preferences" page. Again the text method didn't work, so this time I identified the link via the src tag of its image.

Clicking on submit buttons

The command you need to use for clicking on form submit buttons is click. The target of the command is the element locator for the button. This can be either the button's name attribute or its value attribute. An example of clicking a submit button by its name in TestNewUser is:

click

form.button.Register

This represents the command for clicking on the "register" button at the bottom of the new user registration form.

Here is an example of clicking a submit button by its value, using an XPath expression:

click

//input[@value='Log in']

This represents clicking on the "log in" button on the Welcome page that is shown immediately after a successful registration.

Clicking on check boxes

The click command again accomplishes this. You need to indicate the name of the check-box field as the target of the command. The click command, when applied to a check box, toggles the value of that box. Here is an example from TestNewUser, where the check box for "Listed in searches" on the Personal Preferences page is clicked. The name of that box is "listed":

click

listed

Note that the click command can also be used for clicking on radio buttons.

Entering text in input fields

If you need to fill in input fields in forms, use the type command, which takes 2 arguments: the name of the input field (you'll need to figure what that name is by inspecting the HTML source) and the value you need to type in that field. Here are some examples from TestNewUser, for filling in the user name and email information on the registration form:

type

username

${random_user}

type

email

${random_user}@example.com

Selecting values in drop-down lists

This is accomplished via the select command, which also takes 2 arguments: the name of the drop-down list field and the value you need to select in that list. An example from TestNewUser shows how to select the Epoz editor in the Personal Preferences page:

select

wysiwyg_editor

Epoz

Verifying the state of the application

So far I have shown how Selenium can drive an embedded browser via "Selenese" commands. This is only one aspect of testing a Web application, since it indirectly verifies that the HTML elements it expects to click or open are indeed present. For direct verifications, Selenium provides a variety of "verify" commands that check the values of the different elements under test. Perhaps the simplest check is verifyTextPresent, which makes sure that a given snippet of text is present in a Web page. This example from TestNewUser checks that the default Plone home page contains "Welcome to Plone":

verifyTextPresent

Welcome to Plone

Values for elements of a form (input fields, drop-down lists, check boxes) can be verified via the verifyValue command:

verifyValue

wysiwyg_editor

Epoz

verifyValue

listed

off

(note that for a check box such as "listed", the value that we compare with is either off or on)

To verify that a Web page contains a specific URL, use either verifyAbsouteLocation (which checks that the URL of the page is identical with a given string) or verifyLocation (which checks that the URL of the page ends with a given string).

This example from TestNewUser checks that the URL of the new user registration page ends with /join_form:

verifyLocation

join_form

There are many other types of check commands available on the Selenium TestRunner reference page.

To be continued...

As I mentioned before, new and exciting features are added to Selenium on a daily basis. If you're interested in the tool and want to see what is going on with its development, browse the selenium-devel mailing list archive and consider joining the list and contributing.

I intend to follow up this article with other Selenium-related posts that will cover things such as:

• using wildcards and regular expressions in verification commands
• embedding JavaScript code in Selenese commands
• organizing and reusing tests
  
• using Plone-specific features such as Setup, TearDown and PostResults pages