- Running Selenium WebDriver tests using Firefox headless mode on Ubuntu
- Running headless Selenium WebDriver tests in Docker containers
In this post I will show how to add the final piece to this workflow, namely how to fully automate the execution of Selenium-based WebDriver tests running Firefox in headless mode in Docker containers. I will use Jenkins for this example, but the same applies to other continuous integration systems.
1) Install docker-engine on the server running Jenkins (I covered this in my post #2 above)
2) Add the jenkins user to the docker group so that Jenkins can run the docker command-line tool in order to communicate with the docker daemon. Remember to restart Jenkins after doing this.
3) Go through the rest of the workflow in my post above ("Running headless Selenium WebDriver tests in Docker containers") and make sure you can run all the commands in that post from the command line of the server running Jenkins.
4) Create a directory structure for your Selenium WebDriver tests (mine are written in Python).
I have a directory called selenium-docker which contains a directory called tests, under which I put all my Python WebDriver tests named sel_wd_*.py. I also have a simple shell script I named run_selenium_tests.sh which does the following:
#!/bin/bash
TARGET=$1 # e.g. someotherdomain.example.com (if not specified, the default is somedomain.example.com)
for f in `ls tests/sel_wd_*.py`; do
echo Running $f against $TARGET
python $f $TARGET
done
My selenium-docker directory also contains the xvfb.init file I need for starting up Xvfb in the container, and finally it contains this Dockerfile:
FROM ubuntu:trusty
RUN echo "deb http://ppa.launchpad.net/mozillateam/firefox-next/ubuntu trusty main" > /etc/apt/sources.list.d//mozillateam-firefox-next-trusty.list
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CE49EC21
RUN apt-get update
RUN apt-get install -y firefox xvfb python-pip
RUN pip install selenium
RUN mkdir -p /root/selenium/tests
ADD tests /root/selenium/tests
ADD run_all_selenium_tests.sh /root/selenium
ADD xvfb.init /etc/init.d/xvfb
RUN chmod +x /etc/init.d/xvfb
RUN update-rc.d xvfb defaults
ENV TARGET=somedomain.example.com
CMD (service xvfb start; export DISPLAY=:10; cd /root/selenium; ./run_all_selenium_tests.sh $TARGET)
I explained what this Dockerfile achieves in the 2nd post referenced above. The ADD instructions will copy all the files in the tests directory to the directory called /root/selenium/tests, and will copy run_all_selenium_tests.sh to /root/selenium. The ENV variable TARGET represents the URL against which we want to run our Selenium tests. It is set by default to somedomain.example.com, and is used as the first argument when running run_all_selenium_tests.sh in the CMD instruction.
At this point, I checked in the selenium-docker directory and all files and directories under it into a Github repository I will call 'devops'.
5) Create a new Jenkins project (I usually create a New Item and copy it from an existing project).
I specified that the build is parameterized and I indicated a choice parameter called TARGET_HOST with a few host/domain names that I want to test. I also specified Git as the Source Code Management type, and I indicated the URL of the devops repository on Github. Most of the action of course happens in the Jenkins build step, which in my case is of type "Execute shell". Here it is:
#!/bin/bash
set +e
IMAGE_NAME=selenium-wd:v1
cd $WORKSPACE/selenium-docker
# build the image out of the Dockerfile in the current directory
/usr/bin/docker build -t $IMAGE_NAME .
# run a container based on the image
CONTAINER_ID=`/usr/bin/docker run -d -e "TARGET=$TARGET_HOST" $IMAGE_NAME`
echo CONTAINER_ID=$CONTAINER_ID
# while the container is still running, sleep and check logs; repeat every 40 sec
while [ $? -eq 0 ];
do
sleep 40
/usr/bin/docker logs $CONTAINER_ID
/usr/bin/docker ps | grep $IMAGE_NAME
done
# docker logs sends errors to stderr so we need to save its output to a file first
/usr/bin/docker logs $CONTAINER_ID > d.out 2>&1
# remove the container so they don't keep accumulating
docker rm $CONTAINER_ID
# mark jenkins build as failed if log output contains FAILED
grep "FAILED" d.out
if [[ $? -eq 0 ]]; then
rm d.out
exit 1
else
rm d.out
exit 0
fi
Some notes:
- it is recommended that you specify #!/bin/bash as the 1st line of your script, to make sure that bash is the shell that is being used
- use set +e if you want the Jenkins shell script to continue after hitting a non-zero return code (the default behavior is for the script to stop on the first line it encounters an error and for the build to be marked as failed; subsequent lines won't get executed, resulting in much pulling of hair)
- the Jenkins script will build a new image every time it runs, so that we make sure we have updated Selenium scripts in place
- when running the container via docker run, we specify -e "TARGET=$TARGET_HOST" as an extra command line argument. This will override the ENV variable named TARGET in the Dockerfile with the value received from the Jenkins multiple choice dropdown for TARGET_HOST
- the main part of the shell script stays in a while loop that checks for the return code of "/usr/bin/docker ps | grep $IMAGE_NAME". This is so we wait for all the Selenium tests to finish, at which point docker ps will not show the container running anymore (you can still see the container by running docker ps -a)
- once the tests finish, we save the stdout and stderr of the docker logs command for our container to a file (this is so we capture both stdout and stderr; at first I tried something like docker logs $CONTAINER_ID | grep FAILED but this was never successful, because it was grep-ing against stdout, and errors are sent to stderr)
- we grep the file (d.out) for the string FAILED and if we find it, we exit with code 1, i.e. unsuccessful as far as Jenkins is concerned. If we don't find it, we exit successfully with code 0.
No comments:
Post a Comment