5/31/12

Easy Automated Web Application Testing with Hudson and Selenium | Think Vitamin

Learn with Treehouse

Accessibility, CSS3, Design, Django, HTML & CSS, HTML5, JavaScript, jQuery, NoSQL, PHP, Responsive Web Design, Ruby, Ruby on Rails, Tools, UX,
Version Control, WordPress, iOS and more

Easy Automated Web Application Testing

with Hudson and Selenium
By Ben Rometsch
02 September 2008 | Category: Code

Lets face it, developing web applications is, in some ways, getting more and more complex as time goes on. Sure, there are frameworks like Django, Rails
or Seam to help you get sites built in record time with minimal resources, but at the same time applications are becoming more complex and more
demanding. Being able to support Ajax-heavy applications through multiple browsers across multiple operating systems is now a primary requirement, as is
being able to scale to thousands (or, if youre lucky, millions) of users.

The frameworks help primarily with the initial effort of building the functionality of your site, but they are less useful addressing the longer-term problems
associated with application development. The old school maxim that you spend 90% of your time supporting (as opposed to writing) your application is still
fairly accurate. This article looks at one way of cracking the problem of regression testing (retesting previously working parts of an application following a
new build) a modern web application, using two superb open source projects: Hudson and Selenium.

What will you get out of this article? Our set-up achieves the following:

It checks our Subversion repository every hour to see if anyone has committed any changes.
If they have, it updates the project from Subversion and builds it.
It then creates a clean version of our application database, loads in reference data and deploys the application on our app server.
A job is triggered that runs through a series of tests in a remotely-controlled web browser on the fresh application.
Anything that deviates from the accepted norm is logged and screenshots of the web browser are taken.
Screenshots of the browser are also taken for key pages of the site for later checking by a human.
If any of the tests fail, the developers responsible for the changes are notified by email of the problems.
Our issue tracker is updated with any issues that were fixed in the build.

Heres what the architecture of the setup looks like:

thinkvitamin.com/code/easy-automated-web-application-testing-with-hudson-and-selenium/

1/8

5/31/12

Easy Automated Web Application Testing with Hudson and Selenium | Think Vitamin

The best thing about our set up is that it doesnt rely on a single shell script. Really! For those applications that do need some form of scripting, Hudson
offers relevant hooks in the build process to trigger these easily.

If You Build It, They Will Come

The first piece of the puzzle is Hudson. Hudson is a build server designed for Java applications, but it will work quite happily (albeit with slightly reduced
functionality) with any language you choose to throw at it. Hudson is responsible for checking if your application needs building, building it, triggering any
testing that you have defined and, finally, reporting on the state of the build.

The first step is to download Hudson and install it. To create a new project you need to provide some details:

1. The source code repository that you are using. SVN or CVS support are provided out of the box.
2. Under build triggers choose Poll SCM and in the schedule enter something like 5 * * * *. This is a CRON-like notation that basically means every five
minutes past the hour
: Hudson will check your repository every hour. If it has changed, a build will be triggered.
3. Select a build step. If you are running a Java project using either Ant or Maven, youre in luck, because its as simple as adding the details of the
appropriate script and target here. If not, choose Execute Shell; you can enter shell commands here to carry out your build and deployment tasks.
4. Add an Email Notification post-build action to enable Hudson to notify the developers who were responsible for breaking the build! If youre running a Java
app, you can also include all your JUnit test results in the application reports.

thinkvitamin.com/code/easy-automated-web-application-testing-with-hudson-and-selenium/

2/8

5/31/12

Easy Automated Web Application Testing with Hudson and Selenium | Think Vitamin

The most important aspect of the build step is to get the source code updated and deployed on your test web server. If you arent writing a Java application,
you can still achieve this relatively easily using the Execute Shell build process. At its simplest, it can just copy the files from the Hudson server to the
development server web root.

If the build fails, you will receive an email stating what went wrong, with a link to the Hudson page giving further details. If Hudson can figure out who
committed the offending code, it will even email them too. If the tests failed, you can get good insight into where in your code the failure occurred, helping
you to diagnose exactly what went wrong.

thinkvitamin.com/code/easy-automated-web-application-testing-with-hudson-and-selenium/

3/8

5/31/12

Easy Automated Web Application Testing with Hudson and Selenium | Think Vitamin

Browser Testing Is Boring. Get Someone Else To Do It For You

The second piece of the puzzle is Selenium. Selenium is a bunch of JavaScript craziness (well, they call it a Web application testing system) that is a real
lifesaver. Basically, you give it a bunch of web tests, each consisting of a series of actions to perform on the site, and it replays the commands in a browser
window automatically.

Were going to use Selenium Remote Control, which allows you to set a machine aside as the browser testing box, and control it remotely from your Hudson
build. As if by magic, the browser will start up and run through your recorded tests. Hudson will alert you if anything out of the ordinary appears.

Download Selenium Remote Control, and install it. Once you have Selenium Remote Control up and running, you will want to install the Selenium IDE, a
plug-in for Firefox. The IDE allows you to go onto your site, hit the record button, and then browse around as if you were using the site normally. It records
any user actions you make, like clicking on a button, or entering data into a form field. You can then save these base tests to a file ready for regression
testing.

You can drive Selenium Remote Control with a number of different clients: Java, .NET, Perl, PHP, Python, Ruby and JavaScript are all supported. Export
your Selenium IDE tests into the chosen language, and then hook it up with Hudson as part of the build process. Driving the Selenium server using the
language of your choice means that you have a large amount of control over your testing. You can take the base tests you created using the IDE and then
augment them with conditional logic, loops, whatever you like . As an example, the following method tests one part of our application (in Java):

public void testSendMessage() throws Exception {

try {
selenium.windowMaximize();
selenium.open("/mtb.profilepage.action");
selenium.type("text1", "testuser1@mailinator.com");
selenium.type("name", "123456");
selenium.click("btnLogin");
// Add the second user as a contact
selenium.open("test.user.contact.add.action?&profileId=2");
selenium.open("test.user.contact.approve.action?targetProfileId=1&profileId=2");
selenium.open("/mtb.profilepage.action");
selenium.click("navigation.inbox");
for (int second = 0;; second++) {
if (second >= 60) fail("timeout");
try { if (selenium.isElementPresent("xpath=id('inboxToolbar')/li[1]/a")) break; } catch (Exce
Thread.sleep(1000);
}
assertTrue(selenium.isTextPresent("Ben Rometsch"));
selenium.select("recipientSelect", "label=Ben2 Rometsch2");
selenium.type("subject", "This is the subject");
selenium.type("message_body", "This is the body");
selenium.click("link=Send");
selenium.click("link=Sent");
assertTrue(selenium.isTextPresent("This is the subject"));
selenium.click("link=Sent");
assertTrue(selenium.isTextPresent("Ben2 Rometsch2"));
selenium.click("xpath=id('header')/ul/li[2]/a"); // Log out
} catch (Exception e) {
super.takeScreenshot("error-" + this.getName());
}

thinkvitamin.com/code/easy-automated-web-application-testing-with-hudson-and-selenium/

4/8

5/31/12

Easy Automated Web Application Testing with Hudson and Selenium | Think Vitamin

saveDraftThenSend();
checkMessageReceivedOK();
deleteDraftMessage();
deleteSentMessage();
}
You will also be able to replay your tests with the language IDE you use. If you make use of a debugger, you can also step through your tests to isolate any
problems that might occur.

Selenium has a number of ways of testing that the application is working as intended:

Any HTTP errors (like 404 or 500) will be classified as a failure.

Assertions against text can be made. So, for example, if you are expecting a certain string to be present, you can make a simple assertion stating that it
should be.
Assertions against the DOM structure of your page can also be made using XPath (a killer Firefox addon that can really help with this is XPath Checker)

Do you support IE for Mac?

Regression testing application functionality is one of the main benefits of Selenium and Hudson, but with some tweaking you can also make cross-browser
testing far less painful than it normally is. You will still need to have a real live human being checking things over, but the process becomes much quicker.

Selenium Remote Control has a method available to save a screenshot of the current browser to the file system of the local machine. We can build on this
functionality to have screenshots of the site generated for key pages across multiple browsers, all running from the same host client machine.

The final test we run simply saves key screenshots of the site across four different Selenium browser sessions (Firefox, Internet Explorer, Safari and
Opera). The screenshots are stored onto a shared drive on our development server. The Hudson build number is passed into the Selenium session, and
we make use of this to organize all the screenshots. This way we can wait for a Hudson build to complete successfully, then sanity-check the screenshots
taken in multiple browsers to ensure that nothing has been broken.

Talking to JIRA
We use JIRA to do all our bug tracking. Its really superb. If youre working on an open source project, its also free. Hudson has a plug-in architecture that
means it can be easily extended. One of the available plug-ins connects the two applications.

Once the connection between a Hudson project and a JIRA project is made, the typical bug tracking process becomes:

1. A user finds a bug and raises an issue in JIRA.

2. A developer fixes the bug and commits the code changes to SVN. Crucially, they include the unique JIRA bug ID (which might be something like PIPELINE157) as part of the SVN commit message. So the commit message might read: Fixed PIPELINE-157. Re-factored the email daemon to better handle
multi-part messages
.
3. Hudson recognizes that the new changes have been committed, and performs a build. It also notices the JIRA bug ID in the SVN message and includes a
link to the JIRA issue as part of the build change-set.
4. If the build completes successfully, and there are no failed tests, Hudson posts a comment to the JIRA issue concerned, with a link back to the Hudson build
within which it is fixed.
5. The user that raised the issue receives an email of the message added by Hudson and checks that the new build has fixed the problem. They then close
the bug.

Note how much of this process is automated.

When performing Staging or Production builds, the JIRA integration can help generate a list of fixed issues since the last build.

Below is a change-list from Hudson. The links to things like MTB-211 point back to the JIRA issue.

And here is the comment from the JIRA issue itself.

thinkvitamin.com/code/easy-automated-web-application-testing-with-hudson-and-selenium/

5/8

5/31/12

Easy Automated Web Application Testing with Hudson and Selenium | Think Vitamin

No Silver Bullet
Writing good Selenium tests is tricky. The core assertion you can make are that either text is present, or that it is not. Although that can get you a very long
way, it is no silver bullet your application can break in many ways!

Earlier versions of Selenium also had serious problems with the timing associated with browser-based testing, and required the tester to place a lot of wait
commands within their tests to ensure that page elements had loaded completely before any assertions could be made. A lot of progress has been made in
this area, but the problem rears its ugly head when testing applications that make lots of dynamic page calls. Writing tests that are not horribly brittle when
working on an Ajax-heavy site can be testing (pun intended!). Sometimes, unfortunately, there is no replacement for a human pair of eyes.

All that being said, our time-sheet and invoicing application, Pipeline, has benefited greatly from the set-up described above. Getting the environment and
associated tests set up took about five days worth of work, but the feeling you get when you trap your first bug, automatically, is really great. More
importantly, you can make changes to your code safe in the knowledge that the computers whirring away in the background will be checking your work as
you go.

Follow @thinkvitamin on Twitter

Please check out Treehouse

Other Posts You Might Find Interesting

Sorry - No Related Posts Found

Comments

thinkvitamin.com/code/easy-automated-web-application-testing-with-hudson-and-selenium/

6/8

5/31/12

Easy Automated Web Application Testing with Hudson and Selenium | Think Vitamin

thinkvitamin.com/code/easy-automated-web-application-testing-with-hudson-and-selenium/

7/8

5/31/12

Easy Automated Web Application Testing with Hudson and Selenium | Think Vitamin

Copyright 2006 - 2012 Think Vitamin.

A Blog for Web practitioners Designed, Built and Curated by Carsonified. Feel free to check out our HTML tutorial on Think Vitamin Membership.
Linux Web Hosting by (mt) Media Temple
Contact Us editor@thinkvitamin.com.

thinkvitamin.com/code/easy-automated-web-application-testing-with-hudson-and-selenium/

8/8