Maintaining an Agile

Test Environment

Maintaining an Agile Test Environment

Test automation is certainly an area that more and

Contents
Test automation:
Exploring automation
case studies in Agile
development
Succeeding in test
automation:
Collaboration between
Agile testers and
programmers

more organizations are investing in, executing tests in a

fraction of the time and freeing up testers and other team
members to focus on those tasks that a computer cant
replicate. Some of the most essential skills a tester needs to
succeed with test automation are collaboration and
communication skills. In this E-Guide, readers will hear reallife automation case studies in Agile development and hear
tips of the skills a tester needs to be agile.
Test automation: Exploring automation case studies in
Agile development
By: Yvette Francino
Test automation is certainly an area that more and more organizations are
investing in, executing tests in a fraction of the time and freeing up testers
and other team members to focus on those tasks that a computer cant
replicate. In Experiences with Test Automation, Dorothy Graham and Mark
Fewster have compiled 28 case studies from industry leaders with their
experiences of test automation. Chapter 29 includes a collection of short
experience stories and the Appendix includes a comprehensive table with
information about the 79 tools mentioned in the book. The book starts with a
summary of the common themes and lessons learned.
Authors Dorothy Graham and Mark Fewster discuss test automation and
some of what they learned from these case studies.
SSQ: You originally co-authored another book, Software Test
Automation. Tell us more about that book. What types of automation
and tools were covered?
Dorothy Graham and Mark Fewster: The first book focused on test execution
automation but did not cover any specific tools. Rather, the first half of the
book covered general principles of test execution automation (such as why it
can be good to do, what can go wrong, what to be aware of and how to best

Page 2 of 7

Sponsored by

Maintaining an Agile Test Environment

structure and organize the automation to get the best results). The second
half of the book comprised a number of case studies from other people,

Contents
Test automation:
Exploring automation
case studies in Agile
development
Succeeding in test
automation:
Collaboration between
Agile testers and
programmers

using a variety of tools that were around at the time.

SSQ: How has test automation changed since your original book came
out?
Graham/Fewster: It has become much more prevalent. There is a much
richer range of tools that support testing. More people use automation
(though not necessarily to its best advantage). One of the major changes is
that there are now many open source tools available. Agile development is
also raising the importance of test automation, since test-driven design is by
definition automation-first, at the developer level.
SSQ: Though the book is titled Experiences with Test Automation, it
looks like it covers a huge variety of types of automation and tools.
Does it include experiences with automation of the workflow or the
build, or other parts of the lifecycle outside of test?
Graham/Fewster: Not specifically, although some of the chapters mention
automation as part of the build (continuous integration tools), and modelbased testing is close to requirements analysis. Some of the chapters also
mention performance testing as part of what they did. We are amazed at the
variety and scope of the application of test automation today and are pleased
to be able to share our contributors stories in this book.
SSQ: Todays tools and processes, such as continuous integration,
include automatically executing test suites and then feeding results
back in to development. Do you see such automation strategies
eventually removing the need for test or QA departments?
Graham/Fewster: No. The tools execute tests and check results against an
expected result. The test cases and expected results have to be created and
although some of this can be automated too, good testing is still the domain
of human creativity, so people will always be needed in testing. Whether or
not there is a test or QA department is more a function of how a company is
organized rather than being a technical issue. Some companies may include
automation as part of the responsibilities of a test or QA department. But
tools and automation do not replace people they support people.

Page 3 of 7

Sponsored by

Maintaining an Agile Test Environment

Continuous integration and rapid feedback of results to developers is simply

more efficient practice, whatever department is doing it.

Contents

SSQ: Its very interesting that you, again, include such a variety of

Test automation:
Exploring automation
case studies in Agile
development

experiences, application domains and tools. Obviously, even if we stick

Succeeding in test
automation:
Collaboration between
Agile testers and
programmers

strictly to test automation (rather than extending to automation

throughout the lifecycle) there are many different types of test
automation. Can you list some of those?
Graham/Fewster: Test automation can be done at different levels; for
example, test-driven development (TDD) is done by developers at the unit
test level, but can also be done at system/business/acceptance test level
using tools that interact with the Graphical User Interface (GUI) this is
where many people have used commercial tools to automate regression
tests. Test automation can also be very effective in between these levels,
where the tests interact with APIs or directly to a database, for example.
In additional to automating test execution, tools can also be used to help with
various tasks around the execution, such as setting up pre-requisites for
tests (including for manual tests). This kind of out of the box automation is
described well in Chapter 19 by Jonathan Kohl.
As well as supporting test execution, tools can also help in other types of
testing, particularly performance testing and security testing.
We continue the conversation with Graham and Fewster in How IT leaders
can boost ROI with test automation.

Succeeding in test automation: Collaboration between

Agile testers and programmers
By: Lisa Crispin
What automation skills do you think are most important for Agile
testers?
In my experience, the most essential skills a tester needs to succeed with
test automation are collaboration and communication skills. If you are

Page 4 of 7

Sponsored by

Maintaining an Agile Test Environment

working within a software development organization where there are

programmers writing well-designed production code, those programmers are

Contents
Test automation:
Exploring automation
case studies in Agile
development
Succeeding in test
automation:
Collaboration between
Agile testers and
programmers

the right people to write the automated test code. Testers know the right
things to test. A tester-programmer pair can decide together which tests
should be automated, and how. They should at least start this process before
coding begins, so that the programmer can design the code in a way that will
make it easier to code automated tests.
There are many test frameworks that promote this tester-programmer
collaboration. For example, my own team uses FitNesse for tests at the API
level, behind the GUI. For each user story that can be tested at this level, a
tester gets together with a programmer working on the story to discuss the
test design, and possibly write the first simple test. The programmer writes
the fixture which will take test inputs, pass it to production code which will
operate on it, and return the results to compare to expected results. The
tester can write multiple test cases with different inputs and expected
outputs. We follow a similar process automating GUI tests with Selenium 2.0
and Webdriver. The programmers write page objects for the various GUI
pages in Groovy using a Geb framework, and testers can write multiple test
cases using the page objects. This collaboration helps ensure properly
designed test code, which is maintainable for the long term and provides a
good return on investment.
The programmers on my team feel most comfortable talking to someone who
speaks their language, who has some technical knowledge, who uses the
same or similar integrated development environment (IDE) as they do, who
has a basic grasp of object-oriented concepts. If youre a tester without much
programming experience, start familiarizing yourself with some basic
concepts. A book such as Everyday Scripting with Ruby: For Teams, Testers
and You (Brian Marick, 2006) provides examples you can work through to
develop familiarity with object-oriented concepts, and programming
terminology, even if your team doesnt use Ruby. For some introductory
object-oriented design knowledge, check out Keep it Dry, Shy, and Tell the
Other Guy by Dave Thomas and Andy Hunt. Ask a programmer to help you
install the IDE she uses, and help you create a project where you can check
out your teams production and test code

Page 5 of 7

Sponsored by

Maintaining an Agile Test Environment

If you dont have any access to programmers perhaps you work for a

Contents
Test automation:
Exploring automation
case studies in Agile
development

testing outsourcing company the resources in the above paragraph will get
you started with skills you need to design your own automation. If you cant
pair with a programmer, pair with other testers on your team to learn
together.

Succeeding in test
automation:
Collaboration between
Agile testers and
programmers

Page 6 of 7

Sponsored by

Maintaining an Agile Test Environment

Contents

Free resources for technology professionals

TechTarget publishes targeted technology media that address your need for

Test automation:
Exploring automation
case studies in Agile
development

information and resources for researching products, developing strategy and

Succeeding in test
automation:
Collaboration between
Agile testers and
programmers

webcasts, podcasts, videos, virtual trade shows, research reports and more

making cost-effective purchase decisions. Our network of technology-specific

Web sites gives you access to industry experts, independent content and
analysis and the Webs largest library of vendor-provided white papers,
drawing on the rich R&D resources of technology providers to address
market trends, challenges and solutions. Our live events and virtual seminars
give you access to vendor neutral, expert commentary and advice on the
issues and challenges you face daily. Our social community IT Knowledge
Exchange allows you to share real world information in real time with peers
and experts.

What makes TechTarget unique?

TechTarget is squarely focused on the enterprise IT space. Our team of
editors and network of industry experts provide the richest, most relevant
content to IT professionals and management. We leverage the immediacy of
the Web, the networking and face-to-face opportunities of events and virtual
events, and the ability to interact with peersall to create compelling and
actionable information for enterprise IT professionals across all industries
and markets.

Related TechTarget Websites

Page 7 of 7

Sponsored by