Sunteți pe pagina 1din 36

January/February 2013



Precision is key


A long walk for an

important lesson

Test Studio

Easily record automated tests for

your modern HTML5 apps

Test the reliability of your rich, interactive JavaScript apps with just a few
clicks. Benefit from built-in translators for the new HTML5 controls, crossbrowser support, JavaScript event handling, and codeless test automation
of multimedia elements.

Volume 15, Issue 1 JANUARY/FEBRUARY 2013







Lessons learned long ago from reviews and inspection can be effective today,
particularly in collaboration within agile teams. Learn how an organization used
review techniques as part of its agile collaboration, including the advantages
and potential problems of this ancient wisdom.
by Robert Sabourin and Dorothy Graham





Test planning is often thought unnecessary in an agile project. However, if our

mindset is on planning rather than plans, we see that test-planning activities happen throughout the project, taking advantage of levels of precision
i.e., what is absolutely necessary at each level.
by Janet Gregory




Agile developers see frequent releases as a way to mitigate risk, while production control and upper management see frequent releases as unstable and
high risk. DevOps addresses this problem by simplifying the hand-off of software releases between development and production control.
by Tracy Ragan



in every issue
Mark Your Calendar
Editor's Note

Dave and his friend, Bob, hiked Wisconsins Ice Age Trail and returned home
with more than just sore legs and hiking experience. Learn some of the project
management tips Dave picked up while adventuring in the wilderness.
by Dave Katauskas


From One Expert

to Another 13
Techwell Spotlight 25


FAQ 31
Ad Index 33

or call 800.450.7854.

by Jonathan Kohl
Skilled testingan important part of creating valuable technologyis under
attack. We need to rethink the problem, analyze why people resist testing, and
create approaches to make it more engaging. Gamification provides one model
to help us improve our approach.

Announcements 29

Better Software magazineThe

companion to brings you the
hands-on, knowledge-building information
you need to run smarter projects and deliver
better products that win in the marketplace
and positively affect the bottom line.
Subscribe today to get six issues.





by Michael Bolton
There are people who believe that emotions have no place in software testing
when, in fact, the opposite is true. Decisions about quality are always emotional. If you want to be a better tester, get in touch with your feelings.


software tester

Advanced Certification Training

February 25March 1, 2013
San Francisco, CA
Foundation Certification Training
February 1921, 2013
Atlanta, GA

training weeks

Software Quality Engineering, Inc.
Wayne Middleton
Vice President of Communications
Heather Buckman
Publications Manager
Heather Shanholtzer

February 2628, 2013

Houston, TX
Sacramento, CA

Testing Training Weeks

March 1822, 2013
Boston, MA

March 57, 2013

Denver, CO
Philadelphia, PA

April 2226, 2013

San Diego, CA

Better Software Editor

June 1014, 2013

Chicago, IL

Online Editors
Jonathan Vanian
Noel Wurst
Community Manager
David DeWald

March 1214, 2013

Tucson, AZ
LA Area, CA

Requirements Training Weeks

March 1115, 2013
San Diego, CA

March 1820, 2013

Boston, MA

Agile Software Development

Training Weeks
June 24, 2013
Las Vegas, NV

March 1921, 2013

Vienna, VA
Columbus, OH

April 711, 2013
Delta Chelsea
Toronto, ON

September 29October 4, 2013
Disneyland Hotel
Anaheim, CA

April 28May 3, 2013
Rosen Shingle Creek
Orlando, FL

Better Software Conference East
November 1015, 2013
Sheraton Boston Hotel
Boston, MA

Better Software Conference West
June 27, 2013
Caesars Palace
Las Vegas, NV

Agile Development Conference East
November 1015, 2013
Sheraton Boston Hotel
Boston, MA

Agile Development Conference West
June 27, 2013
Caesars Palace
Las Vegas, NV


Joseph McAllister

Production Coordinator
Cheryl M. Burke
Creative Director
Catherine J. Clinger




Sales Consultants
Daryll Paiva
Kim Trott
Production Coordinator
Desiree Khouri

Subscriber Services:
Phone: 904.278.0524, 888.268.8770
Fax: 904.278.4380
Better Software magazine
Software Quality Engineering, Inc.
340 Corporate Way, Suite 300
Orange Park, FL 32073

Editors Note

Season Cycle Moving Round



Welcome to the first Better Software magazine issue of 2013! December is

nearing its end as I write this. In fact, today is the winter solstice, and the lateafternoon sunset outside my window signals the coming of the longest night of
the year in the Northern Hemisphere. Starting tomorrow, our days will get brighter
and brighterat least for a while.
Even though its somewhat arbitrary to say that the year ends in December and begins in January, there still is something about this time of year that helps us focus on trading the old for the new. Perhaps the dark nights and the holidays
offer us a little extra time to ourselves to think about change. For some, it may be a time of self-transformationperhaps
learning a new skill or starting a new project. For others, it may be an opportunity to give or request forgiveness. We have
come through the harvest, and the earth is quietly preparing for the inevitable spring.
As we look forward, lets not forget to remember the past, too. In this issues cover story, Robert Sabourin and Dorothy
Graham look back to the ancient days of softwarethe 1970sto explore how the forgotten art of software inspections can be applicable to todays software projects. In her article, Janet Gregory explores how the traditional practice of
test planning can apply to agile projects through collaboration, simplicity, precision, and visibility. And, Dave Katauskas
shares the lessons he learned firsthand while recently hiking Wisconsins Ice Age Wilderness Trail and applies them to
death march projects.
What will you remember from 2012? What are you anticipating in 2013? No offense to the Mayans or the band R.E.M., but
Im looking forward to a year with fewer Its the End of the World as We Know It headlines.
Im also looking forward to a year in which the STAR conferences acquire a new sibling in the form of STARCANADA in
Toronto. In fact, some of the authors featured in this issue of Better Software will also be presenting at STARCANADA and
STAREAST in 2013. So, after youve read the articles, visit the conference websites to find out where and when you can
meet these folks in person.
We hope your days are getting brighter in more ways than one and that 2013 brings you comfort when you need it and
adventure when you want it (and vice versa).
Yours today and tomorrow,

Joey McAllister

Steve Berczuk is an engineer and ScrumMaster at Humedica, where he's helping to build next-generation SaaS-based clinical
informatics applications. The author of Software Configuration Management Patterns: Effective Teamwork, Practical Integration, he
is a recognized expert in software configuration management and agile software development. Steve is passionate about helping
teams work effectively to produce quality software. Contact Steve at, visit, and follow his blog at
Tester, consultant, and trainer Michael Bolton is the co-author (with James Bach) of Rapid Software Testing, a course that presents
a methodology and mindset for testing software expertly in uncertain conditions and under extreme time pressure. Michael is
a leader in the context-driven software testing movement with twenty years of experience testing, developing, managing, and
writing about software. Currently, he leads DevelopSense, a Toronto-based consultancy. Prior to DevelopSense, Michael was with
Quarterdeck Corporation. Contact Michael at
In testing for more than thirty years, Dorothy Graham is co-author of four booksSoftware Inspection, Software Test Automation,
Foundations of Software Testing, and Experiences of Test Automation: Case Studies of Software Test Automation (2012). Dot was
a founding member of the ISEB Software Testing Board, a member of the working party that developed the first ISTQB Foundation
Syllabus, and served on the boards of conferences and publications in software testing. Dot holds the European Excellence Award
in Software Testing. Her website is
Janet Gregory, co-author (with Lisa Crispin) of Agile Testing, specializes in helping teams build quality systems. As tester or coach,
she has helped introduce agile development practices into companies and has successfully transitioned several traditional test
teams into the agile world. Janet is a frequent speaker at agile and testing software conferences in North America, including the
STAR conferences.

Linda Hayes, is the founder of three software companies and holds degrees in accounting, tax, and law. She is author of the Automated Testing Handbook and co-editor of Dare to be Excellent with Alka Jarvis on best practices in the software industry. Lindas
article Quality is Everyones Business won a Most Significant Contribution award from the Quality Assurance Institute and was
published as part of the Auerbach Systems Development Handbook. You can contact Linda at

As Genecas CTO, David Katauskas focuses on defining and developing services, including Geneca's "Getting Predictable" approach
to software development. Additionally, he helps Genecians stay focused on success for clients, Geneca, and themselves. A twentyyear software veteran, David has deep expertise across diverse industry sectors and government. His experience ranges from
software developer, enterprise architect, and system designer to IT strategist and client partner. Find what else is on Daves mind at
Jonathan Kohl is an internationally recognized consultant, technical leader, and popular author and speaker based in Calgary,
Alberta, Canada. The founder and principal consultant of Kohl Concepts, Inc., Jonathan assists teams with testing; helps companies define, design, and implement their ideas in products; coaches practitioners as they develop software on teams; and works
with leaders to help them realize their strategic visions. As a thought leader in mobile application testingwith a newly published
book, Tap Into Mobile Application TestingJonathan develops policy and strategy, helps teams adjust to methodology changes,
and actively works with teams to deliver the best possible products.
Tracy Ragan has extensive experience in the development of business applications. She started her consulting career working
with Fortune 500 organizations specializing in the areas of build, test, and deploy. It was during these years that Tracy recognized
the lack of automation for the distributed platform that had long been considered standard on the mainframe. Before co-founding
OpenMake Software, she worked with development teams in implementing a team-centric development-to-operations process.
Tracy also served on the Eclipse Foundation board of directors as an add-in provider representative for five years.
Rob Sabourin, P. Eng., has more than thirty years of management experience leading teams of software development professionals.
A well-respected member of the software engineering community, Rob has managed, trained, mentored, and coached hundreds
of top professionals in the field. He wrote I am a Bug!, the popular software testing children's book; works as an adjunct professor
of software engineering at McGill University; and serves as the principal consultant (and president/janitor) of AmiBug.Com, Inc.
Contact Rob at



Technically Speaking

Software Testing Is a Game

It may feel like skilled testing is under attack. Fight back by changing
your perspective.
by Jonathan Kohl |
Were undergoing an enormous shift in technologies right now.
people find enjoyable or even addicting. Here are some gameAs much as we innovate, failures inevitably occur. One way we
like concepts we can use to analyze testing [1]:
mitigate against the damage those failures can cause is through
Context and rules around game play
testing. As new technology makes computing more pervasive,
Goals and desires
we have more computing power in cars, homes, public build Strategies and tasks
ings, and in medical and life-support systems than was imag Risks and rewards
inable not too long ago. As computing devices become more
Skills and chance events
enmeshed in our everyday lives, there is an even greater op Cheating and compliance
portunity for them to cause harm. With new technology comes
new opportunities for great things, but there are inevitable,
The rules and context of testing differ from team to team,
unintended consequences. Skilled, systematic testing can help
but when we start out, they are basically spelled out in our
discover many of these.
roles and our job descriptions (if we have them) and taught
But, it feels like skilled testing is
and enforced informally on teams.
under attack. People who see testing
Other team members make it clear
Analyzing human group
as a necessary evil want to outsource
what information they like and
it to tools or to other people. This
dont like for us to provide them and
activity by looking at
undermines the craft and threatens
the activities they think we should
the value of the products we depend
undertake. Once we have more exgame mechanics is helpful
on. Sure, there is a lot of repetition
perience, education, and training,
in testing that might seem boring,
we start to take a more active role in
but many of us are perfectly happy
the context and rules around game
when we want to challenge
to perform repetitive tasks in other
play for ourselves and try to match
situations. Why do we dislike it in
that with the expectations and styles
our assumptions.
testing? Maybe our approaches to
of the team.
testing need a rethink.
The goals and desires are related to what we hope to
Is software testing like a game? Before you start thinking
achieve. For many testers, our goal is to find important bugs.
about hopscotch or gold stars, hear me out. In our article in
Thats often what team members ask us to do: find and report
the September/October 2012 issue of Better Software magabugs. Our desires are based on a need to provide value to a
zine, David McFadzean and I describe a game as any situateam and to be promoted to a job that fits our needs for money
tion that involves cooperation and conflict. As testers, we coland self worth at various points in our career.
laborate with team members to help solve problems, and as we
Strategies revolve around how we prioritize or how we pick
point out quality-related information, we end up in situations
different activities and skill applications to make the best use
of conflict. Its a fascinating balance that many testers are exof our time. Tasks are the day-to-day activities we engage in,
perts at reaching. Much of what testers do can be thought of in
related to our testing work.
gaming terms.
Risks are something we understand well from a product
Gamification is one approach to provide a structured analperspective, but what risks do we face as testers? One is that
ysis. A familiar perspective can blind our observation, while
we spend too much time on low-value activities and miss
looking at the same issue from different perspectives can highimportant bugs that trip up our customers. Rewards can be
light observations that we might otherwise miss. Analyzing
categorized as extrinsic and intrinsic. Extrinsic rewards are
human group activity by looking at game mechanics is helpful
external, often linked to quantitative metrics like coverage
when we want to challenge our assumptions and learn more
and bug counts. Intrinsic rewards are often more qualitative,
about what we are doing well or potentially missing out on.
looking at the value the person gets for doing the activity itself.
Gamification in application design means improving user
Skills are interesting to analyzefrom those that we deengagement by applying the mechanics of game play that
velop as testers, such as observation, evaluation, communica-

Technically Speaking

tion, and technology skills, to investigation, people, and questioning skills. Chance events are fascinating. In games, they
help level the playing field so that skilled players cant always
dominate. In testing, we often discover important issues seemingly by accident, but our skills help put us in the right place at
the right time to observe a chance event.
Cheating and compliance are also fascinating. How can you
cheat at testing? As James Bach and I point out in the satirical
Secrets of Faking a Test Project, [2] a test-case-management
system is easy to cheat. You just periodically pass or fail tests in
the system to demonstrate progress. Here is a challenge for test
managers: How do we encourage compliance with organization goals and discourage cheating?
Ive introduced within this article a structure that I would
like you to analyze your own testing work with. What areas
above can you fill in with details? What areas are lacking? Are
there areas that people want to avoid? These are often the best
places to start improving.
Games and gaming provide us with extrinsic (numbers,
counting, and scores) and intrinsic (doing something because



its enjoyable) rewards. In testing today, we have two polarities:

metrics-heavy, scripted testing and qualitative-focused efforts
like exploratory testing. If we look to game mechanics, then
we can look beyond polarities and focus on effective testing.
Games provide wonderful lessons to help us analyze our work.
If we take it a step further and apply gaming mechanics to software testing, then we can help make it more engaging, creative,
productive, and fun. {end}

For more on the following topic go to


Tune In to


April 28May 3, 2013

Orlando, Florida | Rosen Shingle Creek
RegisteR by

maRch 1, 2013

aND saVe

UP tO $400
groups of 3+
save even more



Choose from a full

week of learning,
networking, and more.

sses Begin
Multi-day Training Cla
d Full-day Tutorials
34 In-depth Half- an
urrent sessions,
5 Keynotes, 42 Conc
g events,
the eXPo, netw
ssions, and More

adership summit
Testing & Quality Le
lated software
Workshop on regu
Testing (WresT)

Sponsor: 2013







SuPer eArly Bird offer

receive up to $400 off the regular conference registration fee if payment is received on or before
March 1, 2013. (depending on the conference package selected)

GrouPS of 3 or More SAve uP To 30% off

register a group of three or more at the same time and save up to 30% off each registration. To take
advantage of this offer, please call the Client support Group at 888.268.8770 or 904.278.0524 or email
them at and reference promo code GrP3.

AluMni diSCounT
sTar alumni receive up to an additional $200 discount off their registration fee. (depending on the
conference package selected)

MulTi-dAy TrAininG ClASS + ConferenCe

save an additional $300 when you attend any of the multi-day training classes and the conference.
(discount already reflected in the conference pricing)

please noteWe will always provide the highest possible discount and allow you to use the two largest discounts that apply to your registration.

The exPo May 12

Visit Top Industry Providers Offering the Latest in

Testing Solutions
Explore this one-of-a-kind Expo designed to bring you the latest
solutions in testing technologies, software, and tools.
Support your software testing efforts by participating in technical
presentations and demonstrations conducted throughout the Expo.
Meet one-on-one with representatives from some of todays most
progressive and innovative organizations.
For Sponsor/Exhibitor news and updates, visit
To become a sponsor/exhibitor, please contact

Combine with training and save!

From One Expert to Another

Dan Wellman
Years in Industry: 13

If some developers on a team practice TDD and


others dont, then the team may not get all the benefits

Interviewed by: Steve Berczuk


of TDD. Ive seen the benefits pay out when everyone

on a team is following the same practices, encouraging
each other, and learning.

Reading Michael Featherss Working

Test-driven development is a tool that

Effectively with Legacy Code helped

helps me improve the quality of my code

me immensely when I started doing

and design. Because I write the tests

this. If you want to start practicing

before I write the production code, Im

test-driven development and youre

likely to end up with a comprehensive

swimming in a legacy code base, I

test suite. Its a lot harder to skip the

cant recommend this book highly

make sure theres a test for this code


step if I write the test first!

As Ive practiced test-driven

development more, Ive learned to slow
down a bit and think about the design of
my codein the small at the object level,
and in the large at the system scale.

Ive worked at other jobs that didnt use an agile

process, but I practiced TDD on some of the code I
wrote for those projects. When I did use TDD, I felt
more confident in the code I delivered, and I was
less worried about making a mistake that caused an

When Im not sure how to design or implement a

I really like J. B. Rainsbergers explanation

feature, I start an experiment without worrying about

of how test-driven development works. He

tests and with the aim of getting some fast feedback:

mentions a distinction in terminologytest-

Is this even possible? What happens when I try this?

first (writing the test code first, then the

Once Ive settled on what the design would look like,

production code) versus test-driven (writing

I revert the code and start applying the test-driven

the tests first and using the feedback from

development process from scratch.

the tests to change the design).

If a new startup has six weeks to prove itself as a worthy investment to funders, a rigorous suite of
tests and a highly flexible design might not be the most important use of developer time and money.
Sometimes that means not writing tests at all. Sometimes that means dialing back the scope of the tests
for example, writing high-level acceptance tests but not writing as many unit tests, or vice versa.

For the full interview, visit






This past summer, Dot Graham and I, Rob Sabourin, were

working together in the hallowed offices of Grove House in
the picturesque hamlet of Macclesfield, England. I was sharing
some recent research and task-analysis experiences in the area
of agile project collaboration. In many of the successful agile
teams I work with, testers collaborate frequently and directly
with programmers, business analysts, customers, and other
team members.
I collected many examples by interviewing team members
and by observing collaboration in action. I documented each
collaboration story. My goal was to build resources to help
teach collaboration over and above the generic warm and fuzzy
team-building approaches. I want to look deeply into how collaboration takes place.
As I started to share collaboration stories, Dot observed
that there were several aspects of agile collaboration that bore
a striking similarity to a team-based collaboration technique
known as software inspection. In software inspection, a
small group of individuals work together to identify defects,
weaknesses, and potential improvements in any software development work product. Software inspections have been used
since the late 1970s and are implemented independent of the
lifecycle model in use. I have been implementing software inspections since the 1980s, and Dots book, Software Inspection, co-authored with Tom Gilb, has been an important inspiration and guide.
Dot has taught software inspection techniques to inspection
moderators and inspectors over many years, focusing in later
years on a lean version of the process called agile inspection.
However, interest in inspection seems to have declined in recent
years, or at least people arent admitting to doing it. It is no
longer an attractive topic at conferences or discussed much in
blogs, magazines, or forums.
We both feel that it is a real shame that these techniques,
which are extremely effective, have been abandoned. What is
the reason for this? Have these techniques just gone out of
fashion, or have they stopped working?
The agile story in this article is just one example. The story
is real, with the company and context sanitized to protect the
innocent. Note that the practices described are imperfect and
include adaptations that may vary from recommended agile
practices or strict adherence to the Agile Manifesto or agile
guiding principles.
I present the story with our comments in italics. Dot comments on similarities to recommendations from the ancient
wisdom but also highlights possible dangerslessons learned
from the past that can help you in the future.

A Software Review story

Sunsoft is the world leader in its product market. It has built
this leadership position through rich product innovations and
many strategic corporate acquisitions. Its products run on highend workstations and desktops, and typical development projects involve adding new capabilities to existing product families.
Sunsoft products have been on the market for more than
ten years. They are based on a large amount of legacy code

that has an eclectic history, poor documentation, and is very

difficult to maintain. Frequently, small changes to this code
introduce regression bugs that are difficult to identify during
development sprints. Implementing new features often requires
the modification or complete refactoring of code. Sunsoft does
not have automated regression testing of legacy features. Automated unit and story tests are being created for new features
but not for legacy enhancements.
Sunsoft implements a variation of Scrum. Each product
has several feature teams. Each feature team includes a ScrumMaster, designer, test lead, development lead, and documentation lead. Each team also includes a mix of developers, testers,
and writers. Team size does not exceed ten members.
Sprints are three weeks long. Products experience a beta
release cycle of about two months before commercial deployment. There is one major and one minor product release to
the market per year. Patch releases are made to correct urgent
field-reported concerns. Major and minor releases take place
at fixed dates and are synchronized between all product teams.
When a storys coding and unit testing is complete, all new
or changed code is submitted to a formal review process. An
automated review-management service governs and regulates
the review workflow. Only reviewed code can be checked into
the source control system.
A small team of architects and senior developers does the
reviews. The author does not participate in the review.
Dot: A small team with a variety of participants is a good
idea. Having domain experts involved is also a good idea.
However, excluding the author may not be a good idea. Authors are normally excluded to protect them from review
comments that may be perceived as aggressive or threatening.
But, if the author is not present, she misses out on the opportunity to hear detailed technical discussions that could be of
immense benefit to her future work. There are better (and very
important) ways to protect the author while including her in
the review meeting.
Another surprising thing is that the author often has the
most to contribute to the review as well, due to her more-detailed understanding of the artifact being reviewed. You would
think that the author has already said everything she knows,
but the discussions and questions that come up in the review
can trigger deeper understanding and identify more significant
problems when the author is present.
Rob: Good point. In this case, the author was excluded
from the review because the author was not familiar with the
legacy code. My recommendation in this area was to involve
the author a bit more in the review process to improve knowledge sharing and make sure reviewers were aware of the intent
of the changes to legacy code.
Code is reviewed in order to:
1. Determine if there is a risk of breaking existing legacy
2. Check conformance to coding standards
3. Identify features that use any modified legacy code
4. Focus regression testing


Dot: Having clear objectives for the review is critical to success. These are good objectives. However, it may be possible
to check conformance to coding standards using static-analysis
tools, which could be run prior to the review.
Rob: Static-analysis tools might do the trick for new code,
but from what I saw, a lot of the legacy code did not conform
to the coding standards. I suspect static-analysis tools would
have flagged too many violations in the original code. The
team was trying to minimize change.
Developers must correct any defects identified. Changed
code is re-reviewed prior to check in.
Dot: Correction of defects outside of the review meeting
gives the developer a chance to take a step back and think
about the defect from a wider perspective. It also gives her an
opportunity to look for and fix other areas of the code where a
similar defect could occur.
Re-reviewing changes is a good idea, where at least one
other person confirms that the changes are correct (but not
necessarily another full review meeting).
Reviewers include senior architects and developers. Reviewers are familiar with the history and evolution of the code
base. A review can be completed within a few hours of code
Dot: Having senior architects and developers as reviewers
is good, as their knowledge will contribute to the quality of
defect identification and fixing.
However, there is a danger of relying too much on individual expertise. If these individuals were to leave, their knowledge may leave with them. It would be better to include other
reviewers who were not as experienced so that they can begin
to pick up some of this knowledge. It may also be useful to try
to encapsulate some of their knowledge in checklists that could
later be used by less-experienced people if the experts were not
available for any reason.
Its excellent to have quick feedback to developers within a
few hours of submitting the new or changed code. The sooner
feedback is given, the fresher the ideas are in their minds and
the more effective the learning experience.
However, this assumes that the reviewers and experts are
available at short notice for the review and that they are all
willing and able to drop everything in order to do the review.
Rob: Good point about the dependency on experts who
may be very busy themselves. The experts were always available for reviews and were seemingly dedicated to the review
process Sunsoft established. I can imagine that a backlog could
develop if several teams modified legacy code at the same time,
but the review workflow seemed quite fluid.
I spent several months performing a detailed task analysis
of all testing and development projects at Sunsoft. Although
several process changes were proposed, it was determined that
the Sunsoft review process effectively minimized regression risk
due to modifications and enhancements to legacy code, especially those made by new developers unfamiliar with the legacy
code base. Note that Sunsofts automated unit and story tests
effectively exercised new features but did not help identify regression bugs in legacy code.



I consulted some independent agile transition experts about

the Sunsoft review process. These agile development experts
suggested that the Sunsoft review method was anti-agile.
Dot: Its great to hear that this review process was so successful. They followed many of the principles of software inspectiona process that still works!
However, this is a surprising reaction from agile experts! It
seems almost as though agile theory is more important than
practices that work.
Sunsoft code reviews were implemented as a process bridge
during the companys agile transition. At the time of this task
analysis, Scrum had been in use for just under two years. It is
anticipated that, eventually, automated regression testing at the
unit and story levels will mitigate the need for such a comprehensive review of all code changes at Sunsoft.
Dot: It would make a very interesting study to see what regression bugs (if any) creep in after reviews are abandoned at

Some Concluding Comments

In this story, we have seen how a traditional review process was used on changes to legacy code in an organization
using agile development. The tried-and-true techniques are
shown to work well at Sunsoft, regardless of how some agile
experts categorize it.
There are many great lessons that we can apply from reviews and inspections to the many challenges that agile teams
face in collaborating today. One of the most important lessons
is that we can find problems early with minimum effort by getting a group of people with diverse skills and experience to
work well together.
The old technique, rather than being past its sell-by
date actually has some powerful advice for todays development. Of course, most ancient texts do need some interpretation for the current day. If you read Software Inspection, you
will find things that dont apply today (e.g., heavy process and
documented planning), but you will find many nuggets of information and tips for collaborating better that are eminently
applicable now and in the future. These ancient techniques
can help you improve collaboration on agile teams. We hope
that the tips we have outlined in this article will help you do
better reviewing and receive more benefits from reviews, however they are done in your workplace. {end}

For more on the following topic go to

Further reading

What is this freakin stuff?

OpenMake Software has the Answers.

Learn about our Dynamic DevOps Suite | 312 440-9545


On typical, traditional, phased and gated projects, the workflow might look something like this: develop requirements,
create a high-level test plan, create test scripts for each requirement, execute tests, update tests, and report. A set of requirements is delivered to the delivery team, who must figure out
how to design, code, and test the final implementation. Architects decide how to design the system, which is then handed
over to programmers to code. Testers may or may not have
been involved in reviewing the requirements, but they are expected to create a high-level test plan following a template such
as IEEE provides. The testers then create test scripts or procedures. When the code is delivered, whether it is piecemeal or
all at once, the tests are executed and bugs are reported, fixed,
and retested. Test cases are updated. Tests may or may not be
automated once functionality is stable.
In agile, we dont know all the details of each feature at the
beginning of the project, so we need to adapt our approach.
Using agile principles of simplicity and iterative development,
we look at levels of precision and ask what information the
team and stakeholders need to know at each level. The levels of
precision that are important for test planning are:
Product release: Multiple teams working on a single
Feature teams: A single team working on a productthe
only team, or perhaps part of a larger product release
Feature or epic: Some capability or piece of functionality
that is useful to the business
Story: A small, testable chunk of functionality, implementable within an iteration



For consistency of terms, lets assume we have a threemonth product release with two-week iterations for each of
four feature teams. There are multiple features that each team
completes in the release cycle.
In the next sections, well explore the level of planning,
types of testing, and the documentation that might be expected
at each of the four levels described above.

Product Release
At the product-release level, teams should have a good idea
of the product vision. An overall test approach should take into
account dependencies between teams or between featuresfor
example, how to coordinate performance and load testing
that individual teams might not be able to do sufficiently in
their test environments. Consider economies of scale for tests
such as interoperability, browser compatibility, or mobile devices. Also, think about what additional testing needs to be
done during the end gamethe time needed before releasing a
product to production to commercialize it. One example might
be final user acceptance tests. This test approach should cover
what is important to this product release and may also include
the need for new test tools or environments. The size of the
document might vary depending on the number of teams involved and the complexity of the product, but I recommend
keeping it as simple and clear as you can.

Feature Teams
Each feature team works with its product owner to determine how much work they might get done during the release


cycle. Often, a release-planning session is held when teams

size the stories and features in their product backlog. Testers
help during the planning session by asking clarifying questions that may help the team determine the bigness of the
story. Impacts to the system as a whole should be considered
at this time. Use an exploratory mindset to consider what performance or security concerns might there be, or whether this
story will affect other functionality. The answers to these questions may be captured in the story, but I also like to take this
opportunity to start writing test ideas in my notebook for future use. It doesnt cost anything to capture these ideas and, if I
dont use them, I can cross them out later.
At the release level, sometimes a test document is required.
Before you create one, think about who will use it and why they
want it. If you need some form of document, keep it as simple
as possiblepreferably one page long. Since the functionality is
not set yet, there is no point including any details of scope unless there is a risk involved that others need to know. If a document is not needed, I record the testing risks and assumptions
I want to monitor on a whiteboard that is visible to the team.
Recently at a conference, a colleague, Huib Schoots, said
that instead of embedding high-level feature and story tests in
a document that agile-resistant managers demanded, he used a
mind map to capture those details. The mind map satisfied the
managers, allowed him to keep to agile principles of simplicity,
and enabled him to stick to a one-page test plan to capture the
important information.
Think about developing personas [1] and tours for extra exploratory test ideas and coverage of the application. Another

easy tool that is effective at the release level for capturing highlevel test ideas is a test matrix. For maximum effectiveness, use
collaboration to generate the test ideas for the matrix. When
tools like these are used and made visible, it provides a different viewpoint into testing. [2]

Features or Epics
As features (some business-value capability) are broken up
into stories, we need to make sure they are testable so that the
team gets into the cadence of coding and testing each story
until it is done. There is a danger that teams work at too low
a level and forget about the bigger picture: the feature. Work
with your teams product owner to create high-level acceptance
tests (examples of expected behaviors and misbehaviors) for
the feature. This will help define the scope and keep the business value visible. Mind maps are powerful tools for generating
test ideas at any level, [3] but I find creating them for each feature the most valuable. Try collaborating with the whole team
on a testing mind map before breaking it down into stories.
It is a way to explore some of the possible issues before they
Like in the release-planning session, these test ideas can be
saved for when we really need them at the story level and you
have a simple documentmaybe a picture of the whiteboard
or flipchart. Again, I often jot down additional test ideas in my
notebook for later.
One idea that I have found works well for teams is to create
an extra story for each feature called Test the Feature. The
tasks will be mostly testing tasks, such as Explore the Fea-


or backlog grooming). There are many variations on how to

define these tests, and it often depends on the tool you use.
Once we have these acceptance tests, we can start the
conversation either in pre-planning or in the iteration-planning session. We get into more details, and again we can
explore other scenarios to help ensure that the team shares
a common understanding. Our tests for the story evolve
from these conversations and can be expanded during the
iteration. I find it valuable to start writing my test ideas and
variations during story-readiness meetings, continue during
iteration planning, and then use all the tools in my toolbox
to define other tests that will prove the
story works as expected. Once this list is
complete, I suggest reviewing with the
product owner or another tester and then
collaborating with the programmers who
are working on the story to ensure that
you have the same understanding and to
plan the needed automation.
Remember to write down exploratory
test ideas through all of this, so the exploratory test charters will be ready when
the code is complete.

ture, Perform Load Test, etc. I also recommend including

a task called Automate the Workflow that can be done
through the GUI when the feature is stable. Including a story
like this ensures that the feature testing is not forgotten.

Once we are working at the story level, we start getting
into more detail. For each story, we need high-level acceptance
testsan example of expected behavior and at least one example of misbehaviorto define the scope of the story. I encourage doing this during story readiness sessions (pre-planning


time to
Distinguish yourself from your
peers and gain a competitive edge
ALPIS TrAInIng OfferS:

Attend class
in-person or
virtually from

Technology and Methodology Courses

HP: Quality Center ALM, QuickTest Pro, and LoadRunner
Microsoft: Test Manager, Coded UI, and Load Test
Process Improvement: ISTQB Certification, Managing Change,
Requirements Definition, Enterprise Quality

Interactive Learning Method

Bring your Workplace to the Classroom & Take the Classroom
to your Workplace

Post Training Support

Bulk Training Program

Free refresher courses

Savings of up to 40% with credits

good for a year

Process and tool

implementation services

Test Results
Test results are strongly related to test
planningespecially for detailed tests. Visibility and transparency are some of the biggest benefits in agile projects, so I suggest
that test results should be readily available
all the time. Ideally, you want to extract
them from your automation framework or
continuous integration environment. Find
simple ways to keep track of your exploratory testing so you can review the results
when you need to. Another way to make
results visible is to use tools like low-tech
dashboards. Spending time collating results
does not add value to the customer or the

In summary, plan collaboratively, keep
it simple, make working at the right level
of precision valuable so you have the right
information, and keep test results visible.
Since 1993, ALPI has empowered clients with innovative solutions delivered by our staff
of flexible and creative professionals. Trainings are held virtually, at our state-of-the-art
facility located just outside of the Nations Capital, or onsite at your company location.

Contact or 301.654.9200 ext. 403

for additional information and registration details



For more on the following

topic go to

Further reading


Integrated test management software.

Enterprise Tester offers a flexible test management and execution feature set,
full coverage from requirements to defects, TQL-based dashboard and reporting,
a REST API, and more.

Intuitive Interface and Dashboard

Version Rollback, Comparisons, and Baselines

Integration with JIRA, Enterprise Architect

Testing Query Language [TQL]

and Microsoft Team Foundation Server

Activity Streams and Notifications

Ease of Test Setup and Execution

LDAP Synchronization and Scheduling

Script, Project, and Report Templates

Extend with the REST API






Its a production application in development, explained a development manager to a production control administrator, and that is why we need to do our software releases so frequently.
The production control team looked at the development
manager like he was crazy. A production application in development can be a difficult concept for a production control administrator to fully grasp or support. According to production
control, a production application should be fully baked and not
in what would be considered a development state. However,
having frequent releases is a basic requirement of rapid development methodologies like agile, and this impacts the way in which
development teams and production control teams must interact.
In essence, applications go to production before they are fully
baked and get better over time until the last iteration has been
completed. At that time, the application goes into a maintenance mode, in which scheduled releases are approved by the
production control team but often managed by developers.
Production control teams are ready to support scheduled
releases but, for many reasons, struggle with continuous deployments. For example, production control prefers to manage
releases in a stricter manner, including an approval process that
may take several days. The application moving to production
is added to a release calendar and scrutinized through release
meetings and discussions. It is not unusual for the production control team to freeze a release just to make sure new
changes are not added. This is contrary to continuous deployment, where frequent, small releases are moved to production
in a more automated fashion with less scrutiny or discussion.
Production control teams that support distributed platforms
have never fully taken over the task of performing the build
to deploy steps for managing production binaries. In many
large companies, the production control team serves as an approving body, providing the development team access to production servers for the purpose of managing its releases. When
releases were less frequent, this potential violation of the separation of duties flew under the radar. If there was a problem
with production, the production control team was responsible
for fixing the issue and answering to upper managementalways with the development teams doing the work to get production back on track.
With the introduction of agile development, where frequent
releases are a core part of the process, the production control
landscape has changed. Production releases are more likely
to cause upper management to question the frequency of releases to the production environment. The answer to the question Why frequent releases? is Its a production application
in development. Yes, we have come full circle, which is why
the DevOps conversation has become much more prominent
in discussion blogs, technical journals, and in the conference
rooms of companies looking to improve the moving of applications between development and production control. Frequent
releases are being reported to upper management, exposing
that there is a change in the way applications are moved into
the production environments. Instead of scheduled releases,
production control is being asked to push through frequent re-

leases, changing the standard release processes that production

control has followed for the past thirty years. While agile teams
argue that smaller, more frequent releases reduce risk, upper
management may see frequent releases as an unstable, high-risk
production environment.
At its core, DevOps attempts to streamline and simplify the
transfer of the software application from development to production, as well as to calm the nerves of upper management
who are wary of frequent releases. Giving production control
what they need to manage a production application in development is critical to simplifying the transfer. The most critical
information that production control needs from development
is the ability to confirm that the application has been built on
the correct production level of the underlying technology stack.
They want to see that the binaries have matching source code
and can be rebuilt for an emergency release or a corporate
audit. They often want sign off from a testing team that confirms that the new release has been fully tested in an environment that models production. Details on server configurations,
environment variables, and release steps are needed from the
development teams in order for production control to repeat
the deploy process at the production state.
In order to satisfy upper management, production control
needs to report and manage key facts about the applications
moving into production. Production control members need to
report that the new release is compatible with the production
environment, that emergency fixes are possible, and that there
is an audit trail for tracking who requested the release, the time
and date of the release, and any new features or fixes initiated
in the releaseoften down to what source code was updated
and what new binaries were moved to production. Production control needs to confirm that the application moving to
production is built on the production technology stack, not
the development technology stack. Additionally, production
control members need a clear topography of the servers that
are used, the steps necessary to perform the deployment, and
details about the servers configuration. Even though the team
members may not execute the deployment themselves, they
are responsible for it. This means that team members need the
knowledge of what is occurring at the production-deployment
level. In other words, production control needs basic information about how applications are built and deployed.
A lack of shared information is the primary reason why the
transfer between development and production is problematic.
This information is hidden in one-off, script-driven processes
that manage both the build and deployment. These processes
are known by only a few individuals on the development team
and in most cases cannot be known to production control.
Furthermore, these processes impede the sharing of control
between development and production. Ultimately, this creates
some nasty politics between the two teams. What appears to be
a constant struggle for power is, in reality, simply a struggle for


shared knowledge and control.

Most organizations use scripting to manage builds and deploys. Each development team determines how its build and
deploy scripts will operate. Because of this, a company with
fifty java teams can have more than fifty build and deploy
scripts that look and act completely differently. All scripts can
be building and deploying a JBoss application, but each does it
in a unique way. And, no, there is never a reason for the difference other than the names of files and servers. The Java compiler works the same regardless of the file name. JBoss applications are deployed the same way every time, with the exception
of binary names and some server configurations. Each developer
who writes his applications build and deploy scripts simply




Your CEO will love you when you show them
how you can achieve an outstanding testing
ROI with ISTQB Software Tester Certification.
With the average cost of a software defect in the range
of $4,000 $5,000[1], if ISTQB Certification helps your
tester eliminate even just one defect, the result is
nothing less than, well, loveable: an ROI of up to 2000%.
ISTQB Software Tester Certification is the most widely
recognized and fastest-growing software tester
certification in both the U.S. and the world. Discover
how ISTQB certification can pay for itself in a matter
of days: Thats a testing result any CEO will love.

Want an even better ROI?

Take advantage of our new
Volume Purchase Program.



manages the logic in different ways, making the scripts nonstandard and often hard to debug, fix, and re-execute. Some developers do an outstanding job of writing their build and deploy
scripts, and others not so much. As a result, production control
must manage a web of scripts with no standards, no common
reports, no method for matching source to binaries, no method
for validating the technology stack, and no method for repeating
a build or deployment if needed. Thus, the constant struggle for
control of the deploy process continues between teams. Production control team members ask for what they consider to be
basic information, which development teams are unable to produce in a standard and repeatable way across the organization.
Static scripts hide critical information, such as technology stack
dependencies, and generate no common reports
that should be shared across teams.
We must strive toward using real automation and standardization. A one-off script executed by a job scheduler or workflow tool is
not real automation. Creating reusable build
and deploy scripts will ultimately be the answer
for delivering standardization to both build
and deployment. Todays DevOps solutions are
quickly beginning to address the problem of
one-off scripts and lack of repeatability.
Moving away from one-off, script-driven
processes is not entirely new. Look at the transformation that mainframe teams went through
in the early 1980s. Compile and deploy Job
Control Language (JCL) used to be the most
common way for build and deploys to occur
on the mainframe. It is now extremely uncommon to see private compile and deploy JCL
used on the mainframe. You may argue, The
mainframe is easier to manage, so they can
standardize their scripts, but, as a person who
has done both, I would disagree. There are as
many variables on the mainframe as there are
in distributed environments, from Windows
to Unix to Linux. And, yes, mainframers can
do a check-in and build (continuous integration) and move only the updated files through
the lifecycle quickly (incremental continuous
deploy) while production control manages the
entire process without using a single one-off
JCL script. Ultimately, a similar process will be
adopted on the distributed platforms, allowing
Learn more
production control teams and development
now at
teams to share the knowledge and management
of software builds and deploys. Developers
can then spend less time and money on writing
one-off, script-driven processes and instead
focus on what they do best: quickly delivering
awesome business software to maximize the
profits of their employers. {end}

Capers Jones, A Short History Of The Cost Per Defect Metric, Randall Rice,
The Value of ISTQB Certification



TechWell Spotlight
Featuring fresh news and insightful stories about topics that are important to you, is the place to go for what
is happening in the software industry today. TechWells passionate industry professionals curate new stories every day to
keep you up to date on the latest in development, testing, business analysis, project management, agile, DevOps, and more.
The following is a sample of some of the great content youll find. Visit for the full stories and more!
Generation Y's Upcoming Impact on IT

The Iron Dome and ALM

By Joe Townsend
Much has been said about Generation Y regarding their work
ethic and their constant need for recognitionjust for showing
up with the desire to run the show from day one. Members of
Generation X and most certainly the baby boomers have certain misconceptions about Generation Y.
A conversation on LinkedIn confirmed my suspicionWe
just dont understand Generation Y. offers us a white
paper on managing the next generation of the workforce. This
white paper is not specific to Generation Y, but it looks at ways
the new workforce will be more mobile and in need of technologies that prior generations have not always had or expected.

By Bob Aiello
Complex software systems are developed to support mission
critical systems including life support and missile defense systems. One system that has recently been in the spotlight is
Israels Iron Dome Missile Interception System, which is a
computer-aided rocket system that intercepts incoming missiles and destroys them before they reach their intended target.
The system is being credited with saving lives, property and, in
many ways, overcoming terrorism itself.
The Iron Dome has three components: the detection and
radar installation, battle management and weapons control,
and the missile-firing unit itself. This means that these three
components must be configured to work together between
the hardware and software components. This task can be very
complex and virtually impossible to achieve without a mature
development process with many of the practices associated
with application lifecycle management (ALM).

Continue reading at

New SEC Software Breaks up Insider-Trading
By Jonathan Vanian
The next time you come across some top-secret information
regarding stock tips, Id recommend second-guessing yourself
if you plan on doing a little insider wheeling and dealing. According to the Wall Street Journal, the Securities and Exchange
Commission (SEC) has been using new computer software
to monitor stock purchases.
Because of the new software, the SEC was able to put an
end to a recent insider-trading scheme in which a senior editor
at the popular financial news site was implicated. The insider-trading ordeal was linked to last years $316
million purchase of Orthovita Inc. by medical-device maker
Stryker Corporation.

Continue reading at

Need for Mobile App Developers Outpaces
Number of Job Candidates
By Noel Wurst
While many people across the country are looking for work,
theres one field that continues to offer employment opportunitiesfor freelancers and full-timers alike. Mobile app developers are in high demand, and in some cases, there are even
more jobs than qualified applicants.
Brandon Passley, the founder of Vokal, a Chicago application company, tells Medill Reports that its not just that there
arent enough applicants. Its that tech job seekers havent received the proper skills training or even earlier education that
prepares them for a future career. "They teach English and
German in high school, why not programming languages?
wonders Passley. This would increase the likelihood of students demanding additional education at the college level.

Continue reading at

Continue reading at

Project Lessons from the Great Train Robbery
By Adrian Reed
UK history was made when 2.6 million in cash was stolen
from a Royal Mail train. The Great Train Robbery of 1963
has entered the collective consciousness of British culture as a
daring heist executed by cool criminals, many of whom stayed
on the run and evaded capture for years.
The facts behind the Great Train Robbery are astonishing.
The cashequivalent to about 43 million todaywas carried
in an unguarded compartment and was stored in sacks, rather
than a safe.
Looking at this arrangement in the cold light of day, it's
easy to ask why the risk hadnt been foreseen. Im no expert
in rail security, but if I were transporting more than 40 million, Id make sure it was locked and guarded! In many ways, it
could be seen as a robbery waiting to happen.
However, the train had run consistently and without issue
or loss for 125 years. Knowing this fact makes it easier to understand why nobody perceived the risk. If something works
successfully for 125 years, why change it?

Continue reading at

The Case of the Proliferating OpenStack
By Beth Cohen
What do Dell Computer, Red Hat, SuSE, Cloudscaling, Piston
Cloud, Cisco, and StackOPS all have in common? Each has its
very own OpenStack distribution, and this is potentially a major
Continued on page 28



Miles away from the comfort of my office chair are hiking trails
that give both experienced and amateur hikers exciting challenges in the wilderness. Interestingly, these trails bear some
surprising parallels to the daily challenges of software projects.
While hiking trails and software projects may have their own
kinds of a death marches, they share some eerie resemblances
in how people make decisions while traversing them.

The Project Kick-Off

I coach IT organizations for a living and am responsible for
best practices, capabilities, and training for my own organization, so I am more than familiar with death marches. However,
making comparisons between my decision-making process and
hiking never really occurred to me until now. Let me explain.
Recently, my friend Bob and I decided to hike on the Ice Age
Trail in Wisconsin. This was in preparation for our bucket-list
goal of walking a portion of the Appalachian Trail. We knew
any hike would be a challenge merely because we were non-active desk jockeys, more than fifty pounds overweight, and past
middle age. In spite of our condition, we were determined
and very eager to get started.
On the first day of our hike, we set out to explore the wilderness and just enjoy ourselves. We saw many great sights and
regularly rested with food and water from our backpacks. It
was a great day with perfect weather. All of the elements indicated that this day would run flawlessly. In an eerie sort of way,
this was strangely similar to the optimism and energy we have



when starting a new and exciting project.

Knowing that our journey was only a day hike, at some
point in the day we would need to turn around and track back
to our car. The morning went well and we felt strong, full of
energy, sharing a great time on a beautiful day. The morning
was going quickly and, before we knew it, we reached the
five-mile mark. It was just after lunch that we started to think
that turning back would be smart. The second half of the hike
would probably go slower.

Pushing the Team

We looked at the map and saw that there was a ridge ahead
that had some spectacular views. Feeling physically able and
still eager, I suggested that we squeeze in a few more miles
before turning back. Bob, however, had a gut feeling that we
should turn back because we were probably reaching our physical limits. Bob is older, more out of shape, and naturally more
risk adverse. We all have a Bob on our projects.
We clearly werent in agreement. Without definitive proof
that we were reaching our limits, we decided to press on. While
his version of this story is that I convinced him to press on, we
actually agreed to turn back at mile seven.
Around mile six, the path changed to a steep inclinea
painful, slow inclinebut we made it to the top and to mile
seven. We felt great when we arrived at the top of the ridge and
rested on the trail near the tall grass. Unfortunately, the view
wasnt as spectacular as we had imagined. In fact, there was no

view at all except for tall brush and grass, but we still felt great
about making it to mile seven.
After a break, we decided that it was time to turn back. The
first seven miles were great, so the next seven should be equally
great, right?
We were oblivious to the fact that we were perilously close
to the real point of no return.

Signs of a Death March

About two miles into the return trek, we realized that we
were out of water and our legs and feet were becoming very fatigued and sore. Without options, we stoically pressed on. Two
miles later (thats eleven miles total), we were now becoming
worn out. The storm clouds started rolling in, the wind picked
up, and our flawless day was no longer looking so flawless. We
now had an additional sense of urgency to stay out of the impending storm. Yet, we still had about three miles to go. With
plenty of trail in front of us, we felt the impending gloom and
we were literally in pain.
Have you ever experienced a software project similar to
this? Doesnt this sound like the beginning of a death march?

Houston, We Have a Problem

The next mile became a struggle, as our bodies were not actually equipped to deal with the demands now being placed on
us. The final two miles would only get worse. Our legs started

cramping, each step was painful, and we were thirsty. We

limped along, genuinely unsure if wed even make it back to the
car without assistance.
How many times have we committed to more than we can
successfully handle on projects? Or committed without really
knowing what we were committing to?
On the plus side, at least we knew we were very close to
needing rescue.
It became apparent to us that our short-term view of what
we thought was feasible became a decision that ultimately created pain, anxiety, and the possibility of failure. To add insult
to injury, it started raining.
After a long while, we finally reached our destination and
found a covered bench. We sat for a few minutes and then,
after what seemed like an eternity, cramped and limped toward
the cara distant twelve feet away. Somehow, we managed to
hold back the tears.

A New Perspective: Scrutinize Your Goals

When Little or No Experience Exists
Pain is certainly one of the best teachers. And, while Ive
certainly been on projects where the team has limped across the
finish line, our hiking experience gave me a new perspective.
I noticed how making decisions for short-term goals without
the proper insight, perspective, or experience can have significant effects on long-term outcomes. For example, our decision


TechWell Spotlight
to continue after mile five was clearly a mistake. I reflected on
that precise momentthat specific choicethat led us to continue on. We couldnt see the impacts of that decision at that
point in time. We also had no point of reference or metric to assess whether we were making a good or bad decision. It boiled
down to a lack of awareness and an eagerness to do a little
Then, I thought about some of the bad decisions Ive seen
in directing project objectivesdecisions that created pain for
the team and, in some cases, established additional barriers for
corrective action. Instead of having conversations about our
decision-making process, we would just state that it was poorly
executed. We rarely considered that the original decision was
the culprit.
In the case of our hike, I can now see that our execution was
fine. The decision to do more was flawed.

My Take Away: Are We Kidding Ourselves?

When making any kind of project decisions today, I now
ask, How can we prevent this from becoming our fourteenmile death march? Are we just kidding ourselves?
I was actually able to put this into practice the other day
when our leadership group was discussing strategic staffing.
We knew that a decision to handle a short-term, nonstrategic
project would certainly limit our capability to engage in any
larger, on-target opportunities that might come along. So,
we had good discussions around that topic and looked at the
varied perspectives with our eyes wide open. We stuck with the
strategic goals in spite of a small, immediate win. Since then,
weve been able to apply that same concept to other growth
Are you surprised to learn that Bob and I have hiked again?
We now have changed our behavior and are more in touch with
our real capabilities. Our experiences are much more pleasant.
So, next time you have a tough choice and everyone is debating an approach, think of yourself at mile five of your hike.
Sometimes, its better not to push the limits, because youll feel
the pain for quite some time afterward. {end}

Consider This Before You Make Your

Next Decision

1. Look at each decision from different perspectives and

with a level of caution.
2. Accept that there may be consequences if you choose to
do just a little more than your original goal.
3. Choose the slam-dunk approach when youre not sure or
have little experience in a particular area. The result may
be that youll get a quick team win and gain insight into
real future capability.
4. When you believe an execution has gone wrong, step
back and look at decisions made before execution
started. It may be the decision and not the execution that
was flawed.




Continued from page 25

problem for the nascent OpenStack development community.
Based on the attendance at the October 2012 Folsom Design Summit, it is clear that OpenStack has the potential to be
a real game changer in the Infrastructure as a Service (IaaS)
industry. The OpenStack Foundation has been favorably compared to its Linux and Apache predecessors as an advanced
example of open source governance, according to Jonathan
Bryce, the Foundations executive director.
That is all well and good, but it seems that a growing number
of companies are not sticking with the main (or the so-called
trunk) OpenStack program and making the decision to fork
off their own distributions for sale to their customers. Soon, if
OpenStack follows the same development path as Linux, there
will be hundreds of distributions with subtly different features
only noticeable to hardcore Linux kernel engineers and just
plain confusing to the majority of the user community.

Continue reading at

How to Manage Work-from-home Employees
By Naomi Karten
Who wouldnt want to work from home? For many, its the
ideal setup, and its not surprising that work-from-home arrangements are more common than ever. Then again, when
your home becomes uninhabitable, as so many homes did
during Hurricane Sandy, some of the potential pitfalls become
more visible.
But even during this disaster, imaginative telecommuters
sought alternative places to use as temporary offices. All they
really needed, after all, was their laptop, a roof, a working
light bulb, and an Internet connection. Coffee shops and other
WiFi-enabled locations quickly became packed.

Continue reading at

What Jobs Do Your Stakeholders Want Done?
By Kent J. McDonald
David Lee, a founder and managing partner of the financial
firm SV Angel, recently wrote a blog post on three key insights
regarding the jobs-to-be-done (JTBD) theory, originally described by Clayton Christensen, a professor at Harvard Business School, in his book The Innovators Solution.
For those unfamiliar, the JTBD theory is intended to help
company leaders think about their products and services by
considering how their customers like to use them. Company
leaders can do this by asking themselves, "What is the job of
our product team?"
Lee's post explores how the theory applies to both enterprise-driven teams (those creating solutions for their own organization) as well as market-driven teams (those creating solutions to sell to other organizations). Additionally, Lee writes of
the complexity of defining the job. For enterprise-driven projects, defining the job means you must understand what your
stakeholders are trying to accomplish and what problem they
are trying to solve.

Continue reading at

Product Announcements
Rally Software Updates Agile Software
Development Management Platform
Rally Software, an agile software development company,
added new capabilities to its agile software development platform to meet the unique and evolving needs of global organizations. Enhancements to Rallys enterprise-class platform enable
customers to customize and extend functionality for creating
a single system of record and enabling decision-making across
global development organizations. Rallys ability to continuously release allows quick implementation of customer-requested enhancements, resulting in new features in the three
main areas of usability, productivity, and scalability. Rallys enterprise-class platform is available in cloud-based or on-premises deployment models. New features and enhancements are
available in Rally Unlimited Edition, which supports unlimited
users and projects.
CollabNet Partners with emotive to Offer
Mobile App for TeamForge
CollabNet, an enterprise cloud development and agile ALM
organization, announced the availability of emotive Connect
for TeamForge (TeamForge2Go), a free mobile application that
connects on-the-move developers to the TeamForge platform
through mobile devices and tablets. CollabNet partnered with
Oakland, California-based emotive, a mobile development
platform company, to deliver the downloadable app.
The new application supports CollabNets strategy to bring
the convenience and collaborative benefits of cloud development and mobile IT to the enterprise. Coupled with the CloudForge cloud development platform and the mobile application,
TeamForges access and deployment capabilities are extended
through on-premises, public and private cloud, and mobile
The emotive Connect for TeamForge app is not just a replication of TeamForge functionality but offers a true mobile
app experience with practical levels of use for easy viewing and
fast response. It includes voice-to-text input for conveniently
adding comments to a work item and lets developers monitor
and edit critical project artifacts, including epics, stories, defects, and tasks. Users also have the ability to see changes to
artifacts in real time and to visualize data graphically or textually, update tracker information, and post comments, all from
their mobile devices.
SOASTA Announces User Measurement
Solution for Mobile Apps
SOASTA, a cloud and mobile testing company, announced
mPulse, a user measurement data analysis solution to deliver
real-time information based on actual mobile user behavior.
mPulse delivers actionable intelligence to operations and marketing professionals.
mPulse captures performance metrics, such as bandwidth
and page load time as well as engagement metrics like bounce,
exit, and conversion rates. Mobile user metrics like user loca-

tion, device type, carrier speed, and application usage complete

the total perspective. mPulse presents actionable views via an
interactive, in-memory monitoring dashboard. Users are provided a second-by-second view of real user activity broken
down by continent, region, country, or city. Business, development, test, and operations teams gain the real user intelligence
needed to correlate the impact of performance on the bottom
line, take immediate action as needed, and feed critical information back into the development lifecycle.
HP Creates Single Architecture for
Enterprises of All Sizes
HP released its Converged Storage portfolio that handles complexities and inefficiencies with a single architecture for all
sizes of client deployments.
Designed for virtualized, cloud, and big data retention environments, these solutions include new HP 3PAR StoreServ
Storage, HP StoreAll Storage, and HP StoreOnce Backup
product offerings.
HP Converged Storage helps clients simplify storage infrastructure and respond to unpredictable demands while
increasing the value organizations generate from their investments in three areas: return on information, return on infrastructure, and return on individuals. This new storage concept
enables a single-system architecture to exist in several forms,
shapes, and sizes from low to high, but retains common data
services for block, object, and file applications.
Serena Software Introduces Orchestrated IT
for the Agile Enterprise
Serena Software announced advancements to its orchestrated
IT solution set that was designed to help todays agile enterprise take full advantage of mobile, social, and the cloud. Serenas solutions are geared for IT stakeholders who want to
automate the continuous delivery of applications on premises
and to the cloud. The new solutions allow users to view release
activity, analytics, and key metrics from mobile devices, such as
smart phones and tablets.
Serenas new and enhanced orchestrated IT products add:
developer-driven, continuous delivery with cloud provisioning;
an IT catalog, control, and analytics on smart phones and tablets; and full-cycle social IT capabilities.
The new enhancements include: the updated Serena Release
Manager supports continuous deployment and cloud deployment as well as social IT collaboration; Serenas new mobile
request center extends capabilities to mobile devices; and Serenas new mobile dashboard introduces IT analytics to the iPad.
SmartBear Software Launches
LoadComplete 2.6
SmartBear Software launched LoadComplete 2.6, designed so
organizations can load and stress test their web applications on
premises or from the Amazon cloud.


Product Announcements
LoadComplete is a performance-, stress-, and load-testing
tool from the makers of TestComplete and soapUI. Designed
from the ground up to help users focus on key business tasks of
performance testing, it requires no programming knowledge to
start and accelerates workflow with automation and wizards.
With LoadComplete, users can record and play user scenarios
and optimize the load model using the freehand workloadmodeling tool. LoadComplete collects server performance data
and integrates it in performance reports.
Enhancements to LoadComplete 2.6 help to ensure optimum website performance, even with heavy traffic, and include custom page breaks that can be inserted both during and
after scenario recording.
Orasi Software Introduces New Mobile
Testing Division
Orasi Software, a software testing services provider, launched its
Orasi Mobile Testing division. Orasi now offers fully integrated,
automated, mobile-application-testing solutions through a partnership with HP and Perfecto Mobile.
Orasis mobile testing services provide organizations with
an automated and cloud-based approach to delivering mobile
applications. Customers can now access a new private cloudtesting environment that contains popular mobile devices to:
conduct localization, usability, and device compatibility tests;
test application performance in real-world network conditions
prior to deployment; and use new security services to remove
application security vulnerabilities.
ComponentOne Releases Studio Enterprise
2012 v3 with Controls for Windows 8
ComponentOne, a division of GrapeCity, a component solution
provider in the Microsoft Visual Studio Partner Ecosystem, has
released ComponentOne Studio Enterprise 2012 v3. The third
of three releases scheduled for 2012, ComponentOne Studio
Enterprise is a suite of data and UI controls for Microsoft Visual Studio, including Windows Store and HTML5 tools.
ComponentOne recently announced the release of two new
studios, Studio for WinRT XAML and Studio for WinJS. The
Windows 8 controls in these studios are now included in the
comprehensive Studio Enterprise suite of controls.
VersionOne Releases TeamRoom Environment
VersionOne, an agile project management tools company, announced the availability of its new TeamRoom Environment.
TeamRoom is a lightweight, team-based environment dedicated to supporting the daily activities of todays agile development teams. In addition to TeamRoom, innovative new visualizations, kanban improvements, and new reporting capabilities
round out the release.
TeamRoom supports team members daily activities, providing
interactive storyboards and taskboards, integrated team commu30



nication, and helpful information for daily standups. Developers

can easily customize their TeamRoom with mascots and avatars,
select the information panels to display that are important to
them, and even implement their own team-based WIP limits.
Neotys Releases NeoLoad 4.1
Neotys, a maker of load- and performance-testing tools for
web and mobile applications, announced the release of NeoLoad 4.1, an enhanced version of its load- and performancetesting solution.
A new feature of NeoLoad 4.1 is its ability to simulate realworld mobile network conditions, making it a complete mobileperformance-testing solution. NeoLoad 4.1 is the industrys first
tool that combines WAN emulation and performance testing in
a single, integrated solution. The enhanced 4.1 product also expands NeoLoads technology support, which means it can now
record all desktop and mobile applications. Version 4.1 also
introduces the support of SPDY, an open networking protocol
developed by Google to transport web content with the goal of
reducing web page load time. Other enhancements include support for web standards including HTML5.
WSO2 Launches WSO2 Carbon 4.0, Stratos
WSO2, a lean enterprise middleware company, launched WSO2
Carbon 4.0, the industrys first enterprise middleware platform
designed from the core to run both on premises and in the cloud.
The latest release of the WSO2 Carbon platform adds
multi-tenancy to all thirteen WSO2 Carbon middleware products. Running directly on server hardware or in a virtualized
environment, such as an Amazon or VMware cloud, these
products can bring the efficiencies of multi-tenancy and selfservice to on-premises deployments. Additionally, the cloudready WSO2 Carbon 4.0 middleware products can plug into
WSO2 Stratos and operate as part of a PaaS, adding cloudnative capabilities, such as elastic scaling, expanded self-service
provisioning, metering, billing, and storage, among others.
PostgreSQL Global Development Group
Releases PostgreSQL 9.2
The PostgreSQL Global Development Group announced the
release of PostgreSQL 9.2, an open source database system.
With the addition of linear scalability to sixty-four cores,
index-only scans, and reductions in CPU power consumption,
PostgreSQL 9.2 has significantly improved scalability and developer flexibility for the most demanding workloads.
Improvements in vertical scalability increase PostgreSQL's
ability to efficiently utilize hardware resources on larger
servers. Advances in lock management, write efficiency, indexonly access, and other low-level operations allow the database
engine to handle even larger-volume workloads.

How Can I Become a Test Automation

You might think that automating tests is at best a productivity play for a particular application,
but think again. There are more stakeholders than you might imagine, both in your company and
outside of it. You may be able to convert an expense into revenue, turn a project into a product,
or even grow a job into a company. Think about your automation expertise and assets as a form
of capital, and look around for options to leverage them into more budget, resources, or even new
ventures. Consider these options:
Collaborate within your company. Find others who can share the costs and benefits of your
efforts. Other application teams in your company may be able to share the development effort for
reusable framework components, giving you access to greater resources and more functionality at
no extra cost.
Look outside IT. Identify other stakeholders in the software you have automated. For example,
audit and compliance may be able to leverage your test automation capabilities to verify and document the internal controls in the application, displacing audit fees or costly manual resources. Negotiate for your team to keep a share of the savings.
Target the business. Test automation is really just a particular application of task automation.
Look for automation needs or problems in the production environment, such as streamlining complex processes, converting data or monitoring availability. Involvement in production extends your
influence into the business, where the money is.
Transform costs into revenue. If your customers have to configure the software you deliver, then
they also have to test it. Consider offering test automation services to your customers as a way of
generating revenue, gaining coverage through test case sharing, and funding your own investment
in automation.

by Linda Hayes

Turn your project into a product. If you are testing a third-party application, join the user
group and propose a consortium to develop and support a framework and reusable content. Members can share the development and support costs, and you end up with a proven product you can
sell to other users.
Convert a career into a company. If you have solved a tricky problem for your own company,
chances are that other companies have the same problem. More than one framework has grown
into a company. If you are an employee, be mindful about your employers rights and, if necessary,
do the work on your own time and offer your employer a paid-up license.
Think outside the application. Testing by definition requires an application under test, but automation can be an application in itself. My favorite entrepreneur used a test automation tool to
automate online entry of health insurance claims forms and today owns a multi-million dollar
Every item on this list is based on actual experience. It is by no means exhaustive but offered
only to stimulate your own creativity. Often, opportunities are masked as problems that you or
others are trying to solve. An entrepreneur is anyone who can identify a need, create a solution,
and deliver it to market, whether that is in the next office, another building, or across the country.


The Last Word

Ive Got a Feeling:

Emotions in Testing
Decisions about quality are always emotional. Use your feelings to
improve your testing.
by Michael Bolton |
that in Rapid Software Testing we call an issue. An issue is
A few years ago, I gave a conference presentation on the role
anything that threatens the value of the testing effort, of the
of feelings and emotions in software testing. One audience
project, or even of the business.
membernow a colleaguemet me again about a year later.
To detect problems, testers apply oracles. An oracle is a way
She told me that she had gone back to work after the talk and
to recognize a problem. Historically, testers have tended to
had discussed the conference with her boss.
think of oracles in terms of principles (such as consistency with
I saw an interesting presentation on the role of emotions in
user expectations, with some purpose, or with a comparable
software testing, she said.
product) or mechanisms (such as a test program that provides
Theres no role for emotions in software testing, he rea comparable result, or a finite state model, or a specification).
plied, blandly.
One way to recognize a problem would be to walk dispassionReally? she said. I liked the points the speaker was
ately through a specification or requirements document and
making, and the talk made a lot of sense to me.
compare the operation of the product
He stared at her. His face got red,
to a description of how it is intended to
and he became tense. There is no role
for emotions in software testing! he Confusion and ambiguity behave. Yet, peoples intentions or desires for a product might be unwritten,
It was amazing, my friend said
are the floorboards under tacit, or implicit. There might be errors
in the specification, or the market may
later. I suggest to him that there is a
have changed such that the requirerole for emotions in testing, and he has
which bugs build
ments document doesnt reflect what a
this completely emotional reaction to the
customer might want now, as opposed
their nests.
to when the document was written.
Quite so, and its interesting to conMoreover, the document is a represider why that might be. In Quality
sentation of some persons desires or inSoftware Management, Volume 1: Systention to create a product that fulfills someone elses desires.
tems Thinking, Jerry Weinberg points out that decisions about
An inconsistency between the product and the document might
quality (value to some person) are always political and emobe surprising, frustrating, or annoying to that person, but what
tional. Product and project decisions start with a political dereally matters is inconsistency between the product and the decisionwhose values might matter more than otherswhich
sire. So, rather than referring to an artifact, we might confirm
immediately puts peoples feelings into play. Even when there
that theres a problem by asking a person directly.
are rational elements to evaluating some thing or some situThat kind of conversation begins with an even more direct
ation, decisions always come down to desireswhat people
way of recognizing a possible problem. In my own experience,
like or dont like, want or dont want. Decision-makers might
even on those rare occasions when Ive had relatively clear,
say that theyre making a purely rational decision based on
consistent, and up-to-date information, my initial recognition
the numbers, but theyre actually deciding based on how they
of a bug begins with a feelingsurprise, confusion, frustrafeel about the numbers. What we call rationality is based
tion, suspicion, impatience, or annoyance. Sometimes I notice
on ones internal political decisions as to which feelings take
a bug because something literally looks funny. I might use a
precedence over others. The desire to be rational is a feeling
document or consult a product owner to justify calling it a
in itself.
bug, but my awareness is triggered when whatever Im doing is
A bug is a kind of problemone that threatens the value of
interrupted by a feeling of some kind.
the software by potentially thwarting some persons desire. In
Feelings can help me recognize issues, too. When I dont
software development, theres another kind of problemone




The Last Word

have immediate access to the programmers or product owners
on a project, I may become frustrated or impatient, because
long feedback loops may give bugs more time and more opportunity to hide. When Im feeling bored, that may be a signal
that the testing Im doing is unimportant, redundant, or mechanicalin which case it might be a good idea to mechanize
it. When Im confused by some ambiguous requirement, the
programmers. The programmers and the other testers might
be confused, too. Confusion and ambiguity are the floorboards
under which bugs build their nests.
As a tester, Im investigating the product on behalf of other
people. Sometimes, Im focused on acting as a representative
of the products customers. At other times, Im acting as an
extra set of eyes, ears, and hands for the programmers and
the product owners. In both cases, its important for me to be
aware of my own feelings about the product. That way, I'm
Im better attuned to the things that might represent problems
for my testing clients and to ways in which my feelings may be
influencing or distorting my perception of the products value.
When Im reporting on a problem, its usually bad news for
someone. A programmer might be disappointed because he
thought he was finished with development work on a tricky

routine. A business analyst might be embarrassed at having

forgotten to account for a particular use case. A program manager might be worried that fixing a bug will cause a schedule
slip. Awareness of other peoples feelings is the basis of empathy. When Im empathetic, Im less likely to be judgmental or
accusatory in my reporting, and that leads to a more productive relationship with my colleagues.
Empathy can also be a tool for helping to understand other
peoples reactions. For example, when I recognize the range,
variety, and intensity of emotions in a development project, I
can see why some people might be overwhelmed or frightened
by having to acknowledge them. When Im empathetic, I realize that fear could even lead someone to claim, however inaccurately, that theres no place for emotions in software testing.

index to advertisers
ALP International


Display Advertising
All Other Inquiries



Better Software (ISSN: 1553-1929) is published

six times per year January/February, March/

Catch Technologies


April, May/June, July/August, September/

October, November/December. Back issues
may be purchased for $15 per issue (plus

OpenMake Software






Entire contents 2013 by Software Quality

Engineering (340 Corporate Way, Suite 300,

Inside Back Cover

Orange Park, FL 32073), unless otherwise

noted on specific articles. The opinions


expressed within the articles and contents

herein do not necessarily express those of
the publisher (Software Quality Engineering).

SQE TrainingLive Virtual

Inside Front Cover

All rights reserved. No material in this

publication may be reproduced in any form
without permission. Reprints of


articles available. Call for details.


TesT at a HigHer LeveL

Mapping iT ouT
The Leading Conference on
S o f t wa r e t e S t i n g a n a lyS i S & r e v i e w

apriL 711, 2013

T o r o n T o , o n Ta r i o
D e LTa C H e L s e a

regiSter by february 8, 2013

anD Save uP to $300

grouPS of 3+ Save even more!

Choose from a full week of

learning, networking, and more
multi-day training Classes begin
9 in-depth Half- and full-day
3 Keynotes, 28 Concurrent
Sessions, the eXPo, networking
events, receptions, and more

s t a34 r c a
c h w e l l 2013
. c o m