Scaled Agile Framework and Testing Tutorial Testnet 2018 Handouts

Scaled Agile Framework and Testing
Half-Day Tutorial
By
Mette Bruhn-Pedersen
Safe Journey
TestNet Autumn Event 2018
10-10-2018
NBC Nieuwegein, Nieuwegein, Netherlands
My Background
QA & Testing Agile

+10 years of software testing and +10 years of agile experience
QA experience • Agile Team Member
• Testing • Test Manager
• Test Management
+10 years in DSTB and ISTQB +4 years of SAFe® experience

• Chair of DSTB • Agile Team Member
• ISTQB Governance Officer • Community of Practice (CoP)
• Vice Chair of ISTQB Examination Facilitator
Working Group • Agile Coach
• Agile Transformation Leader
Co-author of the eBook:

“Testing and Quality in the Scaled Agile Framework for Lean Enterprises”
Scaled Agile Framework and Testing 2
Safe Journey
Agile Transformation & Personal Growth
Agile Methodologies Training

We help teams, departments We train and educate people in
and companies become more all roles to help them gain
agile using their own knowledge and increase their
methodology or a framework Agile competencies and skills
like SAFe
Agile Maturity & Growth Coaching

We facilitate and measure We coach people one-to-one to
effective, intentional growth of help them grow and safely
people, teams, departments overcome personal challenges
and organisations

What is your Level of Experience with Agile?
Value on
Description
Scale
0 No experience at all
Theoretical knowledge (articles,

1-4
training course, etc.)
Less than 1 year practical

5
experience (the origin)
Between 1 year and 15 years of

6-9
practical experience
More than 15 years practical

10
experience
Go to www.menti.com and use the code

Agenda
• Introduction to • Organisational
Agile at Scale Challenges
• QA and Testing • Implementation

Challenges when and Customisation
Scaling Agile Challenges
Photo by Nathan Dumlao on Unsplash

Introduction to
Agile at Scale

Why Scale
• Business agility is key as disruption of sectors and industries

becomes the new normal.
• Typical business benefits from scaling agile using SAFe®*:
Employee Satisfaction Productivity

10-50% happier, more 20-50% increase in productivity
motivated employees
Time to Market Quality

10-50% faster time-to-market 25-75% defect reduction
* SAFe, scaledagileframework.com/about/ Scaled Agile Framework and Testing 7

Levels of Scaling
Individual Teams Collection of Teams The Entire Company

One or a few Agile teams A program, department or All departments and
business unit business units

Agile Methodologies at a Glance
Scalability
DSDM
SAFe®
DAD
Spotify LeSS
Nexus
SoS
Scrum
Kanban
XP
Prescriptiveness
Information combined from several sources:
Agile Scaling Knowledgeable (ASK), Agile Scaling, agilescaling.org
12th annual State of Agile Report, Collabnet VersionOne, collab.net
World Quality Report 2017-18, Ninth Edition, Capgemini, Micro Focus, Sogeti, capgemini.com
Kanban and Scrum making the most of both, Henrik Kniberg & Mattias Skarin, 2010, C4Media
Which Agile Methodologies

Does your Company Use?

SAFe® Big Picture v. 4.6
Copyright © Scaled Agile, Inc.
See latest description on http://v46.scaledagileframework.com/# Scaled Agile Framework and Testing 11

The SAFe® House of Lean
See latest description on www.scaledagileframework.com/lean-agile- Scaled Agile Framework and Testing 12

mindset TestNet Autumn Event 2018
Long-Lived Agile Release Train (ART)
See latest description on www.scaledagileframework.com/agile-release- Scaled Agile Framework and Testing 13

train TestNet Autumn Event 2018
Fully Cross-Functional Agile Release Train (ART)
See latest description on www.scaledagileframework.com/agile-release- Scaled Agile Framework and Testing 14

train TestNet Autumn Event 2018
Similar Roles in Scrum and SAFe®
Agile Team Agile Release Train
Product Owner Team Backlog of user stories Product Mgmt ART Backlog of Features
Release Train
Scrum Master Team Ways of Working Engineer
ART Ways of Working
Define, build, test Architectural runway

Dev Team System Architect
and implement solutions and technical excellence

Similar Events in SAFe®
Agile Team Agile Release Train
Iteration Planning Program Increment Planning

2-4 weeks 8-12 weeks
Standup Scrum of Scrums & ART Sync
Daily Weekly
Review System Demo
User stories - end of sprint Features - end of program increment
Retro Inspect & Adapt
More tactical More strategic

QA and Testing Challenges
when Scaling Agile

What are your Biggest Challenges?

Transformation Challenges - Literature Review*
Challenge categories Percentage of case organisations
Agile difficult to implement
Integrating non-development functions
Change resistance
Requirements engineering challenges
Hierarchical management and organizational boundaries
Lack of investment
Coordination challenges in multi-team environment
Different approaches emerge in a multi-team environment
Quality assurance challenges
0 5 10 15 20 25 30 35 40 45 50
* Challenges and success factors for large-scale agile

transformations: A systematic literature review, Digert, K. et al, The
Journal of Systems and Software, 119 (2016) 87–108 Scaled Agile Framework and Testing 19
Quality Assurance Challenges when Scaling Agile
• Accommodating non-functional testing

Quality
Assurance
Challenges
• Lack of automated testing
• Requirements ambiguity affects QA

Building a New Online Application
- An Example of Dealing with QA Challenges
• Target both consumer and

business customers
• Focus on customer
experience
• Replace an existing
application
• Reuse existing data sources
Photo by rawpixel on Unsplash

Project Background
• Project initiated
• Approx. 80 people
• Teams by function
• Several countries
• Demo version ready in 6
months

Introducing Agile Using SAFe®
Organise
Build Start rolling- Balance UX Deal with
teams in Agile
development wave PI and business dependencies
Release Train
infrastructure planning rules to other teams
(ART)

Changes in QA & Test

• Test managers and testers became part of the Agile teams
• Some Agile teams did not have professional QA or test

professionals
• The Agile teams were responsible for all QA and test

activities
• Established a Community of Practice for testing (topic-based

instead of role-based)

Accommodating Non-Functional Testing
See latest description on www.scaledagileframework.com/features-and- Scaled Agile Framework and Testing 25

capabilities/ TestNet Autumn Event 2018
Requirements Ambiguity Aﬀects QA

Picture created by Ole Luntang Christensen Scaled Agile Framework and Testing 26
Lack of Automated Testing
Picture created by Ole Luntang Christensen Scaled Agile Framework and Testing 27
Lessons Learnt
Technical Excellence Agility
• DevOps • Team focus vs. solution focus

• Infrastructure • IP sprint not HIP sprint
• Built-in quality • Active topic-based CoP
Organisation Time Horizon
• Include teams outside the ART • Short-term vs. long-term

as much as possible
• Project delivery vs.
• Get the business to lead organisational growth
• Leader involvement

Organisational Challenges
- Transitioning to New Roles


QA Specialist or Tester in a Dev Team
Daily
Scrum
Develop- Sprint
Sprint Sprint
ment & Retro-
Planning Review
Release spective
• Test planning • Test planning • Test planning • Test

• Test analysis • Test monitoring
• Test completion
and control completion
• Test analysis
• Test design
• Test
implementation
• Test execution
• Test
completion

Test Manager in a Dev Team
Daily
Scrum
Develop- Sprint
Sprint Sprint
ment & Retro-
Planning Review
Release spective
• Test planning • Test planning • Test planning • Test

• Test analysis • Test monitoring
• Test completion
and control completion
• Test analysis
• Test design
• Test
implementation
• Test execution
• Test
completion

Which Roles in an ART Inherit the Typical
Responsibilities of a Test Manager?
Test Reporting
Defect
Management
Product
Management
Test Planning Shared Services
(features)
Test Approach
Test
Release Train Environments System Team
Engineer
Test Tools
?
System Architect
Agile Team


Test Managers on Large Solution Level
- An Example
See latest description on www.scaledagileframework.com/large-solution-level/
• Support the Agile teams in the ARTs throughout the PI

• High level coordination of test activities across ARTs and
programmes
• Implement the corporate test strategy
• Define QA enabler epics in collaboration with Solution Trio
• QA and Testing CoP Lead

Challenges Implementing SAFe®

Implementation Diﬃculties when Scaling Agile
• Misunderstanding agile concepts

• Lack of guidance from literature
Agile difficult
to implement • Agile customized poorly
• Reverting to the old way of working
• Excessive enthusiasm

Crucial Aspect of SAFe®
“PI planning is essential to SAFe:

If you are not doing it, you are not doing SAFe.”

Program Increment (PI) Planning
• All people on the ART and relevant stakeholders meet face-to-

face for 2 days to plan the next PI
• Remote teams are planning at the same time using video
conferencing.
See latest description on www.scaledagileframework.com/pi-planning/ Scaled Agile Framework and Testing 39

A Typical Agenda for PI Planning

Program Backlog
• Top 10 features
• Both Business and Enabler Features
• General and specific NFRs
See latest description on www.scaledagileframework.com/ Scaled Agile Framework and Testing 41

program-and-solution-backlogs/ TestNet Autumn Event 2018
Team Planning Area
During PI Planning the Agile teams plan each iteration by:

• Defining and estimating stories
• Setting team PI objectives with assigned business value
• Identifying and managing risks
Sprint 1 Sprint 2 Sprint 3 Sprint 4 User Story

Exploration Enabler
Risks & Dependencies
PI Risks
Objectives

Program Board
The plan for the next PI is documented on the Program Board

Program Risks
• Risks, Impediments, Opportunities (RIO) are ROAM’ed on the

RIO Board:
• Resolved - The teams agree that the issue is no longer a

concern (different from mitigated)
• Owned - Someone on the train takes ownership of the item
since it cannot be resolved during PI Planning
• Accepted - Potential problems that has been understood and
accepted by the teams and stakeholders
• Mitigated - The teams have planned to reduce the likelihood
and/or the potential impact of the problem

Committed PI Objectives

What Would You Customise?

Suggestion to Enhance the Program Board
Sprint 1 Sprint 2 Sprint 3 Sprint 4 IP PI 2 >>

PI 1 PI 1 PI 1 PI 1
• Dedicated swim lane for

Business Segment 1
Releases/
Milestones Segment 2
larger or more complex Technical Releases
Release Testing
testing activities, e.g., Team 1
Team 2
Release Testing Team 3
Team 4
Team 5
Team 6
Deliveries to or from others

(e.g.,suppliers, other ARTs)
= Feature = Business Release = Release Testing
= Dependency = Technical Release

Simple Example of a Dedicated Swim Lane on the

Program Board
Sprint 1 Sprint 2 Sprint 3 Sprint 4 IP PI 2 >>

PI 1 PI 1 PI 1 PI 1
Business Segment 1
•
Releases/
A feature completed by one Milestones Segment 2
Technical Releases
team is release tested in Release Testing
Team 1
the same iteration Team 2
Team 3
Team 4
Team 5
Team 6


Another Example of a Dedicated Swim Lane on the
Program Board
• Feature completed by three Sprint 1

PI 1
Sprint 2
PI 1
Sprint 3
PI 1
Sprint 4 IP
PI 1
PI 2 >>
teams Business
Releases/
Segment 1
Milestones Segment 2
• Two teams complete their Technical Releases
Release Testing
part in the first iteration Team 1
Team 2
• Third team complete their Team 3
Team 4
part in the second iteration Team 5
Team 6


Suggestion to Enhance the Team Planning Area
Sprint 1 Sprint 2 Sprint 3
• Team planning area for PI

Planning
• Add an area for putting PI Risks Quality
things related to QA & Objectives & Testing
Testing
User Story
Exploration Enabler
Simple Example of an Enhanced Team Planning Area
• Add an area for putting Sprint 1 Sprint 2 Sprint 3

things related to QA &
Testing e.g.,: X Y
• Things which impact

multiple stories
PI Risks Quality
• QA & Testing activities Objectives & Testing
unique to this PI X
• Help to other teams, e.g., Y

create test data
User Story
Exploration Enabler
Summary
• SAFe is one of the most popular and prescriptive frameworks

• QA & testing are typically not the biggest challenges
• Guidance on how to map typical QA and testing roles besides
testers is missing
• Customising a framework like SAFe is crucial for success

Feedback on the Tutorial

Contact Details
Mette Bruhn-Pedersen
Agile Transformation Leader and Partner
E: contact@safejourney.dk
W: safejourney.dk


Scaled Agile Framework and Testing Tutorial Testnet 2018 Handouts

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Scaled Agile Framework and Testing Tutorial Testnet 2018 Handouts

Încărcat de

Drepturi de autor:

Formate disponibile

Scaled Agile Framework and Testing

TestNet Autumn Event 2018

NBC Nieuwegein, Nieuwegein, Netherlands

QA & Testing Agile

+10 years in DSTB and ISTQB +4 years of SAFe® experience

Co-author of the eBook:

Agile Methodologies Training

Agile Maturity & Growth Coaching

Scaled Agile Framework and Testing 3

What is your Level of Experience with Agile?

Theoretical knowledge (articles,

Less than 1 year practical

Between 1 year and 15 years of

More than 15 years practical

Go to www.menti.com and use the code

Scaled Agile Framework and Testing 4

• QA and Testing • Implementation

Photo by Nathan Dumlao on Unsplash

Scaled Agile Framework and Testing 5

Scaled Agile Framework and Testing 6

• Business agility is key as disruption of sectors and industries

Employee Satisfaction Productivity

Time to Market Quality

* SAFe, scaledagileframework.com/about/ Scaled Agile Framework and Testing 7

Individual Teams Collection of Teams The Entire Company

Scaled Agile Framework and Testing 8

Which Agile Methodologies

Go to www.menti.com and use the code

Scaled Agile Framework and Testing 10

Copyright © Scaled Agile, Inc.

See latest description on http://v46.scaledagileframework.com/# Scaled Agile Framework and Testing 11

The SAFe® House of Lean

Copyright © Scaled Agile, Inc.

See latest description on www.scaledagileframework.com/lean-agile- Scaled Agile Framework and Testing 12

Copyright © Scaled Agile, Inc.

See latest description on www.scaledagileframework.com/agile-release- Scaled Agile Framework and Testing 13

Fully Cross-Functional Agile Release Train (ART)

Copyright © Scaled Agile, Inc.

See latest description on www.scaledagileframework.com/agile-release- Scaled Agile Framework and Testing 14

Agile Team Agile Release Train

Define, build, test Architectural runway

Scaled Agile Framework and Testing 15

Similar Events in SAFe®

Agile Team Agile Release Train

Iteration Planning Program Increment Planning

Standup Scrum of Scrums & ART Sync

Review System Demo

User stories - end of sprint Features - end of program increment

Retro Inspect & Adapt

More tactical More strategic

Scaled Agile Framework and Testing 16

Scaled Agile Framework and Testing 17

What are your Biggest Challenges?

Go to www.menti.com and use the code

Scaled Agile Framework and Testing 18

Challenge categories Percentage of case organisations

Agile difficult to implement

Integrating non-development functions

Requirements engineering challenges

Hierarchical management and organizational boundaries

Coordination challenges in multi-team environment

Different approaches emerge in a multi-team environment

Quality assurance challenges

* Challenges and success factors for large-scale agile