5

More

Next Blog

Create Blog

A is for Agile, not Anarchy

Sunday, February 24, 2013

Scaled Agile Framework 3/5 - Program level pipeline

management and the program kanban
Introduction
Part 2 of the series concluded with the final stage of the demand management process
1 or more Epics generated from the COE level initiative and handed off to the relevant
delivery groups. In this post we will explore the lifecycle of an Epic as it travels through
the delivery group functioning as an Agile Release Train Strategic Delivery.
As described in Part 1, the adoption of the Agile Release Train in Strategic Delivery
commenced in early 2012 (April). Whilst it is being continually refined, the implementation
has gone through a number of step changes in approach.
After describing the initial adoption context and approach and covering some of the early
experiences, the post will conclude by illustrating the current operating model.

Initial adoption context

At the time of adoption, the group was modelled roughly as follows:
5 project-based Scrum teams all with the same team-shape, following a
reasonably similar model and in the final stages of delivery for their projects
1 project-based pseudo-Kanban team with 3-4 months remaining on delivery
of their project
2 stakeholder-aligned pseudo-Kanban teams with very different team shapes
and 4-5 months of work remaining in their pipeline
A newly formed System Team working on implementing a continuous
integration capability
10-15 projects running under an outsourced/offshored waterfall model
A group of system analysts fulfilling requirements and design elaboration for
the waterfall projects
18 project managers
The initial approach was defined in a series of workshops between the general manager
of the delivery group and her extended leadership team. The objectives of the workshops
were:
Establish a shared understanding of SAFe fundamentals
Determine the initial implementation model and approach to
transitioning/maturing beyond initial implementation
Determine the most effective organisational structure for the group
Upon conclusion, the group had reached agreement on the following plan:
Immediately establish an Agile Release Train involving the 5 Scrum teams,
converting them from project based to stable feature teams. All new work
entering the group would be fed to the train.
Manage the outsourced waterfall projects through to conclusion.
Allow the pseudo-Kanban teams to conclude their current commitments,
whilst gradually reshaping them to be ready to join the train upon completion.
The organisation to support this had the following structure:
Agile Pipeline Management Services (APMS) composed of the group of
project managers from the Scrum teams and the system analysts. Primary
responsibility: Manage preparation of the pipeline of work for PSIs and make
sure the teams had the support they needed in delivery.
Development Services run by the equivalent of the Release Train Engineer
and composed of the feature teams. Primary responsibility: Deliver PSI
outcomes
Deployment Services run by the deployment manager and incorporating the

Popular Posts

Scaled Agile
Framework
Applied 2/5 Demand
Management and
the Portfolio
Kanban
Introduction As described in the
introductory post , the
implementation being described is
not managing an enterprise level
p...
Scaled Agile
Framework 3/5 Program level
pipeline
management and
the program
kanban
Introduction Part 2 of the series
concluded with the final stage of
the demand management process
1 or more Epics generated...
Scaled Agile Framework Applied Part 1/5: Introduction and Context
Since encountering Dean
Leffingwells Scaled Agile
Framework(SAFe) , I have
presented the model to dozens of
senior executives a...
When is a SAFe
PSI not a PSI?
Over the last
couple of months, I
have delivered the
Leading SAFe
course half a dozen times to a
diverse set of companies and
audiences. Wh...
Scaled Agile
Framework
Applied 4/5 - Inplay work and the
program level
Feature wall
In Part 3 , we covered the program
backlog lifecycle. This post will
focus on implementation life and
feature level visualisation. We
have...
Launching your
Agile Release
Train - Preparing
for the first
Release Planning
Event
You've decided to adopt SAFe,
found your first Agile Release
Train and are embarking on the
launch journey. You have a bunch
of Safe Ag...
Scaled Agile Framework Applied
5/5 - Conclusion
What you see is all there is I've
just finished reading a book called
"Thinking Fast and Slow" by
Daniel Kahneman. Set in t...

Sign In

System Team. Primary responsibility: Liaise with enterprise release

management and operations and ensure smooth deployment and ongoing
continuous integration capability uplift.
Transition Services composed of the pseudo-Kanban teams and the
outsourced waterfall projects/PMs. Primary responsibility: Ensure graceful
conclusion of inflight work whilst preparing the teams for transition onto the
train.

Early Days
Some insights into the early experiences of the leadership group in the implementation
can be found in this earlier post, but Ill delve here into a number of key activities.
For the first month or so, it was business as usual for the feature teams. They were
already operating on the same iteration cadence, and all had a number of sprints
committed in order to deliver on existing commitments. The primary focus area was the
formation of the APMS group, but before delving into that Ill briefly cover the feature
teams.
Early days for the feature teams
The feature teams basically had two things to do:
Become stable feature teams instead of project teams
Normalise sizing.
The first was great fun and fairly trivial. Formerly known by the reference number for the
project they were working on (ie the 5531 team), they were asked to select a name for
themselves. The name had to be train related and safe for work so we wound up with
teams such as Maglev and Astrotrain. A simple change but immensely powerful when it
comes to sense of identity.
The second was painful. The concept of story point normalisation is perhaps one of the
most controversial in SAFe. Having been reluctantly convinced of the need by Dean, I still
found that no matter how carefully you approach it you will still be impacted by most of the
evils used to argue against the concept. We began by using the past 8 iterations metrics
for every team to construct a normalisation formula. Simplistically, some teams
downshifted their sizing one notch on the Fibonacci scale and other upshifted by one. It
was deceptively simple, and one of the key lessons learnt in the time since is that we
should have far more actively coached and retrospected on sizing practices.
Early days for APMS
The APMS team had 4 initial objectives:
Establish the Epic Kanban system reflecting both the active work in the feature
teams and work in the pipeline

Moving from belief

to action implementing
WSJF for value
based prioritisation
in SAFe (Part 2)
In my last post , I described the
simulation I use to teach Cost of
Delay (CoD) and Weighted
Shortest Job First (WSJF). This
often provides...
SAFe Release
Planning Tip 3 Plan with Flow
In Tip 2 , we
covered the first
morning of the
SAFe release planning event establishing the vision for the PSI
and starting the whole trai...
Getting from theory
to practice with
Cost of Delay and
WSJF in SAFe
Recently, Ive
noticed a number
of threads on the SAFe linked-in
groups indicating people are
struggling to get started with the
SAFe Weigh...

Blog Archive

2016 (1)
2015 (1)
2014 (10)
2013 (9)
November (1)
October (1)
September (2)
March (2)
February (2)

Prepare the work in pipe and logistics for the first PSI

Scaled Agile Framework 3/5 Program level pipelin...

Work with the rest of the COE (in particular the PMO) to agree governance,
financial and other details of how the release train would mesh with the
broader enterprise.

Scaled Agile Framework

Applied 2/5 - Demand
Manage...

Continue to provide project management support to the feature teams

APMS Challenge 1 Team Formation
Whilst APMS was composed of PMs, System Analysts and an architect, there was a no
exception stance on the release train being Agile top to bottom. So, the team had a
scrum-master and an aspiration to work as a cross-functional team with shared
commitments, standups, retrospectives, transparency and accountability to the feature
teams. Today, it is a high-performance team with an exceptional team ethic but the
journey was more challenging than for any other agile team Ive ever worked with. In the
end, it required a far larger change of mindset for the team members than for a typical dev
team and involved significant tuning of team balance and membership.
APMS Challenge 2 Feature Grooming: How much is enough?
The second challenge lay in the approach to grooming features for the PSI. The reality of
the funding model and the dependency complexity of the work required a fair amount of
analysis and estimation work to create ready to play features. The best way to
summarise this issue lies in the fundamental paradox of SAFe. Part of the reason the
model is so powerful is because when you present it to program/project managers and
architects they can readily map it to their existing mental models. On the other hand, the
most common criticism I hear from agilists is that it looks too formal and RUP-like. What
we experienced was that the intent of the model is too easily interpreted in practice into
waterfall-like behaviours.
The original feature grooming model involved the APMS group identifying the business
intent and features for the Epic, resolving the high level architecture and feeding it through
a feature team for enough discovery work to size it to +-30% confidence estimates. The

January (1)
2012 (8)

About Me

Mark Richards

Follow

103

View my complete profile

teams would thus hold a portion of their velocity available to be dedicated to these
activities, which would be done in parallel with fulfilling their delivery commitments. What
eventuated was the analysts/architects in the APMS group identifying and specifying the
features and handing fully articulated designs to the teams who then simply estimated the
design theyd been handed and handed their estimates back to the PM. This caused no
end of problems, and was eventually heavily course corrected as youll see once we reach
the current-day model.
APMS Challenge 3 The Product Owner construct
Prior to the adoption of the release train, finding the right product owners for agile
initiatives had been a serious challenge for the group. Given the nature of a data
warehouse, there are extremely diverse groups of stakeholders for any given initiative and
finding someone with sufficient diversity of domain knowledge, sufficient availability and
sufficient empowerment had been entirely unsuccessful. Deans Product
Manager/Product Owner separation was the key to unlocking the puzzle. Whilst we
couldnt employ it exactly as specified in the framework, the underlying principal of
separation of concerns was a huge enabler. In standard SAFe, the Product Manager is
basically the overall decision maker on prioritisation and scope for the release train whilst
the product owner travels with the team. In our context, there could be 15 different funding
sources active at once and prioritisation and scope had to belong to the sponsors
providing the money.
The solution adopted was to employ Epic Owners and Feature Owners. The Epic
Owner was the person providing the funding for an Epic, and was looked to for
engagement at key points and strategic prioritisation and scope calls for their Epic. The
Feature Owner, on the other hand, was to have deep domain expertise on a particular
feature, have delegated authority within the scope of the feature and be far more actively
engaged with the team whilst their feature was in play. The construct resolved a vast
number of issues, particularly in addressing the contrast between availability and authority.
APMS Challenge 4: Can we really do PSIs?
Ive left the biggest challenge for last. SAFe deliberately separates the deployment cycle
from the PSI cycle. Whilst the PSI produces a potentially shippable increment, the
framework allows for either deployment on the PSI boundary, multiple PSIs before a
deployment or multiple deployments within a PSI. Our intent at launch was to run 8-week
PSIs and use the PSI as a planning boundary with multiple deployments happening in
any given PSI.
The first difficulty we encountered was feature shaping. Given that the standard guidance
for a feature is small enough to be delivered in a single PSI but large enough to represent
a significant outcome for the business, there was a clash between good feature shaping
and deployment window requirements. Available deployment windows were specified by
enterprise release management, with 1 per month for minor deployments without
significant integration requirements and a varying cadence for enterprise deployments
with major integration involved. Further, the deployment window for a feature was
generally dictated by external dependencies. It was impossible to find a PSI cadence
which aligned well with deployment boundaries, and shaping features to PSIs was also
largely impossible as we would regularly encounter features which would need to be
deployed 1 iteration into a PSI but needed 2-3 iterations to implement.
Feature shaping was difficult but not insurmountable in the end, we could have resolved
it by allowing cross-PSI features whilst still maintaining strong size limitations. However,
the bridge-too-far lay in establishment of sufficient funded pipeline. Once again driven by
enterprise level gating cycles and funding processes, an 8 week PSI that fully funded and
committed the entire release train was just not emerging.
By this point, however, we had established significant momentum with every aspect of the
release train other than the PSI launch. The pipe was operating, the work was flowing into
the teams, deployments were going out the door and momentum was tangible. One large
leadership workshop later, we made the call: know that PSIs will start eventually, but in
the meantime develop compensating mechanisms and keep maturing the release train.
Our focus then turned to compensating mechanisms what were the key things we lost
by not having the PSI, and how could we utilise our knowledge of the principals to achieve
the purpose. To my mind, we identified the first 3 of the 5 key ingredients initially:
The power of having the whole release train in a planning workshop together
from the perspective of developing a sense of release train as one large team
with shared outcomes as well as release train as team of teams
Employing a deliberate set of disciplines to groom out 4 iterations worth of
backlogs at the release train level and ensure that visibility, dependency, risks

and issues were fully articulated at the story level for all active features.
Release train level Inspect and Adapt cycles
Maintenance of a strategic view of capability buildout on the platform that
effectively capitalises on synergies between features.
Strategic showcasing and communication of PSI level outcomes to draw
together the whole of product view of progress for broad stakeholder groups
and avoid fragmented and isolated demonstration and communication feature
by feature/epic by epic to narrow stakeholder groups.
Id like to write that we rapidly implemented effective compensation strategies, but Id be
lying. Being brutally honest, Id suggest that we initiated effective strategies for the first
and third items immediately. Time and painful lessons triggered focus on the second and
fourth, and the fifth is currently in the spotlight.
The first strategy was the concept of PSI-lite, better known as Unity Day. A 1-hour
whole of train session at the start of each sprint aimed at building a sense of train as
team and ensuring everyone started the sprint on the same page. These have been
incredibly powerful from a cultural transformation perspective. They are always attended
by numerous visitors, and repeat attendees regularly comment on the tangible change in
atmosphere over time. (for a deeper treatment, see this post)
The second strategy involved the establishment of Team Loco 131, composed of the
extended leadership group and functioning as a program level continuous improvement
team. Their stories are generated through learning generated by various forms of
retrospective, and the leaders are fully bought into the lean management model of taking
responsibility for improving the system of work.
Whilst the story of addressing backlog grooming challenges is too long and torturous to
describe here, I will offer the following. Many organisations find the investment of putting a
whole program and set of stakeholders in a room for 2 days planning every 8-10 weeks
unpalatable. For Strategic Delivery, this commitment was one of the major sticking points
in management workshops although the general manager bought in and was willing to
support it, most of her leadership group felt it was unjustifiable. Ive heard similar stories
from many other SAFe Program Consultants. Our journey, painful lessons and ongoing
investment required to compensate in the area of release train level backlog
grooming/planning have utterly convinced me of the vital importance of this activity. To put
my systems thinking hat on, failure to implement effective PSI planning creates an insane
amount of failure demand across the entire release train. Weighing the cost of the failure
demand against the investment required for the PSI planning days provides a scale very
heavily tipped towards the investment.

The Program Kanban

Before digging into the specifics of how the Program (or Epic) kanban operates today,

there are two final points to cover off. The first involves a holistic view across both the
initiative level kanban system and the epic level. Astute readers will notice a number of
glaring inefficiencies between the two. Whilst Ill cover this in a little more detail in the
concluding post for the series, I will offer a reminder at this time that the purpose of explicit
visualisation in a kanban is to bring to light waste and opportunity.
The second point is that whilst this blog series focuses on wall visualisations, every wall
depicted is replicated in Rally with far more underlying detail. The Rally Portfolio
Management capability is a key enabler in effective management and measurement of the
system, whilst the walls are vital for tactile insight and facilitation of communication.
The Epic Kanban is, obviously, dedicated to the flow of an Epic through the release train
value chain. Every card on the wall is an Epic. The value chain is divided into 5 phases,
each with a number of states:
Initiate Phase: Transition from demand management and connection with
stakeholders.
Discovery Phase: Feature elaboration and estimation refinement
Launch Phase: Approvals, funding and preparation to drop into teams for
implementation
Evolve Phase: Implementation
Operate Phase: Deployment and follow-through to ensure happy users and
successful transition to operations.
One of the primary aspects of the phases is to indicate the flow of responsibility between
APMS, Development Services and Deployment Services. For any given phase, one group
will be moving the Epic and the others will be either supporting or being kept informed.
You'll notice on the photo there are phases which have "rows" as well as "columns". The
rows indicate teams, whilst the columns indicate states. Any state which has rows is one
where a feature team has the ball - you'll notice photos in the start of each swimlane in the
"Prepare Discover" state indicating which team is which.
On the day PSIs become viable for this train, the phases and states will change very little.
Initiate & Discover effectively involve the identification, sizing and prioritisation of features
in preparation for PSI injection and maintenance of roadmap, Launch equates to locking in
PSI content and executing PSI planning, and Evolve is delivery of the PSI.

The Initiate Phase

This phase is led by APMS and basically loads the Epic onto the train. The primary goals
are as follows:
Execute handover with the demand management group (PMO and Validate
Entry states)
Connect with the Epic Owner, ensure their value drivers for the epic are clearly
articulated and introduce them to the operating model (Envision state)
Identify candidate feature owners ( Envision state)
Determine the feature team which will execute discovery, the sprint when it will
occur, and ensure logistics are organised for discovery to proceed smoothly.
(Prepare discovery state)
This phase is the primary evidence of the response to APMS Challenge 2. It explicitly
limits the amount and nature of the work done by APMS before connecting the Epic and
Feature owners with the teams for further analysis. In effect, it eliminated roughly 70% of
the preparation work that was initially being conducted.
Looking at the wall, you'll see one epic that's just landed from demand management (with
the smurf avatar on it indicating the project manager who's pulled the card). One is in the
envisioning state, one has completed envisioning but is blocked from being pulled into
discovery preparation (the big red "X" on it). Finally, there is one epic being prepared to
drop into Discovery with the Astrotrain team.

The Discovery Phase

Discovery is led by the feature teams, and involves working with the Epic and Feature
owners to discover the features and explore the feature level requirements sufficiently to
articulate and size the outcomes. A feature team will typically have roughly 10% of their
sprint capacity dedicated to discovery work, with discovery for most epics concluding
within a single sprint. The flow of states is as follows:
Kickoff Discovery: Workshop with Epic Owner and all Feature Owners to gain
understanding of the vision for the epic, determine the exploration workshops
required, identify key risks/issues and ensure the stakeholders understand the
discovery process.
Explore Features: Detailed workshops with feature owners to validate the
feature composition, high level stories and feature level acceptance criteria.
Consolidation: Consolidation of the outputs from feature exploration
workshops, estimation and preparation of a high level release plan.
Discovery Outcome: Presentation back to Epic and Feature Owners of
consolidated discovery outputs for validation and agreement of outcomes
The key outcome of Discovery is a set of prioritised, well-groomed features which have
been estimated to a +- 30% level confidence and are ready to play for implementation.
The other potential outcome is an Epic split. Once an Epic starts to look larger than half a
million dollars, the preferred approach is to identify the highest priority features for initial
implementation and carve off the remainder into a new Epic for a future release. In PSI
parlance, its making sure no more than roughly a PSI worth of features are progressed at
once on the Epic.

Looking at the wall, you'll see Astrotrain finalising Discovery for one Epic, Kaizen (3rd row)
finalising one and kicking off another and Maglev consolidating one set of outputs whilst
kicking off another. It's unusual for a team to have more than one Epic in Discovery at a
time, in this case it's likely that they are small Epics which minimal feature elaboration
required.

Launch Phase

This phase is reasonably self-explanatory. The focus returns to APMS, who look after
gaining the appropriate approvals of the discovery outcomes and funding to proceed.
Whilst in an ideal world this phase would be traversed rapidly to achieve a smooth flow
from discovery to evolve, the reality is epics regularly sit here for significant periods whilst
business cases gain approval and enterprise gating cycles complete. It has generated one
simple and obvious insight the better the Initiate and Discovery phases are executed,
the shorter and smoother the launch phase process.
Launch concludes with evolve preparation. The feature team or teams to execute evolve
are identified, logistics are organised for the release planning session at the start of
evolve, the commencement sprint is locked in and the feature teams re-familiarise
themselves with the discovery outcomes.
The key aspect to evolve preparation is feature team selection. Obviously, the ideal
scenario is for the entire epic to land with a single feature team and for the evolve feature
team to be the same one which executed discovery. However, reality intervenes. Whilst
evolve capacity availability is utilised in selecting the discovery team, when the launch
phase is protracted that team is often not available for evolve. In these scenarios, some
capacity is reserved in the backlog of the discovery team to ensure they are available to
support the kickoff and rapid knowledge uptake of the evolve team(s). Likewise, where the
Epic is too large to be executed by a single feature team, work will be spread across
feature teams on a feature by feature basis.
Looking at the photo, you'll notice a few things. Firstly, queues are very visible :)
Secondly, although the APMS team runs a kanban system for their work (as hinted at by
the epic level cumulative flow diagram and control chart on the wall), they also track the
sprint (or iteration) cadence. Their priorities on any given day are driven by where in the
sprint cadence the feature teams are. Finally you'll see the cards with post-its on them

('1', '2' and '4'). These would have been identified at the standup that day as needing
urgent attention.

Evolve Phase

Implementation and deployment of the epic will be covered in the next post, so I will keep
this section intentionally light. The states are as follows:
Plan Release: Execute the release planning which would normally occur in a
PSI planning session. The stories identified in discovery are broken down into
sprint-size and scheduled across sprints based on available capacity to
create an initial release plan. One of the key outcomes of this workshop is
establishment of working agreements with the feature owners. They know
which sprints their feature will be active in, when acceptance testing for their
feature will occur and can plan their availability accordingly.
Implement: Build and acceptance testing of features.
Integrate and Prep Deployment: This phase has a split personality depending
on deployment type. For an independent deployment, it takes roughly a week
and involves standard integration, deployment hardening and operational
handover preparation activities. For deployments associated with an enterprise
release (75-80%), it lasts 4-5 sprints. The work cannot be accelerated, as the
timing and nature of involvement is dependent on enterprise level coordination. Teams thus wind up with a trickle-feed capacity reservation during
this time.
Looking at the wall, you'll see Astrotrain juggling 3 Epics in implementation, with a
'stakeholder concern flag' on one probably giving hints as to why another Epic is being
prepared to drop into the team. Moving down the board, Jacobite are implementing a
single (large) Epic, Kaizen have one in implementation and are most likely getting close to
done there based on the fact they're in release planning on a second. Maglev have a
more even spread, with 5 epics at various stages of Evolve.
Fans of limiting WIP (presumably most readers of this blog) are probably looking at this
picture and saying "why aren't they finishing one Epic before starting the next". My first,
somewhat glib, response is "isn't it great that it's so obvious". On a more serious note, in
a highly integrated world you often can't go at full pace on a single initiative. It has fast
times and slow times as you synchronise with your interface partners and resolve other
such dependencies. Having the flexibility to deal with this has been a very important
aspect of the framework success.

Operate Phase

Under the leadership of deployment services, the epic is deployed and in operation.
Whilst numerous tidy-up activities around production verification testing and governance
closure occur here, the key component introduced has been the Epic retrospective. This
closes a significant gap in the PSI Inspect and Adapt cycle. Whilst numerous release
train feedback cycles existed, what was missing was the stakeholder learning. The
feature team holds a retrospective with their epic and feature owners looking at the whole
of lifecycle experience, and an invaluable source of strategic learnings is introduced.
The second state is Spanked. It is the cause of much hilarity amongst visitors, but has
become part of the cultural identity of the release train. Originally introduced for some
personality on a team kanban wall by an English scrum-master, it stands for done done.
The Epics in production, the customers are happy, and the team has spanked it.Looking
at the wall, it's fairly easy to conclude that it's not long since a major deployment window.

Conclusion
The daily rhythm of all the groups will be covered in the next post, but its worth making a
few concluding comments here. The first is to play back to Deans concept of the release
train as a self organising program. Effective visualisation of the epic lifecycle is a vital
ingredient, as it generates so much knowledge at a glance of the state of play of the
entire value chain. Within seconds, the group can identify what epics are where in the
value chain, who has carriage, where the queues are and what needs focus. And of
course, metric analysis through cumulative flow and cycle time charting at the Epic level.
All of this across a system of work that at any given time generally has 30-40 epics
somewhere in the value chain.
I'll close with a hint of things to come when we talk about results in the final post for the
series. You might remember a mention of 18 project managers in the initial context at the
beginning of the post - there are now 3.
PS Part 4 in the series is now posted and deals with the program level feature wall and inplay work.

Posted by Mark Richards at 6:06 AM

+5 Recommend this on Google

No comments:
Post a Comment
Enter your comment...

Comment as:

Publish

Google Account

Preview

Newer Post

Home

Older Post

Subscribe to: Post Comments (Atom)

Simple template. Powered by Blogger.