Case Study: ATCSoft

The ATCSoft project was launched, and steady progress was made When the team set out to integrate the ATCSoft application with existing RADAR equipment, however, they hit a snag

The team members could not figure out how to integrate the systems
The RADAR system did not have appropriate physical connections, nor was there an appropriate driver for the interconnection

The project manager had to hire engineering consultants to work out the integration details This diversion took 2 months and cost a significant amount of money (additional labour, consultancy fees, and business value)

Case Study: PathFinder 2.0

The PathFinder project was started in August

Rory Thompson was the star developer

His knowledge of neural nets inspired him to suggest that a neural net implementation of a PathFinder would be a good idea He wrote up much of the foundation code for the neural network Initial tests showed a definite improvement in the PathFinder performance, and more realistic, human-like, decisions

Despite the large salary increase he was offered, Rory took a good offer with another firm
When some tests showed that the neural net was not fast enough to make real-time decisions, the team had no immediate answers A bug was found that sent the avatars wandering aimlessly around the maze in a circuit, when certain rare conditions were present Again, the team had no idea how to approach the problem

What happened?
What happened in both of these case studies?

What happened?
What happened in both of these case studies?

In the ATCSoft scenario, a technical problem ended up stalling development

This could easily have become a complete disaster, if it were not possible to integrate the systems

What happened?
What happened in both of these case studies?

In the ATCSoft scenario, a technical problem ended up stalling development

This could easily have become a complete disaster, if it were not possible to integrate the systems

In the PathFinder scenario, a personnel problem was at fault

The teams over-reliance on a hero was their downfall Taking the hero out of the equation stalled development

What happened?
What happened in both of these case studies?

In the ATCSoft scenario, a technical problem ended up stalling development

This could easily have become a complete disaster, if it were not possible to integrate the systems

In the PathFinder scenario, a personnel problem was at fault

The teams over-reliance on a hero was their downfall Taking the hero out of the equation stalled development

In both scenarios, the team neglected their risks

It is critical for a project team to understand and plan for risks

Risk Mitigation
In the ATCSoft project, the team should have investigated the integration of various systems at the start of the project

Given adequate time, the integration could have been worked out before it was needed

Risk Mitigation
In the ATCSoft project, the team should have investigated the integration of various systems at the start of the project

Given adequate time, the integration could have been worked out before it was needed

In the PathFinder project, Rory should have been the projects neural network consultant

He could have thoroughly documented the neural network code as it was developed He could have had seminars for team members, explaining the concepts of neural networks Understanding neural networks, the team would have a better chance of carrying on without Rory

Risk Assessment

60-499 Project Management: Tools & Techniques

Risk Assessment
The following describes the risk assessment process:

1. Identifying risks
2. Estimating a risks cost/effects

3. Estimating a risks likelihood

4. Identifying alternatives

5. Evaluating/comparing alternatives
Once risks are assessed, a project manager should plan for them

Risk Assessment
Risk Identification

60-499 Project Management: Tools & Techniques

Risk Identification
The first step in risk analysis is to identify the projects risks

Each project has its own set of unique risks How do you identify something that could potentially be hidden until it is too late? However, risk identification can be made easier using categories of risk
This leverages the knowledge of many project managers who have experienced risks

Identifying risks seems like a dark art

Categories of Risk
Here is one way to categorize risks:

Technical risks (related to using a particular technology)

Performance Reliability Availability Complexity

Project management risks

Poor resource allocation Poor planning Poor prioritization

Organizational risks
Lack of support or resources Inadequate or inefficient management Interference from other projects & management agendas

p.65

Categories of Risk
Here is one way to categorize risks (continued):

Constraint risks
Deadlines Resources

Business risks
Marketability Timing Vendor delays Economic conditions

External risks
Changing laws and regulations Dependence upon suppliers and contractors

p.65

When do we identify risks?

There are a two major approaches to risk identification

Identify risks before the planning phase

Some risks may be difficult to spot when looking at requirements at a high level

Identify risks after the planning phase

It is useful to know risks before the planning phase, so that extra time can be dedicated to their mitigation

A good compromise is to perform risk identification during the planning phase:

After creating the work breakdown structure Before creating the schedule

Common Risks
These risks are related to schedule:

Feature creep
New features are frequently added after development has started

Requirements gold-plating
Too much documentation

Implementation gold-plating
Developers are working on the perfect implementation

Inadequate design
Too little attention has been paid to design

Overly optimistic schedules

Management pushed schedules down, rather than schedules work their way upward from developers

Poor motivation/weak personnel

Developers are working at a less-than-optimal pace

Silver-bullet syndrome
A trendy technology was expected to produce the equivalent to 10,000 lines of code in only 50 lines of code

Contractor failure
A contractor lacked expertise/commitment needed to do the job on schedule

Risk Assessment
Estimating Risk Cost & Effects

60-499 Project Management: Tools & Techniques

Estimating Risk Costs & Effects

Estimating the costs & effects of a risk is dependent upon the risk

e.g. A project using a new technology might realize that the technology is inadequate or unreliable
Now, the application must be retrofitted to another (trusted) technology Much of the software may need to be replaced The cost in this case is the cost of developing the obsolete components In addition, there may be hidden costs due to delays (such as customer confidence or personnel availability)

Estimating Risk Costs & Effects

Estimating the costs & effects of a risk is dependent upon the risk

e.g. In some projects there is a risk that a key developer will leave the project
If the key developer leaves, what will it take to replace her? Given market conditions, you might estimate a replacement in 2 months Some project deliverables might be delayed by up to that amount in her absence Also, you may have to consider signing bonuses, relocation expenses, travel expenses, and other hiring costs It depends on the project whether or not these costs are considered high

How to estimate cost?

To estimate the cost, imagine a scenario where the event occurs

e.g. Ok, Sarah has left the company. What do we do now?

We could promote Gerard to team leader We would need one more developer with technological expertise

Our corporate headhunter estimates hiring will take 10 weeks

e.g. PHP didnt solve the problem. What now?

We could use Ruby on Rails We would need to send our developers to training courses

In-depth RoR training courses are 4 months

Risk Assessment
Estimating Risk Likelihood

60-499 Project Management: Tools & Techniques

Estimating Risk Likelihood

Like risk cost, risk likelihood also depends on the risk

e.g. The likelihood that a technology will fail can usually be estimated accurately Tristan: How likely is it that the technology (e.g. PHP) will fail to meet our expectations? Carlos: PHP has been used for many projects successfully. There have been many failed PHP projects, but very few cite the technology as the cause of the failure. On the other hand, there have been many very high-profile PHP projects. Carlos: Consider FaceBook, which has similar (but more complex) functionality to our GroupWare application. Tristan: FaceBook uses PHP? That is similar to our project. Carlos: I cant forsee any features we will need that will be impossible in PHP.

Estimating Risk Likelihood

Like risk cost, risk likelihood also depends on the risk

e.g. How likely is it that Sarah Windermere will leave the project?
Her project manager may be aware of her job satisfaction, or her peers However, a face-to-face meeting to assess her job satisfaction has no substitute

Example:
Tristan: Sarah, weve been very happy with your performance on previous projects. You will be an integral part of this upcoming project. Are there any concerns you have about this project or your job? Sarah: Not really. I like the challenges. Tristan: If you do have any concerns, I want you to tell me right away. Your satisfaction and motivation are very important to me. Sarah: Actually, I have been having a tough time working with Gerard. He doesnt respect my leadership, and tries to rally the team against me. Tristan: How does the rest of the team feel about his actions? Sarah: Many of them either have confronted him about or have told me they dont agree with him. Tristan: Ill have a talk with your team to confirm what youve said, but it sounds like Gerard and I need to have a talk about each of your responsibilities.

How to estimate likelihood?

The best way to estimate is to ask the people closest to the problem

e.g. For Sarah, the best person to ask would be her (followed by her direct supervisor) e.g. For PHP technology, anyone who has used PHP for a previous project

Risk Assessment
Identifying Alternatives

60-499 Project Management: Tools & Techniques

Identifying Alternatives
e.g. What are the alternatives to PHP?

JSP/J2EE ASP.NET Ruby/ROR Perl

Identifying Alternatives
e.g. Are there alternatives to Sarah?

Gerard Lepalme: Has leadership, but lacks the technological expertise Helen Driskoll: Knows some of the technology, but is very inexperienced

Risk Assessment
Evaluating & Comparing Alternatives

60-499 Project Management: Tools & Techniques

Evaluating & Comparing Alternatives

e.g. Let us examine the alternatives to PHP:

JSP/J2EE
Much of our existing staff is familiar with this technology Using this technology would require a few days of training Architecture should be more robust

ASP.NET
We have no in-house personnel trained in this technology Using this technology would require at least one month of training Development of user interfaces, however, should be made easier

Ruby/ROR
We have only a few in-house developers trained in this technology Using this technology would require weeks of training Development should be made easier Architecture should be more robust

Perl
We have no in-house personnel trained in this technology Using this technology would require at least one month of training

Evaluating & Comparing Alternatives

e.g. Let us examine alternatives to Sarah:

Gerard Lepalme: Has leadership, but lacks the technological expertise

Gerard is a take charge kind of person He is also a get it done kind of person However, he is not familiar with XML and many other technologies we plan to use

Helen Driskoll: Knows some of the technology, but is very inexperienced

Helen knows XML and a few other technologies we plan to use However, Helen is just starting her career She has difficulty being assertive and taking charge She doesnt command respect from her colleagues Her development itself is slow

Assigning Probabilities
There are many approaches to assigning probabilities

It often doesnt matter if you only care about the value relative to other risks Very low: Low: Medium: High: Very High: 0.05 0.20 0.40 0.60 0.80

However, this is a good rule of thumb:

Risk Assessment
Bringing It All Together

60-499 Project Management: Tools & Techniques

Risk Assessment
Ok, weve collected data What is next?

We need to crunch some numbers to determine which option is the best

Qualitative assessment

Risks are evaluated and planned for if they have significant likelihood and significant costs Risks are given a score for each development phrase The stage scores are totaled to calculate the risks score The risk likelihood is expressed in percentages The risk cost is expressed in weeks These two are multiplied to calculate an expected loss Similar to risk exposure, but percentages are also used for cost

Dynamic risk assessment

Risk exposure assessment

Risk impact assessment

Qualitative Risk Assessment

Low Low Medium High

Medium

High

Ignore Ignore

Ignore Consider

Consider Take action

Consider Take action Take action

Dynamic Risk Assessment

A Requirements Design Implementation Testing Integration Score

Score

1 2 3 3 2 11

3 2 1 1 1 8

1 2 2 4 4 13

5 6 6 8 7 32

Risk Exposure Assessment

Probability of Loss Technology Failure Key Developer Attrition etc

Cost

Risk Exposure

5%
40%

20 weeks 1 week
10 weeks 4 weeks

Risk Impact Assessment

Probability of Loss Technology Failure Key Developer Attrition etc

Impact

Risk Exposure

0.05
0.40

0.60
0.40

0.03
0.16

Evaluating Alternatives
When considering alternatives, you can apply the same techniques to all alternatives e.g. J2EE vs. ASP.NET vs. RoR

Probability of Failure J2EE ASP.NET RoR

Cost

Risk Exposure

0.05 0.05 0.05

1 week 5 weeks 3 weeks

0.05 weeks 0.25 weeks 0.15 weeks