9/14/2009 Dr.

Dobb's | Is Fixed-Price Software De…

Is Fixed-Price Software Development Unethical?

Reducing risks, better serving customers
By Scott W. Ambler, Dr. Dobb's Journal
Jul 18, 2008
URL:http://www.ddj.com/architect/209101238

Scott is Practice Leader Agile Development for IBM Rational.

One of the riskiest decisions you can make in software development is to require a "precise" cost and schedule
estimate at the beginning of the project. Although there is nothing wrong with fixing just the price, as I wrote in
Agile on a Fixed Budget, the situation quickly becomes dysfunctional when you also decide to fix the schedule
and the project scope. Although customers often demand to work in this manner, particularly when the system is
being built by another organization such as a system integrator (SI), as professionals we must question the ethics
of fixed-price IT projects. We know fixed price is a bad idea, our customers inherently know fixed price is a
bad idea, and it's high time that as an industry we choose to abandon this highly questionable approach.

I want to start by defining three critical terms:

Fixed price refers to a project where the cost, schedule, and scope are set early in the lifecycle. "Fixed-
iron triangle project" would also be a valid term, albeit a mouthful. This is different from a "fixed-budget
project" where the budget is set, but at least one (if not both) of the scope or schedule are allowed to
vary.
Customer refers to the person(s) or organization(s) for which an IT solution is being developed (or
purchased, as the case may be).
IT provider refers to the person(s) or organization(s) who performs the work to provide the solution to
the customer. The IT provider may be internal to the customer (for example, the customer is a bank and
the IT provider is the IT department within that bank) or external to it (for example, the bank chooses to
hire an SI to build a system.)

I also need to set the context for this discussion. First, my argument is against fixed-price projects, not fixed-
budget projects. The constraints imposed by fixed-price approaches reduce the flexibility of both the customer
and the IT provider and thereby increase project risk unacceptably. Second, for simplicity I assume that you
work for an IT provider even though some readers are business stakeholders.

Why Fixed Price?

There are many reasons why customers want to take a fixed-price approach to IT projects:

To reduce financial risk. Customers believe that by getting you to agree to a set price, they are capping
their potential financial outlay, effectively offloading the financial risk onto you.
To choose projects with the highest ROI. Customers have many opportunities to invest their money, your
project is just one of them, and therefore they want to choose the best available options. They assume
that by requiring all project teams to produce a common set of cost and benefit estimates, it enables them
to compare "apples with apples".
To force you to prepare a viable plan. By forcing you to give an accurate estimate that you'll be held to,
you are effectively motivated to think through how you're going to perform the work.
ddj.com/article/printableArticle.jhtml?a… 1/4
9/14/2009 Dr. Dobb's | Is Fixed-Price Software De…
To choose between several IT providers. The customer wants the best deal possible so they put out a
request for proposal (RFP) to several IT providers who must then bid for the work. This gives them
options to choose from and motivates the IT providers to reduce their margins in the face of competition.
To support annual budgeting. IT outlays are a critical component any organization's budget, typically 2"15
percent of the overall budget, so the customer wants to know how much money to allocate to the project
in the coming year(s) for the project.
To "hide" IT costs. Politically it can be easier to admit that a project costs $500,000 to implement than
the fact that they're paying you $400 an hour to do so.
To increase their comfort level with potentially large projects. Few people are comfortable with initiating a
three-year, $25 million project. Then again, without a "good" estimate, perhaps it's really a seven-year
$85 million project or a five-year $210 million project.
To conform to what IT tells them to do. For several decades traditional software engineering theory has
told us to do formal, up-front cost estimates. In turn, we have taught our customers that this is the way
things are done.
To conform to what the organization tells them to do. For example, many government organizations have
a defined request for proposal (RFP) process that insists on a fixed-price approach. Neither you nor your
customer may want to work this way, but it is incredibly difficult for them to forgo a fixed-price approach.

These desires are all reasonable, at least on the surface, but unfortunately aren't realistic in practice. The reality
of IT projects doesn't support the theory behind fixed-price approaches.

Some Inconvenient Truths

Formal cost estimation strategies—COCOMO I/II, Function Points, Feature Points, and SLIM, to name a few
—are based on the same fundamental strategy. First you identify the scope of the system and a potential
architectural solution that addresses that scope. The greater the detail of your specification efforts, the greater
your ability to accurately estimate the effort required. Second, you glean critical sizing measures from these
specifications. Third, you take measures from historical project data, ideally from your own organization. Fourth,
you put all of these measures into a "magic formula," which gives you your estimate. The measures and formula
vary between estimation techniques, but the high-level process remains the same. As a result, these formal cost
estimation strategies all suffer from a common set of challenges:

We don't seem to be good at estimation in practice. The Standish Group's "Chaos Report"
(www.standishgroup.com) found that only 31 percent of IT projects are reasonably on time and on
budget and that on average projects cost 189 percent of what was originally estimated. These results
belie the beliefs that fixed-price projects put a maximum on the financial outlay and that it's possible to
reasonably compare projects based on their initial estimates. This makes it difficult to identify the potential
ROI of an IT project, to compare the responses to an RFP, or to perform reasonably accurate annual
budgeting.
We don't seem to be very good at estimation in theory either. In 1987, Chris Kemerer reported in "An
Empirical Validation of Software Cost Estimation Models" (Communications of the ACM, May 1987)
that formal estimation techniques had error rates of between 85"772 percent when applied in an
uncalibrated manner. So, if you don't have historic data specific to your organization, you're likely in
serious trouble. However, Kemerer found that with significant calibration for your environment, often
increasing your overall planning costs past the point of diminishing returns, estimates became 88 percent
accurate. The bad news is that this study was done during the 1980s, a period where we had significant
experience with the dominant technologies of the time (COBOL, C,). This paper is just an exemplar;
there are dozens of others that show that formal estimation strategies do not work as well in practice as
some of their protagonists may claim.
There isn't a very good connection between the theories being espoused and actual practice. Magne
Jxrgensen and Martin Shepperd reported in "A Systematic Review of Software Development Cost
Estimation Studies" (IEEE Transactions on Software Engineering, January 2007) that there still needs
to be significant research into the practical application of software estimation techniques, particularly in
real-world settings as opposed to arbitrary data sets. Although they included several hundred references
to estimation papers, they couldn't find a single study that examined the application of formal software
cost-estimation techniques in a real-world setting. There were many that used actual historical data, but
no study explored how the estimation strategies were applied in practice. In short, there appears to be a
lot of blind faith in the veracity of up-front estimation techniques.
ddj.com/article/printableArticle.jhtml?a… 2/4
9/14/2009 Dr. Dobb's | Is Fixed-Price Software De…
Everyone isn't created equal. As early as 1968, Sackman, Erikson, and Grant reported a 20-to-1
productivity ratio between the best and worst programmers ("Exploratory Experimental Studies
Comparing Online and Offline Programming Performance," Communications of the ACM, January
1968). Since then, flaws in their methodology have been pointed out, and arguably a ratio of 10-to-1 is
more realistic. More recently, Barry Boehm reported a 10-to-1 ratio in Software Cost Estimation with
Cocomo II (Addison-Wesley 2000). The point is that unless you know exactly who will be doing the
work, or at least have a way to guarantee the category of people who will work on the project, then your
historic data proves questionable.
Customers know we can't estimate up front effectively. Dr. Dobb's 2007 Project Success Survey (see
Defining Success found that 87 percent of business stakeholders are clearly more interested in achieving
good ROI than coming in under budget. They might be asking us for fixed-price projects, but that's not
really what they want—begging the question why we're even on this fixed-price treadmill at all.
Historical data isn't accurate. Formal estimation techniques rely on historical data, and even when that
data exists it is rarely accurate. Having a decade or more of historical data is of little value when the
technologies that you're working with change every couple of years, when the composition of your teams
change, when the pool of people from which you draw from change, and when the skillsets of your
people change (hopefully for the better). Part of the foundation on which your estimate is based is
constantly shifting, reducing your ability to be precise.
Up-front estimation motivates Big Requirements Up Front (BRUF), which in turn results in significant
waste. To reduce the risk of uncertainty in your estimates, you need to increase the level of information
that goes into those estimates. This motivates you to do detailed modeling of your requirements and
potentially even your architecture. This unfortunately ignores the facts that people aren't very good at
defining up front what they want and even if they were, the requirements will evolve anyway. The
implication is that the sizing measures that are input into your estimate are also on shifting ground. The
"Chaos Report" shows that projects that took a BRUF approach ended up delivering systems (when they
delivered anything at all) where 45 percent of the functionality was never used by stakeholders. So, by
insisting on a fixed-price strategy, often to minimize financial risk, customers ensure that on average close
to half of their development budget is wasted. (I've written a detailed article about this phenomena.) Yes,
up-front estimating may motivate IT providers to create detailed plans, but apparently they aren't very
good in practice.
Being held to an initial estimate motives IT providers to adopt strict change management, which in turn
prevents the customer from getting what they actually want. The only real option to protect yourself from
the risks of fixed-price IT projects is to adopt a traditional change management strategy where you make
it difficult for the customer to modify their requirements. These processes are usually onerous, make
potential schedule slippage explicit (a good thing), and typically make the customer pay handsomely for
any changes. These factors reduce customer motivation to change their defined requirements, even though
their requirements have in fact changed, effectively turning the change management strategy into a change
prevention strategy. Not only do traditional change prevention strategies exacerbate the challenges
associated with BRUF, they are ethically questionable because you're effectively motivating customers to
pay for functionality they really don't want. You still need a strategy for change management, but it doesn't
need to be the bastardized processes that we see in the traditional world. Agile change management
strategies treat requirements like a prioritized stack that is managed by the stakeholders, and the
development team merely pulls work from the top of the stack each iteration, thereby producing the
greatest ROI as defined by the stakeholders. This lets stakeholders manage the scope, thereby getting
software that actually meets their needs with the trade-off that they are now accountable for these scope
changes.
You waste time and effort on overly precise up-front estimating. We need to question whether the
investment being made in up-front estimating to increase its precision is worth it. It's likely a good
investment of $10,000 to determine that your project will likely come in between $2.3 and $3.5 million,
but it likely isn't worth an additional $100,000 to determine that it will be between $2.85 and $3.1 million.
Customers often don't realize the amount of effort required to obtain just a bit more precision in our
estimates.

Is Fixed-Price IT Ethical?
There are some very clear and obvious challenges with fixed-price IT projects. I believe that this is the direct
result of the inherent nature of IT projects. Traditionally we've wanted to believe in the concept that "software
engineering" is 80 percent science and 20 percent art, but in practice development has proven to be closer to 20
ddj.com/article/printableArticle.jhtml?a… 3/4
9/14/2009 Dr. Dobb's | Is Fixed-Price Software De…
percent science and 80 percent art. Or perhaps the 20 percent of software engineering that is art is simply 16
times harder than the 80 percent that is science. Regardless, after four decades of struggling with fixed-price IT
projects, building a buffer into your estimate is merely a band-aid that reduces some of your risk. It's time that
we start questioning the ethics of approach.

For a fixed-price IT project to be considered ethical, three criteria must be met. First, the customer must
explicitly insist on a fixed-price approach. Second, the customer has been educated as to the inherent risks
associated with fixed-price IT projects (feel free to give them a copy of this column). Third, the customer has
been given other choices, such as staged-gate funding common on large projects or continuous-flow funding
common with agile projects, has had the trade-offs explained to them, and has still explicitly chosen a fixed-price
approach. As professionals, we must actively question the ethics of what we do, even when we've been doing
these things for decades. We know that we have significant challenges around formal approaches to estimation
and to traditional project management in general. Too many IT professionals give up when faced with customer
demands for fixed-price IT projects, thereby choosing to work in a manner that dramatically increases project
risk. We have to stop accepting the status quo and push back against our customers and help them to rethink
this strategy.

We fundamentally know that fixed-price IT projects are a very poor way of working. Luckily, so do our
customers. Whenever a customer insists on a fixed-priced IT project, even within the scope of an RFP that's
been put out to bid, ethically we must attempt to dissuade them from this perilous path. Unless we start
providing a consistent front against fixed-price projects, we will never get off this treadmill that we find ourselves
on. We must actively choose to reduce the inherent risks in our industry, and the desire by customers for fixed-
price IT projects is likely the greatest one that we face.

DDJ

Copyright © 2009 United Business Media LLC

ddj.com/article/printableArticle.jhtml?a… 4/4