MIPRO 2014, 26-30 May 2014, Opatija, Croatia

Modeling of software failure cost

in ERP systems
Frane Urem *, Kreimir Fertalj

**,

Ivan Livaja *

**

University od applied science ibenik/Department of managemet, ibenik, Croatia

Faculty of Electrical Engineering and Computing/Department of Applied Computing, Zagreb, Croatia
e-mail: frane.urem@compco.hr, kresimir.fertalj@fer.hr, ivan.livaja@vus.hr

Abstract - Enterprise resource planning (ERP) systems are

the most important software for many types of businesses.
Software defects inserted on working ERP system can cause
significant cost to their users. Existing software reliability
models usually don't consider any financial impact at all. A
model proposed in this paper links the ERP application
layer with critical business operations and proposes cost
calculation. The described modeling also encourages users
to easily quantify the consequences of software failures that
will always appear in ERP system. The proposed methods
are illustrated by a case study.

I.

II.

DEFINITIONS

Standards as IEEE 1044 [1] are well-defined in

classification of software anomalies such as problem in
software, failure, defect and fault. These definitions are
used in this paper and their interrelation is illustrated in
Figure 1.
Problem

Problem may be
caused by failure

INTRODUCTION

In certain way ERP vendors and their customers have

different perspective of software failure costs. It is
relatively easy to determine failure costs to be borne by
the ERP vendors during failure fixes. They will
investigate reported problems, identify and remove
software defects that have caused failures. That process
defines the main costs for ERP vendor because it includes
certain experts costs (developers, business domain
experts, consultants etc.) for solution search and rework
spent in defect removing from the current software
release. Most of these costs ERP vendor can predict and
charge users before (installation, configuration, training)
or after (maintenance contract) an ERP implementation.
Of course, the ERP vendor may suffer certain indirect
costs connected with user dissatisfaction, legal contracts
etc. but that is hard to capture and it is out of scope of
software reliability modeling anyway.

Failure

Failure may be
caused by defect
Defect

Figure 1 Relationship Problem Model Defect as an ER diagram

According to Figure 1, in every installed ERP system

user can suffer various problems during the execution of
different business operations. Problems may be caused in
ERP system by software failure but it is not a rule because
sometimes the cause may be something else such as
insufficient training of users to work. According to
definition from [1] failure is termination of the ability of
an ERP system to perform required function or to perform
it in previously specified limits. Defect is a source of
software failures but sometimes user will not experience
failures because defect is built in some functionality that is
not actually used on existing ERP installation. In the past
(as an example earlier version of IEEE 1044 standard
published in 1993.) it was presumed that defect is a fault
(error in software) but in complex software systems like
ERP, defect term is more appropriate because defect can
be something else like wrong configuration of ERP
module. Software defect was initially defined by IBM [2],
[3] but in IEEE 1044 standard defect attributes are more
fully described.

User's perspective is different and more complex to

define because every failure is linked with business and
failure cost is dependent on affected business process.
Existing business that is closely related to the installed
ERP system can be seriously compromised in case of
untimely failure fixes.
Therefore, in this paper proposed model links the ERP
application layer with critical business operations and
propose cost calculation from users perspective.
Described model is directly linked with ERP software
failure cost for the customer but proposed procedures can
be also useful for ERP vendors in order to focus on the
customer's needs and know true risks of using their
product.

557

III.

Severity can be simply correlated with expected

frequency of failure. IEEE classification for defect
severity [1] can be used as:

MODEL

An adapted model from [4] is presented in Figure 2

and can be used to determine various ERP failure cost
information. Initial model linked business layer to
hardware/software system (called IT layer) deployed in a
plant. Relevant business processes cost is linked with
software function call with specific failure probability.
Software reliability prediction is combined with financial
calculations and software failure cost prediction is
determined as a result.
In a Figure 2, original model is simplified because
failure probability of ERP function call is not considered.
Critical business operation is mapped with ERP function
through software failure severity (caused by defect
inserted in certain ERP function) which is later displayed
in Table 1.
System is observed from user's perspective on two
layers: business and ERP and some business process steps
are assigned to corresponding functions implemented in
ERP system. In case of success next optimal business
process B will be executed. In case of failure another
business process C will be executed with additional costs.

Blocking (impossible to work on certain ERP

module)
Critical (certain critical business operation is
disrupted)
Major (certain critical business operation is
affected but can proceed)
Minor
disrupted)

Business process
step A

operations

are

ERP module failure costs can be calculated according

to (1) as:
CT= (ALC+DC) x RT x N

Business process
step C

Success

business

Inconsequential (no significant impact on business

operations)

Business layer
Failure

(nonessential

(1)

CT

total failure cost

ALC failure

additional labor costs caused by

DC

downtime induced costs

RT

resolution time

the number of affected workplaces

Every failure is affecting a particular business

operation and business domain experts define the financial
costs according to referenced failure severity.

Business process
step B

As a result, Table 1 can be made for every ERP

module.

ERP layer
ERP module x

TABLE I. USER ASSESSMENT OF POSSIBLE COSTS FOR SPECIFIC

ERP MODULE FAILURES WITH CERTAIN SEVERITY
Function
call x

COST DESCRIPTION:
MODULE:
Failure

Success

FAILURE
SEVERITY
Blocking

Defect tracking system must be implemented as

important prerequisite to apply presented model. Every
failure is reported on defect tracking system with
attributes detailed described at [1]. For failure cost
analysis the following key attributes for reported failure
must be available:
Date/time when failure is observed

Date/time when final disposition is assigned

Defect reference

Severity (the highest failure impact)

Effect (the class of requirement that is impacted)

Insertion activity (the activity during witch defect

was inserted )

Operation 1 Operation 2

...

Operation N

Critical
Major
Minor
Inconsequential

Resolution time is calculating from failure report as

the difference between dates when failure is observed and
final disposition is assigned. Insertion activity can be used
as a parameter for the case when user wants to assess the
total cost in situation when failure is observed and
corrective actions are in progress. For these cases Table 2
can be used. By contractual relations between the ERP
vendor and customer it is usually possible to define and
predict the response time when failures will occur during
coding, configuration and documentation insertion
activities. It is difficult to predict the time for corrective
actions needed for removing of defects that were inserted

Referenced defects should be described with key

attributes such as:

ERP module name

CRITICAL BUSINESS OPERATIONS

Figure 2 Connection between business layer and ERP system

lost labor hours, downtime induced costs

558

during requirements or design phase. Quite certainly the

occurrence of such defects will seriously jeopardize

business operations because their removal requires much

more time.

TABLE II. USER ASSESSMENT OF POSSIBLE COSTS FOR SPECIFIC ERP MODULE FAILURES WITH CERTAIN SEVERITY
MODULE:

ERP module name

DEFECT INSERTION ACTIVITY

VALUE

Definition (with examples)

Expected time for corrective action

Requirements

Defect is inserted during requirements

definition (incomplete use case specification,
incorrect performance specification, incorrect
security specification, incorrectly specified
function etc.)

difficult to predict- business is at risk

Design

Defect is inserted during design activities

(incorrect interface specification, incapable
design etc.)

difficult to predict- business is at risk

Coding

Defect is inserted during coding or similar

activities (incorrect data initialization on
contractually defined- controlled
interface form, module interface does not work losses
correctly, wrong reporting etc.)

Configuration

Defect is inserted during ERP module

configuration (wrong interface to other ERP
modules, incorrect interface to other software
systems etc.)

contractually defined- controlled

losses

Documentation

Defect is inserted in documentation for

installation and operation

contractually defined- controlled

losses

IV.

better reporting integrated in a new interface

CASE STUDY

Nothing has changed on existing technology (.NET,

Microsoft SQL Server) or equipment (only new touch
screens were installed). The specified user was one of the
first who decided to replace the old POS module but the
new one has already been shipped to several smaller sites
and there were no reported problems.
After installation the user has reported failures
described in Table 3. All failures were categorized by
ERP vendor and reported on vendors helpdesk system.

The following case study illustrates proposed model

for software failure cost on already implemented ERP
system. One ERP module upgrade is analyzed on existing
ERP system made by one Croatian ERP vendor. The user
wanted to replace existing POS (Point Of Sale) module
with the new one from the same vendor. Basically, it was
an existing module upgrade with the following
improvements:
new user interface design optimized for use on the
touch screen
more detailed user roles

TABLE III. REPORTED FAILURES

ID

1
2

Title
Fiscalization service is
working slower than in old
module
Discount based on user role
does not work correctly
Cash register report does not
print cash deposit amount
correctly on POS printer (on
screen report is working
correctly)

Severity

Date observed Date closed

Business
operation

Defect
Referenced defect insertion
activity

Major

2.9.2013

4.9.2013

Sales

G2-TSPRM-006

Coding

Minor

2.9.2013

4.9.2013

Sales

G2-TSPRM-007

Coding

Inconsequential

5.9.2013

9.9.2013

Sales

G2-TSPRM-008

Coding

After upgrade bar-code reader

Major
does not work correctly for
Coca Cola products

9.9.2013

10.9.2013

Sales

G2-TSPRM-009

Coding

After upgrade document

replication from central
Critical
location to POS does not work

10.9.2013

11.9.2013

Replication

G2-TSPRM-010

Coding

559

The user did not have serious problems with the observed
module after the failures described in Table 3. According
to Table 3 all defects were inserted during coding and
every defect caused single failure. There were lot of
problems caused by some reported failures but it was
impossible to report all problems in a real time. As a
result, it was impossible to objectively assess the costs
incurred.
After an interview with the user, Table 4 and Table 5
were formed according to Table 1 and calibrated with
available information about real costs that are incurred
during failures from Table 3. Thus some costs were
linked to lost labor hours according to the records of
overtime hours caused by reported failures and some
downtime costs were calculated as reduced earnings
compared to regular working days. Some type of failures
(like failures with blocking severity) was not reported and
costs were estimated with worst case scenario. At the end,
the resulting Tables 4 and 5 represent a projection of
future costs that may be expected when certain types of
failures will occur.

TABLE V. USER ASSESSMENT OF DOWNTIME INDUCED COSTS FOR

SPECIFIC ERP MODULE FAILURES WITH CERTAIN SEVERITY

COST
DESCRIPTION:

downtime
induced costs at
one workplace,
during one
working day
(EUR)

MODULE:

Point of sale

CRITICAL BUSINESS OPERATIONS

FAILURE
SEVERITY

COST
DESCRIPTION:

MODULE:

lost labor hours

at one
workplace,
during one
working day
Point of sale

1.000,00

250,00

125,00

250,00

Critical

1.000,00

250,00

125,00

250,00

Major

250,00

125,00

0,00

125,00

Minor

50,00

50,00

0,00

50,00

0,00

0,00

0,00

0,00

CT = (2x7 EUR+250 EUR) x 1 x 10 = 2.640,00 EUR

The total costs of all reported failures from Table 3 are
calculated in Table 6.

Replication
(new
Version Loyalty
Sales/Returns products,
upgrade programs
prices, cash
register ...)

Blocking

16

Critical

16

Major

Minor

0,5

0,5

0,5

Inconsequential

Loyalty
programs

For an example, expected failure cost with major

severity that affects sales operations for one day is
calculating from (1) and Tables 4 and 5 (in this calculation
salesperson work time corresponds to 7 EUR per hour )
as:

CRITICAL BUSINESS OPERATIONS

FAILURE
SEVERITY

Version
upgrade

Blocking

Inconsequential
TABLE IV. USER ASSESSMENT OF LOST LABOR HOURS FOR
SPECIFIC ERP MODULE FAILURES WITH CERTAIN SEVERITY

Sales/Returns

Replication
(new
products,
prices, cash
register ...)

TABLE VI. TOTAL COST OF ALL REPORTED FAILURES FROM

TABLE 3
Failure ID
1
2
3
4
5
TOTAL

Cost
2.640,00
505,00
0,00
2.640,00
2.640,00
8.425,00

Total estimated costs were amounted to more than

30% of planned upgrade investment. However, such costs
are still within the limits expected. A newer POS version
has introduced new features and added values so total
investment was paid back soon after installation.
Response time from ERP vendor in situations when
failures are occurred was under contract limits so there
was no reason to charge vendor for failure costs. In all
ERP implementations or bigger upgrades failure cost can
be expected but also prevented with stronger system
testing. In described case study all defects were inserted
during coding activities so failure costs were acceptable
because repairs were made relatively fast.
It is also interesting to analyze worst case scenarios
when blocking failures can be occurred during upgrade

560

according to Tables 4 and 5. For an example, the failure

cost for one blocking failure in described ERP installation
is more than 10.000,00 EUR per day. It seems impossible
that issue will happen but failure number 4 from Table 3
occurred only because during system testing bar code
function was not tested with Coca Cola products (QA
tester used different products from other suppliers). In
described case study some defects have simply resulted in
failures although it has not happened to other users
because they use the same ERP module in a different way.
Therefore it is very important for user to provide a
detailed system test on his location before any kind of
ERP upgrade. This is particularly important when it comes
to ERP module that can directly affect the business as in
described case study.

through the user assessment. In this way, ERP users can

compare expected failure costs and investments required
for their prevention. The initial failure cost estimation can
be updated with the new failure records and the associated
costs. The proposed model will be extended in future
research with failure cost prediction according to previous
failure occurrences.
REFERENCES
[1]
[2]

[3]

V.

CONCLUSION

Described modeling can be useful as a simple

assessment of software failure costs that could occur in
installed ERP system. In this model the critical business
processes are linked to the severity of observed failures

561

[4]

IEEE Std 1044-2009, 2009, IEEE Standard Classification for

Software Anomalies
R. Chillarege, I. S. Bhandari, J. K. Chaar, M. J. Halliday, D.
S.Moebus, B. K. Ray, and M.-Y. Wong, Orthogonal Defect
Classification: A Concept for In-Process Measurements, IEEE
Transactions on Software Engineering 18, No. 11, 943956, 1992.
K. A. Bassin, T. Kratschmer and P. Santhanam, "An Objective
Approach to Evaluate Software Development", IEEE Software
Magazine, November/December, Vol.15 No.6,, 2002.
F. Brosch, R. Gitzel, H. Koziolek Combining "Architecture-based
Software Reliability Predictions with Financial Impact
Calculations", Electronic Notes in Theoretical Computer Science
264 317, 2010.