European Journal of Purchasing & Supply Management 8 (2002) 123134

Purchasing performance evaluation: with data envelopment analysis

Liane Eastona,*, David J. Murphyb, John N. Pearsonc
a
School of Business Administration, Penn State Harrisburg, 777 W. Harrisburg Pike, Middletown, PA 17057-4898, USA
Air Force Institute of Technology, Graduate School of Logistics and Acquisition Management, Wright-Patterson AFB, OH 45433-7765 USA
c
Department of Business Administration, College of Business, Arizona State University, Campus Box 874706, Tempe, AZ 85287-4706, USA

Received 26 June 1997; received in revised form 20 August 2001; accepted 8 December 2001

Abstract
Because of supply chain management and other factors, purchasings performance is considered an important element of
corporate performance. Nonetheless, the measurement of purchasing performance, and comparing that performance to other
purchasing departments has proven to be very difcult. These difculties stem from the lack of valid measurement criteria and
adequate methodologies to aggregate individual performance measures into a single index of overall performance. Many
methodologies are unable to account for the relative importance of performance measures, which varies among rms.
This paper examines the application of Data Envelopment Analysis (DEA), which has demonstrated potential as a management
tool to overcome the shortcomings of other techniques and to help purchasing executives improve the efciency of their operations.
A DEA model was developed to compare the purchasing efciency of rms in the petroleum industry. The model introduces
one more method by which managers can obtain information to assist in the decision making process. r 2002 Elsevier Science Ltd.
All rights reserved.
Keywords: DEA; Efciency; Linear programming; Performance evaluation; Purchasing

1. Introduction
In the late 1980s and the early 1990s, purchasing
started to gain increased recognition due to its direct
impact on the nal cost of the end product. Purchased
components constituted over 55% of the sales dollar in
many manufacturing industries, which left purchasing
with the signicant potential to reduce costs for the
company.
Technological advances and economic pressures
caused purchasings inuence to grow. Advanced
technologies reduced labor and operating cost, which
increased the percentage of purchased cost in every sales
dollar. Economic pressures both domestically and
abroad forced companies to reduce costs, allowing
management to realize the important role purchasing
can play in any competitive environment.
In more recent years, the concentration on core
competencies coupled with business practices that focus
on the supply chain, have afforded purchasing the
*Corresponding author. Tel.: +1-717-948-6161; fax: +1-717-9486456.
E-mail address: ple10@psu.edu (L. Easton).

opportunity to play a larger strategic role in the rm

(Carr and Smeltzer, 1999; Narasimhan et al., 2001). This
focus on core competencies and supply chain management signicantly increases the number and importance
of external relationships. Firms that have recognized the
strategic importance of purchasing, now expect this
function to develop and maintain effective and efcient
relationships with suppliers. Purchasing is now looked
upon to lead the way towards analyzing the external
market, developing the appropriate relationships, and
negotiating contracts that will increase the protability
of the rm and the supply chain (Cox, 1996).
Purchasings role in supply chain management includes
communicating with suppliers in an effort to decrease
redundancies and increase efciencies along parts of the
supply chain (Wisner and Tan, 2000). Because of
purchasings strategic impact, the rationale for measuring its performance becomes evident. Unfortunately, the
purchasing department is still one of the more difcult
functional areas to evaluate (Chao, 1989; Hendrick and
Ruch, 1988; Malec et al., 1991; Millen, 1990; Murphy,
1992; Stanley, 1992; Van Weele, 1984). One approach to
the evaluation of the purchasing department is to
compare that departments performance with the

0969-7012/02/$ - see front matter r 2002 Elsevier Science Ltd. All rights reserved.
PII: S 0 9 6 9 - 7 0 1 2 ( 0 2 ) 0 0 0 0 2 - 3

124

L. Easton et al. / European Journal of Purchasing & Supply Management 8 (2002) 123134

performance of other rms in the industry. However,

there are two major problems with comparisons to other
purchasing departments. First, purchasing executives in
the past could not compare departments because
aggregated performance data was hard to obtain, and
even if it was attainable, methods that would provide an
effective comparison of overall performance did not
exist. Second, many metrics in purchasing ignore the
efciency or productivity aspect of performance.1
Purchasing measures such as a suppliers quality ignore
the amount of inputs (e.g. purchasings budget) that had
to be put forth in order to achieve that certain level of
output. Comparing one department to another on just
the outputs is an incomplete appraisal.
Data Envelopment Analysis (DEA) is a linear
programming model developed by Charnes et al.
(1978) which appears to have the characteristics that
will enable development of an improved purchasing
performance evaluation system. The primary advantage
of DEA is that it will establish a composite index of
overall performance, allowing for easy comparison
among departments or rms. DEA also provides
information on the individual performance measurements that made up the aggregate score. This disaggregated information enables the manager to take action
on specic measures. Moreover, DEA provides a
measure of efciency (output/inputs), so that both inputs
and outputs are accounted for, which makes for a more
valid comparison between purchasing departments.
This study materialized, in part, based upon an
initiative that took place at the Center for Advanced
Purchasing Studies (CAPS). CAPS had undertaken a
benchmarking purchasing project and had gathered the
required data from the leading rms in selected industry
groups. Senior purchasing executives met as an ad hoc
committee in order to determine what specic data
should be collected as a basis for benchmarking
purchasing performance in that industry (NAPM,
1991). Each of the different industries, which were to
be benchmarked, established its own measurement
criteria. CAPS then collected the aggregate data, and
rms were able to compare performance against a
standard for each purchasing performance measure.
Even though the benchmarks were in a ratio analysis
format, the measurement evaluation was incomplete. A
rm could not establish where it stood relative to the
other rms, but only where it stood with respect to a
composite standard (typically an average) for each
individual purchasing performance measure. In addition, there was no aggregate measure of overall
purchasing performance from which a rm could
compare performance with other industry members.
1

Productivity is the measure of efciency, which is the ratio of actual

output to actual inputs. For the purposes of this paper, efciency and
productivity will be used interchangeably.

The purpose of this research is to utilize DEA to

develop an evaluation model that overcomes some of
the shortcomings of current techniques, and to explore
the potential of DEA as a management tool to help
purchasing executives improve the efciency of their
operations. There are two research objectives associated
with this paper: (1) to design a DEA evaluation system
which improves on current purchasing evaluation
systems that compare performance among purchasing
departments, (2) to establish where DEA ts within a
framework of performance measurement.

2. Data envelopment analysis

DEA is a linear programming-based technique that
converts multiple input and output measures into a
single comprehensive measure of productivity efciency
(Epstein and Henderson, 1989). DEA provides a
measure by which one rm or department can compare
its performance, in relative terms, to other homogeneous
rms or departments. DEA is mainly utilized under two
different circumstances. First, it can be used when a
department from one rm wants to compare its level of
efciency performance against that of a corresponding
department in other rms. However, one major assumption is that all departments have similar strategic goals
and directions (Metters et al., 1999), which includes the
need for all the rms to be within the same industry.
Second, DEA can be used in a longitudinal nature by
comparing the efciency of a department or rm over
time.
In measuring the relative efciencies of organizations,
the DEA measurement can be dened as the ratio of
total weighted output to total weighted input. With
DEA each organization can utilize different weights for
the set of performance measures. Weights are selected
that will maximize the composite efciency score for
each functional unit. This allows each unit to take
advantage of their own unique areas of specialization
(Sexton, 1986).
This variable weighting allows for the evaluation of
performance while taking into account differences in
goals, responsibilities, and type of procurement.2 The
range of possible weights is controlled by requiring all
weights to be positive, and specifying that if another unit
used the same weight, their total efciency score could
not exceed the value of one. This ensures that all rms
are evaluated on the basis of relative efciency. The
technique also gives information as to the specic effect
each input or output has on overall efciency.
2
If there are major strategic differences in goals between organizations or departments (groups) then these groups should not be
compared directly under DEA, because DEA should only be used to
compare groups with similar overall strategies.

L. Easton et al. / European Journal of Purchasing & Supply Management 8 (2002) 123134

The information supplied by DEA can provide a

major advantage over benchmarking and other techniques where only one measure can be evaluated at a
time, gaining no insight into overall efciency. The single
composite measure of DEA allows for the rank ordering
of all the rms in terms of their overall performance.
Originally, DEA was developed for the non-prot
arena. DEA has been used in situations such as the
identication and measurement of hospital inefciencies
(Sherman, 1986); the assessment of national parks
efciencies (Rhodes, 1986); and the comparison of
results of DEA techniques with accounting approaches
normally used to evaluate efciencies in non-prot
institutions (Ahn et al., 1989). The results of these
studies are summarized in Table 1.
DEA has also been utilized in the private/for-prot
sector. More specically, DEA has been utilized in

125

supply chain management research. Weber and Desai

(1996) employed it to construct an index of relative
supplier performance. Clarke and Gourdin (1991)
applied DEA to the vehicle maintenance activities of
17 separate maintenance shops of a large-scale, nonprot logistics system. Two of the studies demonstrate
how DEA can be used in longitudinal studies to
determine the progress of one unit or department over
time. For example, Metzger (1993) used DEA to
conduct a longitudinal study to measure the effects
of appraisal and prevention costs on productivity.
Kleinsorge et al. (1991) utilized DEA to conduct a
longitudinal monitoring process of one carrier in an
effort to assess expected performance improvements
over time. More details on these studies can be seen in
Table 2. While these studies indicate that DEA can be
utilized in supply chain management research, no one,

Table 1
Sample of DEA studies and major results
Author

Application

Contribution

Sherman (1986)

Medicalsurgical areas of seven acute care

hospitals

1. Able to identify inefcient units not previously identied by

regression or single ratio analysis
2. Tradeoff analysis of inputs/outputs
3. Locate sources of inefciency

Rhodes (1986)

Eighty national parks and historical sites

1. Able to answer policy questions regarding emphasis of proper

goals
2. Tradeoffs of stafng decisions
3. Tradeoffs of resource allocation

Ahn et al. (1989)

Texas state university system

1. Able to account for multiple inputs and outputs

2. Demonstrate differences from single ratios and regression
3. Tradeoff ratios for alternative courses of action

Charnes et al. (1989)

Texas electric cooperatives

1. Compared DEA, ratios, and regression

2. Reduced 670 individual ratios to one index

Table 2
Sample of DEA studies in supply chain management
Author

Application

Contribution

Weber and Desai

(1996)

Comparison of six suppliers performances

for a Fortune 500 Co.

1. Able to identify inefcient suppliers for the purpose of

negotiation leverage
2. Presented how parallel coordinates can be used to determine
which aspects of suppliers performance needs improvement in
order to increase efciency

Kleinsorge et al.
(1992)

Longitudinal study of one carrier

1. Able to demonstrate the use of DEA as a longitudinal measure of

a carriers efciency and performance
2. Presented the use of intangible measures with DEA

Metzger (1993)

Longitudinal study of the effects of quality on

productivity

1. Able to demonstrate the use of DEA as a longitudinal measure of

quality initiatives on productivity
2. Utilized step-wise regression to insure that only signicant input
variables were used in DEA

Clarke and Gourdin

(1991)

Comparison of vehicle maintenance activities

of 17 maintenance shops

1. Able to identify inefcient vehicle maintenance facilities

2. Tested the perceived usefulness of DEA (53% of participants
found DEA a useful management tool)

126

L. Easton et al. / European Journal of Purchasing & Supply Management 8 (2002) 123134

as of yet, has demonstrated its use as a potential

evaluation tool for purchasing performance, which is
one of the objectives of this study.

3. Performance measurement
Performance measurement is dened as the process of
quantifying action, or more specically the process of
quantifying and analyzing effectiveness and efciency.
Effectiveness is dened as the extent to which goals are
accomplished and efciency is a measure of how well the
rms resources are utilized to achieve specic goals
(Neely et al., 1995; Mentzer and Konrd, 1991). This
study focuses only on the efciency aspect of performance measurement.
The need for performance measurement in purchasing
has long been recognized. Performance measures drive
the actions of managers; therefore, the correct metrics
are a critical element of a companys performance. Early
conceptual development of performance measurement
in purchasing focused on cost issues with two major
concepts emerging. The rst was concerned with the
proper utilization of purchasing personnel. The less
overhead it took to perform the purchasing function, the
better (Gushee and Boffey, 1928). The second dealt with
end product cost as the ultimate measure of purchasing
performance (Lewis, 1933). The more that purchasing
could lower end product cost the more successful the
department was. The traditional measures, which
mainly consisted of costs and prots, continued until
the 1980s (Ghalayini and Noble, 1996).
In the late 1980s and 1990s, traditional performance
measurements were generally considered outdated for
several reasons. These traditional measures have been
accused of contributing to dysfunctional behavior,
which in turn impairs the performance of the corporation in the long-term (Neely et al., 1997). Performance
measurement systems have been blamed of encouraging
managers to strive for short-term gains while sacricing
long-term prots, and of inuencing managers to
improve their departments performance to the detriment of another departments performance. More
specically, traditional measures have been criticized
because they are: (1) based too much on nancial
measures and not enough on operational measures such
as quality; (2) incomplete or unidimensional; (3)
contradictory to continuous improvement; (4)
based on outdated cost accounting systems; (5)
inexible; (6) lacking integration with other
departments and strategic focus; and (7) invalid (Neely
et al., 1997; Ghalayini and Noble, 1996; Caplice and
Shef, 1994).
While DEA will not overcome a poor measurement
system, it does provide the means to reduce or eliminate
some of the problems associated with the rst four

concerns listed above. In other words, DEA coupled

with performance measures that promote sound behavior can reduce many of the problems associated with
performance measurement systems. How DEA can help
to improve a performance measurement system in
supply chain management will be discussed next.
With DEA, both nancial measures and operational
measures can be used together to obtain the nal
composite index/aggregate efciency score as long as
normalized data, such as indices, ratios and percentages,
are not mixed with non-normalized data (Dyson et al.,
2001). Therefore, the use of nancial measures does not
necessarily preclude the use of operational measures and
vice versa. Additionally, Kleinsorge et al. (1991) state
that one of the major advantages of DEA is that
perceived measures can also be used as long as they
can be quantied. However, Dyson et al. (2001) does
caution that this quantied data should be in interval or
ratio form.
DEA can also help to eliminate the use of unidimensional measures that may promote dysfunction behavior, because DEA allows for the use of multiple inputs
and multiple outputs. These multiple inputs and outputs
allow managers to develop a more comprehensive
measurement system that promotes behavior that is
of long-term benet to the organization. DEA also
provides enough detail for managers to take action
because it provides an analysis of which individual
performance measures are lagging in comparison to
other departments.
DEA can help measure a companys level of
continuous improvement, because it can be used to
measure a companys progress over time. A DEA
measurement can be taken each month or each quarter
to assess whether or not productivity is improving.
However, great care must still be taken to establish
measures that motivate employees to improve as
opposed to measures which just set norms (Ghalayini
and Noble, 1997). One example of how DEA can be
used with continuous improvement is with the continual
measurement of the effects of an ABC cost accounting
system.
While DEA has little to do with cost accounting
methods directly, it can help to measure the progress
being made with an activity based costing (ABC)
system. ABC is a cost accounting method, and one of
its major objectives is to use a more accurate method of
assigning overhead cost to end products than traditional
cost accounting systems use. Traditional cost accounting
methods typically use direct labor to assign overhead to
products, which many consider to be an outdated
method (Cooper, 1988a, b; Jeans and Morrow, 1989).3
Instead, ABC looks for more direct links between
3
For a more detailed discussion on ABC, please see Cooper
(1988a, b).

L. Easton et al. / European Journal of Purchasing & Supply Management 8 (2002) 123134

overhead cost and certain activities.4 More importantly,

one of the greatest assets of ABC, as quoted from a
KPMG consultant, is that ABC forces managers to
think about how they can reduce the hidden cost of
transactions (Neely et al., 1995). If costs are reduced, the
productivity of the department is increased as long as
the output is not decreased. DEA, which measures
productivity, can be used to access the success of an
ABC initiative over time. Moreover, the input part of
the DEA equation is very similar to the sources of cost
(overhead cost) used in ABC. Consequently, in terms of
the input part of the equation very little if any extra data
is needed above what has already been obtained for
ABC.
DEA can also be used in conjunction with a total cost
of ownership (TCO) program to evaluate suppliers.
TCOs aim is to determine the true cost of conducting
business with a particular supplier for a specic good or
service (Ellram, 1994). TCO goes beyond price to
include cost associated with quality, delivery performance, and cycle-time. Many cost issues used in TCO,
such as quality and delivery are operational measures
and are not normally measured in cost form. However,
for TCO purposes, these measures should be converted
into cost. This is difcult for some companies to do and
this is where DEA can be useful. As mentioned
previously with DEA, nancial gures and operational
measures can be mixed. Measures, which are typically
not maintained in nancial form, do not have to be
converted. Therefore, if the purpose of TCO is to
compare suppliers who provide the same product or to
evaluate one supplier over a period of time to see if there
is improvement, then DEA may be a helpful tool to a
purchasing departments effort to conduct business with
the lowest total cost supplier.
In summation, DEA is a tool that can be used to
reduce some of the existing problems with traditional
performance measurement systems. Additionally, it can
also be utilized to measure the progress of many
purchasing and company initiatives such as ABC,
continuous improvement, or TCO. However, DEA is
only as good as the measures that are utilized. For
example, if short-term as opposed to long-term performance measurements are used then this will motivate
managers to be short-term oriented. DEA is limited to
comparing like units. Therefore, even if you have several
purchasing departments within the industry, but some
compete on low price while others compete on speed,
then these two groups should be analyzed separately.
4
For example, the salaries of purchasing personnel are considered
an overhead cost. This cost exists because the purchasing agents are
responsible for several activities to include the placement of purchase
orders. Hence, with ABC, the purchasing overhead would be assigned
to end product based on the number of POs placed (among other
activities) associated with the end product as opposed to the direct
labor that went into the end product.

127

Another shortfall of DEA is that its only a measure of

relative efciency. Consequently, a separate measurement system will have to be maintained for a department or rm to measure effectiveness.

4. Research design
A DEA evaluation model, based upon the Charnes
CooperRhodes (CCR) ratio model, was developed and
used to compare the performance of purchasing departments of major rms within the petroleum industry. The
DEA model is described in detail by several authors
(Ahn et al., 1989; Banker et al., 1989; Sexton, 1986) and
is shown in Appendix A. The model derives an efciency
frontier which provides an estimate of relative efciency
for each purchasing department in the industry set,
using input and output variables.
The primary data used in this study were the
same types used to perform the 1991 report entitled
Purchasing Performance Benchmarks For the US
Petroleum Industry (NAPM, 1991). The purchasing
departments in the industry represent a relatively
homogenous group of decision making units which
operate under the same broad set of conditions, and are
in agreement on the performance measures most
valuable for comparison among peer rms. Eighteen
of the most prominent rms participated.
These 18 rms provided the data required to develop
a DEA model for the evaluation of purchasing
performance. The 18 rms had a total of $454.63 billion
in 1991 sales (85% of industry total), for an average of
$25.26 billion per rm. Sales ranged from a low of $794
million to a high of $116.94 billion. Neither the identity
of the participating rms nor the specic data they
provided for this study will be revealed. The data were
used to calculate benchmarks and DEA ratings for the
participating rms. Benchmarks, data items, and the
data-collection instrument itself were validated through
a review by all participating rms.
A general model of the purchasing function in the
petroleum industry (Fig. 1) was used to develop the
DEA model. Eighteen potential measures (specic
measurement factors identied in the CAPS benchmark
ratios) were considered to represent the major inputs
and outputs of the system. Firms were asked to rank
these measures in order to aid in nal selection. Those
measures considered to be most important by the
purchasing executives in the study were included in the
nal DEA purchasing model.
4.1. Performance measures utilized in study
If purchasing is to play a strategic role in a
corporation, then its goals and measures must be
aligned with corporate strategic planning (Carr and

128

L. Easton et al. / European Journal of Purchasing & Supply Management 8 (2002) 123134
Available Resources

Major Outputs

Operating
Expense
(OPEXP)
Purchase Dollars
(PURDLRS)
Professionals
(PROF)
Purchasing
Process
Administration/
Clerical
(ADMIN)

Percent of
Company Dollars
(PCTTOT)

Active
Suppliers
(SUPPLIERS)

Fig. 1. General model of the petroleum industry purchasing process.

Smeltzer, 1997). This link to strategic planning is

accomplished mainly through the use of departmental
performance measures, which contribute to the corporate goals. Two major factors, cost and technology drive
the petroleum industrys corporate goals.
Because of the commodity structure, dismal rening
margins, and increasing competitive retail markets of
the petroleum industry, the low-cost producer has the
competitive advantage (Rhodes and Koen, 1996).
Hence, petroleum companies are seeking ways to
decrease operating costs.
Additionally, petroleum companies are trying to gain
advantage by developing key technologies that will
reduce the cost of exploration and production. These
companies are forming alliances and mergers to acquire
the skills and technology necessary to compete effectively (McWilliams, 1997).
Therefore, in accordance with the industrys goals to
reduce cost through decreased operating cost and
advanced technologies, purchasing in turn must also
focus its goals and measures toward reduced cost and
improved technologies.
The purchasing departments in the companies researched were responsible for maintenance repair and
operation (MRO) items and not the raw materials such
as oil. Many of the metrics that have been deemed
essential in the 1980s and 1990s do not necessarily
pertain to MRO items. For instance, while delivery
concerns and quality are important in MRO items, the
level of importance is not as great as it is for raw
material purchases. MRO items, besides capital equipment, cannot directly affect the quality of the product in
most cases because they do not become part of the end
product. Delivery performance and quality are often not

tracked on MRO items because many of these items are

not purchased frequently enough to justify the measurement. With MRO items the major concern tends to be
cost.
Two outputs and four inputs were selected to measure
the relative efciency of the 18 petroleum industry
purchasing departments:
Output One: Total purchase dollars by the purchasing
department (PURDLRS): This is the aggregate measure
of the prime function of the purchasing department, that
is, the purchase of goods and services for the effective
and efcient operation of the rm.
Output Two: Percent of total company purchase
dollars handled by the purchasing department
(PCTTOT): Benchmarks are reported on the percent
of total company purchased goods and the percent
of total company purchased services that are handled
by the purchasing department. These variables
provide an indication of the perceived importance
of the purchasing department within the corporate
hierarchy.
Input One: Total purchasing operating expenses
(OPEXP): This is a measure of both material and
capital utilization. The availability of supplies and
facilities reected in this measure should facilitate the
purchase of supplies and services and the development
of EDI/supplier relations.
Input Two: Total number of purchasing professional
employees (PROF): The number of purchasing professionals measures the available work force for performing the purchasing function at each rm. This variable
provides a measure of the direct labor available to
purchase supplies and services and to manage the supply
base.

L. Easton et al. / European Journal of Purchasing & Supply Management 8 (2002) 123134

Input Three: Total number of administrative purchasing employees (ADMIN): The number of administrative
employees measures the available hourly workforce for
performing support functions needed in the purchasing
department (e.g. secretary, procurement clerk, etc.). This
variable provides a measure of the departments ability
to streamline the procurement process and reduce
purchase order processing time.
Input Four: Total number of active suppliers (SUPPLIERS): This variable provides a measure of the
suppliers available to provide required goods and
services. This approach is particularly valid in light of
the current efforts in purchasing to reduce the supplier
base while improving relationships with the remaining
suppliers. The goal is to reduce the total cost of
conducting business with the supply base.
The rst three input measures represent the costs
involved in operating the purchasing department.
Monczka et al. (1979) conducted the most comprehensive known effort to develop information for use in
improving purchasing measurement. Two hundred and
fty purchasing practitioners were interviewed in 18 US
organizations. While they found no best way of
measuring purchasing performance they did establish
13 purchasing measures, including operating cost per
dollar spent to be an effective measure if employed
consistently. In a commodity type industry such as
petroleum, reducing operating cost can provide a
company with a great advantage over its competitors.
Input measure four, the number of active suppliers,
has taken on greater signicance in the last decade.
Reducing the supplier base down to one or two suppliers
per item has been associated with higher quality, reliable
deliveries, and overall lower administrative costs (Handeld, 1993). In a study conducted in Britain, a reduced
supply base was associated with more partnership type
relationships rather than adversarial ones (Hosford,
1994). A reduced supply base and/or partnerships
facilitate the development of more advanced technologies which has been deemed essential for the petroleum
industry. Because of all the positive aspect associated
with fewer suppliers, a rm may nd that the purchasing
task can be accomplished as well or better with fewer
suppliers making number of suppliers an important
input to measure.

5. Analysis
Seven of the benchmarks included in the 1991 CAPS
study of the petroleum industry can be computed with
the six input and output values used in the DEA
analysis. These seven benchmarks were computed for
each of the 18 rms and are listed in Table 3.
The DEA analysis identied six petroleum rms,
which were rated as one hundred percent efcient (1.0

129

rating). This efciency is relative to the other rms in the

sample. These rms could be identied as best-inclass, for the petroleum industry sample. Ratings of
other rms ranged from a low of 0.12 to a high of 0.77.
DEA ratings of overall purchasing performance for each
of the rms are shown in Table 4. Each was assigned an
arbitrary three letter code in order to protect its identity.
In addition to the DEA rating of overall purchasing
performance, Table 4 also identies the Efciency
Reference Set (ERS) for each of the rms rated
inefcient. These rms, and their associated multipliers,
will form the Hypothetical Comparison Unit (HCU)
which is used for comparing performance with the
efcient frontier. Formulation of the HCU and analysis
of each rms data is discussed in the section addressing
the DEA results.
5.1. Data envelopment analysis
DEA provides a comprehensive evaluation of overall
purchasing performance. This provides management
with an advantage over comparing performance only on
individual measures. DEA also identies a subset of the
top performers that could be designated as best-inclass, which could be further studied to reveal
processes that lead to improved performance. In DEA
this is not an arbitrary classication. These rms
actually have achieved that performance level so it is
known with certainty that it can be reached.
The reason DEA can identify best in this fashion is
that it is an external method of analysis. It does not
depend upon the average of performance in the group in
order to derive either its ratings or its denition of
best. DEA utilizes the extreme, or top performers in
total productivity (ratio of weighted output to weighted
input) to establish these values. It should be emphasized
that DEA does not establish an absolute best, but
only relative to the other purchasing departments that
participated in the DEA performance evaluation.
Another difference in the information DEA provides
is due to the fact that aggregate ratings of performance
utilize a exible weighting scheme. DEA establishes the
weights that each rm uses to combine the individual
inputs and outputs into a single index of relative
efciency. Consequently, as the relative importance of
each of the inputs and outputs varies, the weights used
by the purchasing department may also vary. In this
way, rm characteristics (such as functions, responsibilities, and goals/objectives) may be effectively accounted for.
Additionally, DEA provides insights into the magnitude of potential improvement which other techniques
cannot. Management can identify which inputs and
outputs may be altered, and by how much, in order to
improve overall relative efciency. This can be done by
comparing the departments input and output values

L. Easton et al. / European Journal of Purchasing & Supply Management 8 (2002) 123134

130
Table 3
Benchmark ratings
Firm

A Purchase $
per employee
($M)

B Purchase $
per professional
($M)

C Purchase
expense per
purchase $ (%)

d Suppliers
PER employee

AAA
BBB
CCC
DDD
EEE
FFF
GGG
HHH
III
JJJ
KKK
LLL
MMM
NNN
OOO
PPP
QQQ
RRR

10.00
23.67
15.00
10.20
8.09
14.11
2.29
5.07
7.95
3.82
8.94
4.11
6.75
2.84
2.01
2.35
2.09
0.97

14.29
33.14
27.00
21.39
9.38
32.92
2.75
5.99
16.89
5.90
12.58
10.28
11.17
4.03
4.43
3.87
3.37
1.75

0.5
0.362
0.448
0.666
1.589
0.461
1.384
1.771
1.072
1.536
1.458
1.813
1.05
2.407
4.298
4.395
3.447
7.206

45.0
642.9
111.1
37.9
55.4
157.1
12.5
38.5
594.1
32.3
468.8
64.1
229.2
36.8
64.3
175.3
154.8
29.9

Range

0.97/23.67

Aver.

7.24

1.75/33.14
12.30

0.362/7.206
1.992

64.3
900.0
200.0
79.6
64.2
366.7
15.0
45.4
1262.4
50.0
659.3
160.2
379.3
52.2
141.7
288.1
249.2
53.8

12.5/642.9
163.9

Table 4
DEA ratings

E Suppliers per
professional

15/1262.4
279.5

g Purchase $
PER supplier
($K)
222.0
36.8
135.0
269.0
146.0
89.7
183.0
132.0
13.4
118.0
19.1
64.0
29.0
77.0
31.3
13.4
13.5
32.0
13.4/269.0

h Percent of total
expenditures
(%)
100
87
83
100
72
88
18
95
21
47
33
73
58
40
55
34
26
12
12/100

90.2

58

Table 5
GGG compared with its efciency reference set

Firm

DEA
rating

AAA
BBB
CCC
DDD
EEE
FFF
GGG
HHH
III
JJJ
KKK
LLL
MMM
NNN
OOO
PPP
QQQ
RRR

1.0
1.0
1.0
1.0
1.0
1.0
0.766784
0.757813
0.508976
0.471756
0.448756
0.408673
0.406109
0.333977
0.182493
0.152564
0.143133
0.120604

Efciency reference set

# Times in
efc. set

Outputs and
inputs

6
5
6
5
5
0

PURDLR
PCTTOT

AAA, DDD
AAA, EEE
BBB
AAA, DDD
BBB, CCC, EEE
CCC, DDD
AAA, BBB, CCC, EEE
AAA, DDD
CCC, DDD
BBB, CCC, EEE
AAA, BBB, CCC, EEE
DDD

with those of the DEA generated HCU. The HCU is

comprised of a set of input and output values, which is a
derived composite of the performance of the rms in the
ERS (see Table 4). Table 5 is an example of a
comparison with the HCU for one of the inefcient
rms, GGG.

OPEXP
PROF
ADMIN
SUPPLIER

GGG:
actual
inputs/
outputs
27.5
18
3806
10
2
150

Derived composite
of efciency
reference set (AAA
and DDD)
27.5
18
0.156
1.665
1.0630
115

Excess inputs
of GGG vs.
rms AAA
and DDD
0
0
0.23206
8.335
0.937
35

This analysis will enable the purchasing executive to

determine exactly which input or output levels can be
changed, and by how much, in order to improve relative
efciency. For example, in the case of rm GGG, if all
input levels are reduced to the amounts indicated then
the relative efciency rating would improve to 1.0.
This HCU is derived as a linear combination of the
input and output values from the rms in the ERS. DEA
identies the rms within the ERS and provides
multipliers (the variable value from the linear programs
dual) to be used in computing the HCU. Table 6
provides an example of how the HCU is constructed for
rm GGG. Recall from Table 3, that rms AAA and
DDD were identied as the ERS.

L. Easton et al. / European Journal of Purchasing & Supply Management 8 (2002) 123134

131

Table 6
Derivation of HCU values, rm GGG
Output and inputs

Dual variable for

rm AAA from
DEA

PURDLR
PCTTOT

Outputinput
vector of AAA

Dual variable for

rm DDD from
DEA

100.0
100.0
0.163126

OPEXP
PROF
ADMIN
SUPPLIER

Outputinput
vector of DDD

Composite: efciency
reference set (HCU)

663.0
18.0

27.5
18.0

4.413
31.0
34.0
2466.0

0.156
1.665
1.063
115.0

0.016874
0.5
7.0
3.0
450.0

Management judgment must be used when interpreting these hypothetical numbers. It is possible for the
DEA program to produce numbers that are not feasible.
For example, in some cases the HCU will indicate that
an increase in the output PCTTOT will increase relative
efciency. That PCTTOT gure is sometimes >100%
and it would not be possible to increase the variable to
that level. Also, it would be difcult to employ 1.665
purchasing professionals (Table 6).
The DEA model provides management with additional information that may help in selecting a course of
action for improving performance. DEA assumes that
any point on the efcient frontier is feasible. This
indicates that efciency can be reached using combinations of inputs other than those indicated by the HCU.

6. Management implications
6.1. Management evaluation of DEA
Fifty-four purchasing executives were asked to comment on the acceptability of the existing performance
evaluation techniques and the desired characteristics of
an enhanced evaluation system. Responses showed that
managers desire characteristics that are not present in
current methods of performance evaluation, but are
included in the DEA evaluation.
Specically, managers want to improve productivity
and efciency. They think an evaluation system that
examines several inputs and outputs simultaneously is
important, and they would like to see a system that takes
into account some of the different rm characteristics.
Purchasing managers also believe that too much
emphasis is placed on single performance indexes.
Finally, managers desire a system which provides timely
feedback concerning resource utilization and efciency.
Participants believed that the DEA technique appears to
satisfy the most signicant concerns of the purchasing
managers. Its ability to allow exible weighting and
evaluate several inputs and outputs simultaneously
addresses the signicant characteristics identied by

managers. The characteristics and feedback potential of

DEA make it a valuable tool for enhancing the quality
of decision making and improving purchasing department efciency.
Even though ease of use was not assessed in this
study, Clarke and Gourdins (1991) DEA study on the
efciency of the logistics process surveyed the mangers
on their perceptions of DEA. The managers felt that the
results were easy to comprehend, implying that a
background in operations research is not necessary to
interpret DEA.
6.2. Strengths of DEA
DEA provides additional management information
which enhances the decision making process. Purchasing
executives will now be able to evaluate more alternatives, and immediately see the effects of potential
changes. DEAs exible weighting scheme also gives it
the capability to apply the evaluation measures equitably to all rms within the industry group.
The primary strength of DEA lies in two areas: (1) its
exible weighting scheme which can accommodate the
varying importance of evaluation factors, and (2) its
ability to provide more helpful information to managers
for use in improving performance. DEA provides a
single index of overall purchasing performance exible
enough to account for the different rm characteristics
that affect the required weights. DEA does a good job of
identifying inefciencies; however, it does not provide a
means for distinguishing among the rms rated efcient
(1.0).
The information provided to management allows
them to assess the potential improvement in relative
efciency and develop several alternative courses of
action. These alternatives may involve patterning
behavior after the HCU values. However, managers
should use DEA only as an initial indicator of any
problems or inefciencies. Further analysis into the
situation must be taken before any action is taken or
any behavior modication is initiated (Epstein and
Henderson, 1989).

132

L. Easton et al. / European Journal of Purchasing & Supply Management 8 (2002) 123134

Because DEA provides a unique weighting scheme for

each rm, managers may also explore other possible
uses of resources as those weights indicate the marginal
productivity of each input and output in relation to
increases in relative efciency. Management can then
determine which resources will improve efciency the
most and determine exactly what the change in
performance will be.
Lastly, DEA is an excellent tool for measuring the
progress of one DMU (decision making unit). DEA can
be conducted in a longitudinal fashion to determine if a
DMU is becoming more efcient. Instead of comparing
one DMU to another, the same DMU is compared over
time.
6.3. Limitations of DEA
One potential problem in attempting to implement a
DEA system is the availability of data. This research
was possible because the benchmarking project at the
CAPS had collected aggregate purchasing data from
rms within industry groups. Consequently, most of the
petroleum rms in this study had collected benchmarking type data on a regular basis. This type of DEA
evaluation system is not possible unless a group of
comparable rms collect the data consistently, and on a
regular basis, which could become an arduous task.
Most rms would be unwilling to release such performance data unless an independent party monitors the
DEA system and assures anonymity.
Interpretation of the weights established by DEA
must de done carefully. Management should not
interpret the weights to indicate a value, or measure of
importance, of the individual performance measure
(input/output). DEA establishes the weights in a purely
mathematical fashion. The weights allow each rm to
maximize its efciency rating subject to the constraints
of the problem. Thus, DEA calculates a technical
efciency without regard to any potential social or
economic value that each performance measure may
have. It may be possible, in future research on DEA in
purchasing, to establish a range of possible weights for
each measure. DEA would then establish the technical
weights within the established value bounds.
DEA requires robust metrics, which are widely
accepted measures, interpreted similarly by different
rms or DMUs, and can be used for comparison across
different rms (Caplice and Shef, 1994). DEA has also
been thought of as a tool that can provide valid
measures because it incorporates multiple inputs and
outputs, providing validity over single performance
measures because a greater number of metrics are
utilized. The greater the number of measures, the more
valid the metric, because it is a more accurate
representation of the actual activity performed. Unfortunately, DEAs need for robustness is in direct

contrast with validity. In order to obtain true validity,

some DMUs may want to incorporate idiosyncratic
measures reecting the actual activity of that particular
DMU. Nonetheless, these unique measures are in
contrast with DEAs need for robustness. Summarily,
DEA metrics can only achieve robustness at the expense
of less valid measures.
Another issue with validity and DEA arises because
of the need for it to maintain its power to discriminate.
As the number of inputs and outputs measured
increases, which in turn increases validity, DEAs power
to discriminate decreases. If a group of rms wish to
have both high validity in the measurement and still
allow DEA to discriminate between rms, then a high
number of participants should be involved. The
suggested ratio is 2m
s; where m
s is the product
of the number of inputs and outputs (Dyson et al.,
2001). Therefore, to maintain high validity and discrimination power the number of participating units
should be large. However, this increases the likelihood
that all units are not homogeneous.
DEA is a measure of efciency, not a measure of
effectiveness. Therefore, an additional evaluation system
is needed to ensure that the company is effective in
obtaining its goals. Additionally, DEA is a measure of
relative efciency and not absolute efciency. Companies that lie on the efciency frontier may be lulled into
complacency, which is a dangerous place to be in an ever
changing business environment.
The nal implication for management deals with
implementing DEA recommendations. DEA is only a
tool. It has no knowledge of the business environment of
the rm. It cannot use judgment in calculating the
evaluation. It is possible that certain values derived by
DEA are impossible or undesirable to achieve in reality.
In addition, management may choose corporate or
departmental strategies that require the purchasing
department to operate in a relatively inefcient mode.
For example, the rm may have embarked on a quality
improvement program requiring increases in the number
of purchasing professionals employed. Management has
made a conscious decision on the strategy; however, the
DEA results may indicate that this is inefcient.
Therefore, DEA only provides management with
information to aid in decision making, and is not a
substitute for sound management judgment.

7. Future research
DEA has been shown to possess potential for
improving managements ability to evaluate the performance of the purchasing department within a single
industry group. It is not known whether the technique
can adequately evaluate a less homogenous population
of purchasing departments. One approach to assessing

L. Easton et al. / European Journal of Purchasing & Supply Management 8 (2002) 123134

this capability is to apply a DEA evaluation system to

several industry groups using the benchmarking type
data. In this way, the ability of DEA to compare
purchasing performance across industry groups could be
assessed. If successful, purchasing executives would be
able to assess their performance against a much wider
group of purchasing departments.
The ability to include a more diverse sample of
purchasing departments would also serve to increase the
sample size of any resulting study. This would allow for
the analysis of secondary factors in purchasing performance. Relationships between purchasings efciency
and other rm characteristics such as sales, prots,
company size, organizational structure, and industry
group could be explored. If there is a relationship
between any of rm characteristics and purchasing
performance, the outstanding performers could be
identied more easily. They, in turn, could be further
analyzed to determine what specic practices and
procedures facilitate superior performance.
Evaluation of public sectors purchasing departments
using DEA is currently being explored by researching
the performance of Air Force base contracting ofces.
There are over one hundred of these purchasing
organizations. Each is required to collect the same
performance data on a regular basis. This availability of
data would make the analysis easier to accomplish,
while at the same time greatly increasing sample size,
therefore, enhancing the potential for further secondary
analysis. Evaluation factors which are considered
important to Air Force contracting ofcers are currently
being identied. Many benchmarking type indicators
are kept in the Air Force-wide data base. Once
appropriate factors are identied, a DEA model for
the Air Force base contracting function will be
developed and the performance data analyzed. That
model will then be compared with existing evaluation
techniques and with evaluations previously performed in
the commercial sector. This information should prove
extremely valuable in answering remaining questions
regarding the applicability of DEA.

Appendix A. The DEA model

This is the CCR ratio form of DEA, where there are
n DMUs, m inputs, and s outputs.
Xij > 0 amount of input i used by DMUj ;
Yrj > 0 amount of output r produced by DMUj :
The decision variables in DEA are the unit weights to
be attached to each input and output by DMUj :
Vik unit weight on input i by DMUj ;
Urk unit weight on output r by DMUj :

133

Then, n fractional linear programs are formulated,

one for each DMU. The objective function is the ratio of
total weighted output divided by the total weighted
input. The unit being evaluated under each linear
program is designated as DMU0.
Ps
Ur0 Yr0
1
Max H0 Pr1
m
i1 Vi0 Xi0
subject to
Ps
Ur0 Yrj
Pr1
p1;
m
i1 Vi0 Xij

j 1; 2; y; n

including DMU0 itself

U
Pm r0
Xe; r 1; y; s;
i1 Vi0 Xi0
V
Pm i0
Xe; i 1; y; m;
i1 Vi0 Xi0

where e>0 represents a non-Archimedean constant

smaller than any real positive number, such that keon
for all real values of k and any real number n > 0:

References
Ahn, T., Arnold, V., Charnes, A., Cooper, W.W., 1989. DEA and
ratio efciency analysis for public institutions of higher learning in
Texas. Research in Governmental and Nonprot Accounting 5 (2),
165185.
Banker, R., Charnes, A., Cooper, W.W., Swarts, J., Thomas, D.A.,
1989. An introduction to data envelopment analysis with some of
its models and their uses. Research in Governmental and Nonprot
Accounting 5, 125163.
Caplice, C., Shef, Y., 1994. A review and evaluation of logistics
metrics. The International Journal of Logistics Management 5 (2),
1128.
Carr, A.S., Smeltzer, L.R., 1997. An empirically based operational
denition of strategic purchasing. European Journal of Purchasing
and Supply Management 3 (4), 199207.
Carr, A.S., Smeltzer, L.R., 1999. The relationship of strategic
purchasing to supply chain management. European Journal of
Purchasing and Supply Management 5 (1), 4351.
Chao, C.-N., 1989. Purchasing performanceFviews of purchasing
managers, buyers, and internal customers for different industries.
Unpublished Dissertation, Arizona State University, December.
Charnes, A., Cooper, W.W., Rhodes, E., 1978. Measuring the
efciency of decision-making units. European Journal of Operational Research 2, 429444.
Charnes, A., Cooper, W.W., Divine, D., Ruei, T.W., Thomas, D.,
1989. Comparisons of DEA and existing ratio and regression
systems for effecting efciency evaluations of regulated electric
cooperatives in Texas. Research in Governmental and Nonprot
Accounting 5, 187210.
Clarke, R.L., Gourdin, K.N., 1991. Measuring the efciency of the
logistics process. Journal of Business Logistics 12 (2), 1733.
Cooper, R., 1988a. The rise of activity-based costing systems: part 1what is an activity-based cost system? Journal of Cost Management
2 (2), 4554.
Cooper, R., 1988b. The rise of activity-based costing systems: part 2when do I need an activity based cost system? Journal of Cost
Management 2 (3), 4148.

134

L. Easton et al. / European Journal of Purchasing & Supply Management 8 (2002) 123134

Cox, A., 1996. Relational competence and strategic procurement

management: towards an entrepreneurial an contractual theory of
the rm. European Journal of Purchasing and Supply Management
2 (1), 5770.
Dyson, R.G., Allen, R., Camanho, A.S., Podinovski, V.V., Sarrico,
C.S., Shale, E.A., 2001. Pitfalls and protocol in DEA. European
Journal of Operational Research 132, 245259.
Ellram, L.M., 1994. A taxonomy of total cost of ownership models.
Journal of Business Logistics 15 (1), 171191.
Epstein, M.K., Henderson, J.C., 1989. Data envelopment analysis for
managerial control and diagnosis. Decision Science 20, 90119.
Ghalayini, A.M., Noble, J.S., 1996. The changing basis of performance
measurement. International Journal of Operations & Production
Management 16 (8), 6380.
Ghalayini, A.M., Nobel, J.S., Crowe, T.J., 1997. An Integrated
Dynamic Performance Measurement System for Improving Manufacturing Competitiveness. International Journal of Production
Economics 48 (3), 207225.
Gushee, E.T., Boffey, L.F., 1928. Scientic Purchasing. McGraw-Hill,
New York, NY.
Handeld, R.B., 1993. The role of materials management in
developing time-based competition. International Journal of
Purchasing and Materials Management 29 (1), 210.
Hendrick, T.E., Ruch, W.A., 1988. Determining performance appraisal criteria for buyers. Journal of Purchasing and Materials
Management 24 (2), 1826.
Hosford, N., 1994. Partnerships-whats really going on? Purchasing
and Supply Management, Jul/Aug 3840.
Jeans, M., Morrow, M., 1989. The practicalities of using activity-based
costing. Management Accounting 67 (10), 4244.
Kleinsorge, I.K., Schary, P.B., Tanner, R.D., 1991. The shipper-carrier
partnership: a new tool for performance evaluation. Journal of
Business Logistics 12, 3557.
Kleinsorge, I.K., Schary, P.B., Tanner, R.D., 1992. Data envelopment
analysis for monitoring customer-supplier relationships. Journal of
Accounting and Public Policy 11, 357372.
Lewis, H.T., 1933. Industrial Purchasing. Prentice-Hall, New York,
NY.
Malec, H.A., McClean, J.D., Crain, T.B., Goulet, J.M., 1991.
Benchmarking Purchasing in the Semiconductor Industry with
Sigma Barometers. CAPS and NAPM, Tempe, AZ.
McWilliams, G., 1997. Technology is whats driving this business.
Business Week 3, 146148.
Mentzer, J.T., Brenda, P.K., 1991. An efciency/effectiveness approach to logistics performance analysis. Journal of Business
Logistics 12 (1), 3362.
Metzger, L.M., 1993. Measuring quality cost effects on productivity
using data envelope analysis. Journal of Applied Business Research
9 (3), 6979.
Metters, R.D., Frei, F.X., Vargas, V.A., 1999. Measurement of
multiple sites in service rms with data envelopment analysis.
Production and Operations Management 8 (3), 264281.

Millen, A., 1990. Purchasing 2000: how effective is purchasing?

Purchasing 109 (6), 5862.
Monczka, R.M., Carter, P.L., Hoagland, J.H., 1979. Purchasing
Performance: Measurement and Control MSU Business Studies.
Michigan State University, East Lansing.
Murphy, D.J., 1992. Purchasing performance evaluation: a data
envelopment analysis approach. Unpublished Dissertation, Arizona State University, July.
NAPM, 1991. Purchasing performance benchmarks in the US
petroleum industry. National Association of Purchasing Management, Tempe, AZ.
Narasimhan, R., Jayram, J., Carter, J., 2001. An empirical examination of the underlying dimensions of purchasing competence.
Production and Operations Management 10 (1), 115.
Neely, A.D., Gregory, M., Platts, K., 1995. Performance measurement
systems design: a literature review and research agenda. International Journal of Operations & Production Management 15 (4),
80116.
Neely, A.D., Richards, H., Mills, J., Platts, K., Bourne, M., 1997.
Designing performance measures: a structured approach. International Journal of Operations & Production Management 17 (11),
11311152.
Rhodes, E.L., 1986. An exploratory analysis of variations in
performance among US parks. Measuring Efciency: an Assessment of Data Envelopment Analysis New Directions for Program
Evaluation. Jossey-Bass, San Francisco.
Rhodes, A., Koen, A.D., 1996. U.S. reners nd benet in joint
ventures with foreign partners. Oil and Gas Journal 94 (30), 1620.
Sexton, T.R., 1986. The methodology of data envelopment analysis.
Measuring Efciency: an Assessment of Data Envelopment
Analysis New Directions for Program Evaluation. Jossey-Bass,
San Francisco, Winter.
Sherman, H.D., 1986. Managing productivity of health care organizations. Measuring Efciency: an Assessment of Data Envelopment
Analysis New Directions for Program Evaluation. Jossey-Bass, San
Francisco, Winter.
Stanley, L., 1992. Purchasing performance: a structural model of the
effects of strategic supplier relationships and organizational
structure. Proceedings of the 1992 National Purchasing and
Materials Management Research Symposium. National Association of Purchasing Management and California State University,
Fresno, March 1214, pp. 5570.
Van Weele, A.J., 1984. Purchasing control: performance measurement
and evaluating the industrial purchasing function. Netherlands
Wolters-Noordhoff, Groningen.
Weber, C.A., Desai, A., 1996. Determinants of paths to vendor market
efciency using parallel coordinates representation: a negotiation
tool for buyers. European Journal of Operational Research 90,
142155.
Wisner, J.D., Tan, K.C., 2000. Supply chain management and
its impact on purchasing. Journal of Supply Chain Management
36 (4), 3342.