International Journal of Quality Science

The use of experimental design in the development of new products

Marit Risberg Ellekjær Søren Bisgaard
Article information:
To cite this document:
Marit Risberg Ellekjær Søren Bisgaard, (1998),"The use of experimental design in the development of new products",
International Journal of Quality Science, Vol. 3 Iss 3 pp. 254 - 274
Permanent link to this document:
http://dx.doi.org/10.1108/13598539810229230
Downloaded on: 09 February 2016, At: 11:20 (PT)
References: this document contains references to 27 other documents.
To copy this document: permissions@emeraldinsight.com
The fulltext of this document has been downloaded 1325 times since 2006*
Users who downloaded this article also downloaded:
Zhihai Zhang, (1998),"Application of experimental design in new product development", The TQM Magazine, Vol. 10 Iss 6
pp. 432-437 http://dx.doi.org/10.1108/09544789810239164
Downloaded by New York University At 11:20 09 February 2016 (PT)

R. Konda, K.P. Rajurkar, R.R. Bishu, A. Guha, M. Parson, (1999),"Design of experiments to study and optimize
process performance", International Journal of Quality & Reliability Management, Vol. 16 Iss 1 pp. 56-71 http://
dx.doi.org/10.1108/02656719910226914
Jiju Antony, Daniel Perry, Chengbo Wang, Maneesh Kumar, (2006),"An application of Taguchi method of experimental
design for new product design and development process", Assembly Automation, Vol. 26 Iss 1 pp. 18-24 http://
dx.doi.org/10.1108/01445150610645611

Access to this document was granted through an Emerald subscription provided by emerald-srm:198285 []
For Authors
If you would like to write for this, or any other Emerald publication, then please use our Emerald for Authors service
information about how to choose which publication to write for and submission guidelines are available for all. Please
visit www.emeraldinsight.com/authors for more information.
About Emerald www.emeraldinsight.com
Emerald is a global publisher linking research and practice to the benefit of society. The company manages a portfolio of
more than 290 journals and over 2,350 books and book series volumes, as well as providing an extensive range of online
products and additional customer resources and services.
Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the Committee on Publication
Ethics (COPE) and also works with Portico and the LOCKSS initiative for digital archive preservation.

*Related content and download information correct at time of download.

 IJQS
3,3
254
Received July 1997
Accepted June 1998
Downloaded by New York University At 11:20 09 February 2016 (PT)
International Journal of Quality
Science, Vol. 3 No. 3, 1998,
pp. 254-274, © MCB University
Press, 1359-8538
The use of experimental
design in the development of
new products
Marit Risberg Ellekjær
MATFORSK, Norwegian Food Research Institute, Osloveien, Norway, and
Søren Bisgaard
Center for Quality and Productivity Improvement and Department of
Industrial Engineering, University of Wisconsin-Madison, Wisconsin, USA
Introduction
The economic success of manufacturing companies depends in large part on
their ability to identify the needs of customers, and quickly translate these into
high quality products that can be produced at low costs to satisfy those needs.
Product development has therefore been recognized as paramount in
importance for long-term business survival by many authors (Urban and
Hauser, 1993; Wesner et al., 1994; Wheelright and Clark, 1992; Zangwill, 1993).
Lately increasingly competitive pressures have drastically shortened the
economic life span of products. This means the product development costs have
to be amortized over progressively shorter time horizons, providing further
impetus for focusing on effective product development strategies.
Based on data on the average cycle time in the different steps of the product
development process, Page (1993) found the physical development of the
product consumed most of the cycle time. This and other evidence lies behind
several proposals and initiatives for making the physical product development
process more effective (Cooper, 1993; Erhorn and Stark, 1994; Juran, 1988;
Mørup, 1993; Urban and Hauser, 1993; Wesner et al., 1994; Zangwill, 1993).
However, few, including Page (1993), seem to have recognized that a large
portion of the physical product development cycle time is spent experimenting.
It is thus possible to speed up the product development by performing
experiments more efficiently. Specifically, based on our field experience, we
have found that effective methods of experimentation is the key, obviously
among others, to improve product quality, reduce manufacturing costs, and
shorten the cycle time for the physical development of products in a wide
variety of industries.
This work was carried out while Ellekjær was visiting the Center for Quality and Productivity
Improvement (CQPI), University of Wisconsin-Madison. We would like to thank the Fulbright
Foundation who sponsored Ellekjær’s visit at CQPI. This work was also sponsored by a grant from
the National Science Foundation, grant number DMI 9500140. We also want to acknowledge the help
provided by CQPI’s report committee and their valuable comments on a previous draft of this paper.
 The current literature on methods for effective experimentation, also known The use of
as statistical experimental design, has emphasized the technical aspects of experimental
these tools. Many potential users, especially managers and engineers, may design
therefore not fully appreciate what these tools are, what they can be used for,
and what role they may play in the product development process. In this article
we will provide a conceptual discussion on how these techniques can be applied
to reduce product development time, and as a tool to build quality into products 255
from the early design phases and all the way downstream in the product
development process. Examples, several from our own experience, will be used
as illustrations of the concepts and ideas.
To set the stage, in the following section we will outline the product design
process as we see it. Next we will discuss how experimental design methods can
Downloaded by New York University At 11:20 09 February 2016 (PT)

be applied in every phase of this process, how the application of such methods
may help to reduce the product development cycle time, improve the quality of
the end-product, and reduce the manufacturing costs of the product.

The product design process

Before we enter into a discussion of the role of quality improvement methods,
and in particular the role of effective experimentation in the product
development process, let us first outline the process itself. Many authors
(Cross, 1989; Pahl and Beitz, 1984; Wheelright and Clark, 1992), have discussed
issues related to the product development process; we will present a
modification to the general process, first introduced by Bisgaard (1992a). A
graphical representation of this modified process is presented in Figure 1.
It can be seen from Figure 1 that most of the steps are the same as those
found in traditional textbook depictions of the process. However, we represent it
not as a linear, forward-moving process with a beginning and end, but as an
ongoing cyclic process. We believe that the cyclic product development model is
a better synthesis of the product development process as it actually takes place,
as well as how it should take place in practice: as a continuous (cyclic) learning
process.
In our fieldwork we have observed that most new products on close
examination, but of course to a varying degree, seem to be modifications of
previous products. For example, a “new” car model is not a “new” car but a
modification of the general concept of a car and most likely retaining many
features from a previous model made by the same company. Although some
new products really are “new”, the overwhelming majority is better conceptual-
ized as “new generations” rather than entirely new products. An important
characteristic of the cyclic model, as opposed to the traditional linear process, is
that learning and experience gained from the development and use of previous
generations of a product is shown explicitly to be used to design and develop
the next. The product development process can therefore be viewed as the
Deming-Shewhart cycle also known as the plan-do-check-act (PDCA) cycle
(Deming, 1986). Figure 1 only shows one macroscopic PDCA cycle. However,
 IJQS
k ac
3,3
h ec Ramp-up and
Production t io
c n

Trial
Production
256 Customers
Market

Corrective
Preparation for Feedback
Manufacturing /Learning
Downloaded by New York University At 11:20 09 February 2016 (PT)

Data collection
– customer data
Prototype development – service data
testing, “robustification”;
product optimization – complaints
– benchmarks
Customer – ideas
needs
specifications

do
an
functions

pl
concept
development

forms

Figure 1. technologies
The cyclic product
design model (3)
NEW materials
processes
ideas

within the development process microscopic PDCA cycles also occur repeatedly
(Bisgaard, 1992a; Erhorn and Stark, 1994). By systematizing and institutional-
izing learning and memory, we increase the likelihood of providing the
customers with progressively better products, and more likely prevent
backsliding to lower levels of quality, sometimes manifested in costly recalls
and high failure rates. Systematizing learning and institutional memory is also
one of the key features of experimental design; this will be exemplified below,
but first we will discuss the role of experimentation during product
development.

The role of experimentation during the development of new products

The design and manufacture of products that meet customers’ expectations is a
complex task. What makes product development difficult is the influence of a
multitude of variables on the product’s performance characteristics. Often these
complex interrelationships are little known and poorly documented; in turn this
 leads to uncertainties about future performance, possibly causing quality and The use of
reliability problems. The problem is further compounded by the fact that, by the experimental
very nature of product development, we are likely pushing the limit of our design
knowledge and understanding. Nevertheless, product designers need to make
decisions regarding the specifications of the product involving these complex
interrelations. Some relations may be known from either theory or past
experience, others, and these are often a great majority, must be determined by 257
experimentation if we are not to make decisions based on intuition or
guesswork alone. Thus we have in our fieldwork found that product developers
use experiments daily to test materials, processes and devices, develop new
ideas and concepts, optimize performance, make products robust to variation,
and to improve their reliability. Such experiments may range anywhere from
Downloaded by New York University At 11:20 09 February 2016 (PT)

simple one-shot trials to extremely complex multi-factor experiments based on

statistical design principles.
The common perception is that experimentation only plays a role at the
prototype testing stage of the product development cycle. However, we have
found there are rich and rewarding opportunities in using experimentation
within all the different phases of the product development process depicted in
Figure 1. One use of experimentation is to generate alternative designs for
deliberate selection of future actions. However, in Table I we provide a list of
possible experimental objectives at the different stages of the product
development process.
It is well-known (Wheelright and Clark, 1992) that commitments to major
cost components of a product are made by decisions made in the early stages
of the development process. Moreover, such decisions are often difficult or
expensive to change later. The best opportunities to cut cost, reduce cycle
time, and build quality and reliability into a product, are therefore at the early
stages. However, at those stages we know the least, relatively speaking,
compared to later stages where we have the benefit of what has been learned

Preparing for
Concept Prototype manufacture and Ramp-up and
development development trial production production

• test existing • product performance • product • process yield

theories and concepts • robustness and manufacturability • product and
• establish new reliability • simplicity process quality
theories and concepts • sensitivity to • sensitivity to • product and
• proof of principle component variation component variation process reliability
• what happens if…? • simplicity • tolerancing • process simplicity Table I.
• generating new ideas (specification • cost Objectives of
of the allowed limits experimentation at
of the parameters) different stages of
• reliability the product development
• cost process
 IJQS in earlier stages. Thus, we likely make comparably less informed decisions.
3,3 Ideally, therefore, most of the experimental effort ought to be concentrated on
the concept and prototype development stages. Such an approach will allow
the development team to make more informed decisions based on real data,
not just intuition and guesswork. Experiments with many variables may help
identify the most important variables that have a significant effect on quality
258 and reliability of the product. By identifying these variables early on, we
might prevent, or at least reduce, the number of costly changes later in the
process.
Before the product developers (design engineers) start the development of a
product, it is important that the project team define precise objectives regarding
the performance of the new product as well as the manufacturing goals. This will
Downloaded by New York University At 11:20 09 February 2016 (PT)

ensure a common mindset among the development team with regard to critical
quality features. At this stage customer input is of primary importance (Bisgaard,
1992a; Mørup, 1993; Zangwill, 1993), but suggestions from manufacturing are
also important. If those sources of input are not fully exploited the quality of the
product may suffer. For example, without input from the customers we might
develop a product that does not satisfy the customer’s need and without input
from manufacturing a product may end up too complex, have too many parts, or
require too many steps in manufacturing. Implementation of manufacturing
objectives early in the development cycle also helps the team plan ahead and thus
minimizes the number of design changes later in the process (Mørup, 1993;
Wesner et al., 1994; Wheelright, 1988). An added benefit is, if fully integrated in
the development process, that manufacturing engineers may start to investigate
manufacturing solutions in parallel with the development of the product. This in
turn will further ensure the activities at the different stages of the product
development process are integrated.
To define precise objectives related to product performance and manufactur-
ing, the product development team should ideally be cross-functional and
preferably include at least one team member knowledgeable in experimental
design. Once the product development team has a common understanding of
the critical quality features of the product, it is easier for them to develop a
robust and reliable product with the desired quality characteristics.
Related to the product development objectives is the choice of what to
measure as surrogates for the objectives. For example, in food product
development projects the sweetness, the pH, the colour and many other
measurable responses are related to the general objective of a well-tasting
and appealing product. Thus, prior to experimentation at any stage in the
product development process, it is important to select relevant response
variables (measurable quality characteristics) that are in concord with the
defined goals, and ultimately with the customers’ needs and expectations of
quality. To identify good response variables one should start with the
strategic quality features of the product. The response variables should, if
 possible, also be accompanied with targets or desired conditions related to The use of
the goals. experimental
As implied, in many cases several response variables are needed to describe design
the total quality of a product. Sensory quality of food products, a combination
of taste, smell, texture and appearance attributes, is an example of that. In such
cases, multivariate methods can be used in combination with experimental
design to be able to efficiently view the total performance of the product 259
(Ellekjær et al., 1997; Ellekjær et al., 1996; Schönkopf et al., 1996).

The use of experimental design in product development

Experimental design should be used at all stages during product development
where experiments are performed. It is one of several techniques important for
rapid and goal-oriented product development (Mørup, 1993; Urban and Hauser,
Downloaded by New York University At 11:20 09 February 2016 (PT)

1993). Experimental design is an effective tool to identify the most important

design variables (raw materials and processing), and to select the conditions
that result in lowest cost, highest quality and most reliable product.
In statistical designed experiments (Box et al., 1978) several parameters
(factors) are changed simultaneously. For example, if three factors are involved
and each can be varied at two levels, then there will be 2 × 2 × 2 = 23 = 8 possible
combinations. Indicating the low level of a factor with a minus and the high
level with a plus, these eight combinations can be written out as a table called
the experimental design matrix as shown in Table II. In this table each line
provides the experimenter with guidelines for how each of the eight trials in the
experiment should be executed. Since the experiment shown in Table II includes
all the eight possible combinations of the three factors it is called a full factorial
design. Later we will show experiments where only a fraction of the number of
possible trials will be conducted. Such experiments are called fractional
factorial designs (see Box et al., 1978).

Trial number A B C Response

1 – – – 108
2 + – – 115
3 – + – 101
4 + + – 130
5 – – + 110
6 + – + 150
7 – + + 130
8 + + + 154
Note: Table II.
Minuses indicated the low level and pluses the high level of the factors. Each line in the table A design matrix for a
provides the experimenter with prescriptions for how the three factors should be adjusted for two-level factorial
each of the eight trials. The last column to the right is used to record the results of the trials. experiment with three
The numbers displayed are from the propeller example. factors A, B and C
 IJQS It is important to notice the factorial type of experiments break with the time-
3,3 honoured tradition of varying only one-factor-at-a-time. When appropriately
used, statistically designed experiments provide significant time and resource
savings, reduce the disturbing influence of variation, and can help reveal
interaction effects between different factors that could not be uncovered with
traditional approaches. In particular, it is often found that factors may interact
260 in unexpected and unusual ways. The discovery of such unexpected
interactions may provide the development team with unexpected rewards. To
make this discussion more concrete let us present a simple yet illustrative
example involving the development of a new design concept for an outboard
motor.
Downloaded by New York University At 11:20 09 February 2016 (PT)

An example: the development of a propeller-shaft assembly mechanism

An outboard motor manufacturer was contemplating a new design of the
propeller-drive shaft assembly. Their current design attached the propeller to
the drive shaft with a steel pin through the propeller and the drive shaft. Thus
the pin’s primary function was to secure the transfer of the rotational force from
the drive shaft to the propeller. However, an equally important feature of the pin
was to sheer when overloaded to avoid damaging the gearbox and other vital
parts when, for example, the propeller hit a rock in the water. Although this
simple overload mechanism may save the boat owner expensive repairs, it also
has a serious drawback. When the sheer pin breaks, the boat is without power,
and it is almost impossible to replace the pin while on the water and hence can
possibly cause navigational hazards.
To develop an alternative design solution, the company formed a cross-
functional team consisting of people from the design department, manufac-
turing and service. After a few initial meetings it was suggested that a rubber
sleeve should be placed between the drive shaft and the propeller. A sketch of
this proposed design is shown in Figure 2. The idea was that the rubber sleeve
would provide enough friction to establish a solid joint between the drive shaft
and the propeller for normal operation, but beyond a certain torque the
propeller would slip. This slip mechanism could protect the motor’s vital parts
from overloading.
To develop a prototype for “proof of principle” the engineers needed to
determine the parameters determining the slip torque. It was clear the joint
between the rubber sleeve and the propeller would have to be an interference fit.
That is, the outer diameter of the sleeve would have to be slightly bigger than
the inside diameter of the hole in the propeller. However, because of the
interference fit, the manufacturing engineers pointed out it would be quite
difficult to assemble the propeller on the shaft both in production and by
customers in the field. It was therefore suggested the hole in the propeller be
tapered such that the entry diameter was slightly larger than the outer diameter
of the rubber sleeve forming a funnel for easier assembly. To this suggestion the
design engineers responded that it would weaken the joint, and hence reduce
 The use of
experimental
design

261

Rubber
Downloaded by New York University At 11:20 09 February 2016 (PT)

Figure 2.
Proposed propeller
assembly

the slip torque. However, they conceded that if the press fit was made tight
enough, the additional friction in the smaller end of the hole could possibly
compensate for the effect of making the hole tapered.
Based on this, it was suggested that an experiment be conducted to get some
ideas about what the influence of the basic parameters were on the slip torque.
It was decided the factors to be included in the experiment should be the
hardness of the rubber sleeve, the diameter of the rubber sleeve and whether the
hole was straight or tapered. These three factors and their levels were A: the
outer diameter being either 0.688 or 0.710 inches, B: the durometer which is a
measure of the hardness set at either 55 or 60 on that scale, and C: whether the
bore of the propeller was tapered or straight. An eight-run factorial design
consisting of all the possible combinations of three factors each at two levels as
shown in Table II was then conducted. An alternative representation of this
experiment is as a three-dimensional cube as shown in Figure 3. In this

130 154

60 101 130

Durometer
110 150
Straight Figure 3.
Results of first
Bore
55 108 115 experiment on new
Tapered propeller assembly
0.688 0.710 design
Diameter
 IJQS representation the eight corner points correspond to the eight factor
3,3 combinations in Table II and the numbers displayed there are the eight
outcomes of slip torque from the experiment.
A formal statistical analysis can be performed on the data from this full
factorial experiment. However, simple visual inspections of the numbers in the
plot suffice. By comparing the average of the numbers on one side of the cube
262 with those on the other, we can estimate the effect of each of the three factors.
For example, on the right side of the cube where the diameter is set at the high
level the results are 115, 130, 150 and 154 and therefore on average 137.25.
However, on the left side where the diameter is set at the low level the results are
108, 101, 110 and 130 and therefore on average 112.25. The effect of changing
the diameter is therefore the difference 137.25 –112.25 = 25. Thus increasing the
diameter of the rubber sleeve increases the slip torque by 25 units. This is also
Downloaded by New York University At 11:20 09 February 2016 (PT)

shown in Table III which gives the averages at low and high level as well as the
effects of all three factors.
From Figure 3 and Table III it also appears that the harder rubber increases
the higher slip torque. Moreover, we can see the straight hole, as expected, gave
higher slip torque. Some of these general effects were in this case obvious for the
engineers considering the design. In other cases an experiment may reveal
unexpected or unpredictable effects. However, even in this case where the
general results were as expected the experiment provided very useful
information because what the engineers needed was concrete numbers not just
general relationships to proceed with their design work.
After looking over the results the engineers decided a slip torque of less than
150 probably would cause slippage problems during acceleration or when
pushing big boatloads. Since results of 150 with the current factor levels only
could be achieved by using a straight hole and a straight hole was undesirable
from a customer and manufacturing point of view, it was decided to conduct a
follow-up experiment. In the follow-up experiment the outer diameter of the
rubber sleeve was increased to 0.717 inches but the other factors were kept at
the same levels as before. The follow-up experiment required only four more
trials. The results of those are shown on the cube plot in Figure 4.
From Figure 4 we see the larger interference fit provided by an outer
diameter of 0.717, with the harder rubber of Durometer 60, and a tapered hole, a
satisfactory compromise between slip torque and easy assembly was achieved.
However, because of the large variation in the data, it was decided to replicate
this satisfactory combination. As it turned out, these confirmatory trials were

Table III.
The average slip torque Factor Low level High level Effects
for the two different levels
of diameter (0.688 and
Diameter 112.25 137.25 25
0.71), durometer (55 and
60) and bore (tapered and Durometer 102.75 128.75 26
straight) Bore 113.50 136.00 22.5
 The use of
130 154 292
experimental
60
design
101 130 194

Durometer 263
110 150 144 Straight

Bore Figure 4.
108 115 134 Results of follow-up
55 Tapered
experiment for the
0.688 0.710 0.717 outboard motor propeller
Diameter assembly design
Downloaded by New York University At 11:20 09 February 2016 (PT)

consistent providing the engineers enough confidence in the results to continue

their design work using this combination in the next prototype models.
The example above shows that several technical issues were solved quickly
and systematically by performing simple factorial experiments. In this case the
experiment allowed the project to go forward, and not get bogged down in
endless discussions. Good experiments at the early stages of the development of
a product often shorten the development cycle time, and reduce the risk of
making design decisions on a shaky foundation. This in turn reduces the risk of
quality problems emerging later when the product is manufactured or is on the
market.

Experimental design for different purposes

During the development of a product there are many different reasons for
performing experiments. There are, therefore, also a wide variety of experi-
mental designs available specialized for these different purposes. The experi-
ment presented on the propeller joint is typical for the situation where the
objective is to gain knowledge necessary for developing a new product design
concept. The initial design was a full factorial design in three factors each at
two levels. This type of design is useful when only a few factors are involved.
However, for a larger number of factors, the number of combinations increases
and full factorial designs often become prohibitively expensive. As an alterna-
tive we may then use a carefully chosen fraction of all the possible combinations
also known as fractional factorial designs. Such designs are in particular useful
when the product development team is interested in finding out which factors,
out of a large number of potentially influencing factors, have the biggest effects
on certain characteristics. Such experiments are called screening experiments.
Frequently experiments are used to maximize, minimize or in general
optimize products. Experiments for that purpose are called optimization
experiments or response surface experiments. A third category of experiments
has as the objective to discover design parameter combinations for which the
 IJQS product is insensitive to uncontrollable variability. For example, when
3,3 designing an automobile engine it is important that its performance is
insensitive to the uncontrollable factors of ambient temperature and humidity,
and when developing a household detergent it is important that its cleaning
characteristics are insensitive to water temperature and hardness variations.
Experiments with the purpose of discovering factor combinations that are
264 insensitive to variation in environmental factors are often called robustness
experiments. In the section below we will briefly discuss each of these different
classes of experiments.

Screening experiment
A product development team may collectively have many ideas about the
Downloaded by New York University At 11:20 09 February 2016 (PT)

effects of materials, additives, processing equipment, processing conditions,

etc., that may potentially have significant impact on the quality and cost of the
finished product. Often each team member may have an opinion about which
variables are most important to investigate. However, it is not unusual that
such opinions differ from team member to team member. To identify, on a
factual basis, the few most important variables among the many variables
proposed, and using a minimum of time and resources, a screening experiment
may often be useful. Fractional factorial designs (see Box et al., 1978) are
among the most widely used designs for that purpose. Such designs may help
identify the few most important variables, and help the team focus on these for
further investigation. In other cases, information about variables that have no
influence is also useful because it allows the product developer to choose the
least expensive setting for those variables or show the product is robust to
changes in those variables.
To illustrate the usefulness of screening experiments, let us consider the
following example from a company that develops ingredients for food products.
This company was in the process of developing a new emulgator. However, to
benchmark they wanted to compare its performance to another emulgator that
was frequently used by their customers. From their previous work on emul-
gators the development team knew that several variables might affect the
performance, and wanted to test the emulgator in as broad an environment as
possible. The effect of the following factors were investigated on the stability of
the emulgators: emulgator concentration (A), protein type (B), fat concentration
(C), fat type (D), protein concentration (E), temperature (F), NaCl (G) and
emulgator type (H), i.e. the new versus the standard emulgator. The experiment
therefore had eight different factors, and the two alternative levels for each of
the factors, which are listed in Table IV.
A full factorial experiment for eight factors would require 256 individual
trials. That many tests were clearly out of the question. It was therefore decided
to run a 16th fraction of the full factorial. In the technical notation used for such
designs that is known as a 2 8-4 fractional factorial design. Such a design
includes only 16 trials, and from it we will be able to estimate the effect of the
 Factors Levels
The use of
experimental
A Emulgator concentration 0.5 percent 1 percent design
B Protein type 1 2
C Fat concentration 40 percent 60 percent
D Fat type 1 2 265
E Protein concentration 0 percent 10 percent
F Temperature 40°C 50°C Table IV.
The eight variables
G NaC1 0 percent 1.2 percent
investigated in the
H Emulgator type New Commercial emulgator testing
Downloaded by New York University At 11:20 09 February 2016 (PT)

eight main factors and a number of interaction effects although only in groups.
This is because one of the consequences of not executing all the possible
combinations is that we get confounding of effects. That is, the effects of several
interaction effects will be mixed up and only the joint effect of them can be
estimated.
Without going into the details of this experiment, it was found from these 16
trials that the stability seemed to increase with increasing fat concentration (C)
and protein concentration (E). In addition, the results indicated that there might
be an interaction effect between C and G or E and F. It is not possible from this
experiment to identify which of them is most likely as they are confounded
because we executed only a fraction of all the possible trials. In such situations
it is important to use the subject-matter knowledge to interpret the results.
However, the key finding was that there did not seem to be any difference in
stability between the two emulgators (H).
These results can be seen more explicitly from the so-called Normal plot
shown in Figure 5. A Normal plot (Box et al., 1978) is useful for identifying the
factors with the largest effect in unreplicated factorial and fractional factorial
designs. To interpret the Normal plot the user needs to know that the estimated
effects that fall off the upward sloping line are those which have an effect. The
remaining effects on the line appear to be inert. In Figure 5 we see, as indicated,
that the significant effects are C and E. However, factor H appears to be inert
meaning it does not make a difference whether the new or the standard
emulgator is used, hence the conclusion.
This experiment demonstrates the potential of the fractional factorial
approach to screening a large number of factors in few trials. In this specific
case the screening experiment was used to evaluate two different prototypes
over a wide variety of environmental factors. With such experiments the
product can, for example, be tested and the most robust and reliable prototype
design selected. Moreover, the insight gained from the experiment can often be
used to suggest modifications of the product that may go beyond any of the
tested variants and hence provide genuine breakthroughs.
 IJQS Normal score
3,3 2
E
C
1
3 AB=CG=DH=EF
266 r
c 0
s
n

–1

Figure 5.
Downloaded by New York University At 11:20 09 February 2016 (PT)

A normal plot showing –2

the effect of the different
factors in the emulgator –10 –5 0 5 10 15 20 25
cor 3
experiment
Estimated effect on stability (exp.1)

Ellekjær (Ellekjær et al., 1996) provided another example of the use of screening
design on product improvement of processed cheese. In this article the
effectiveness of using experimental design in combination with multivariate
methods was discussed.

Optimization experiments
Response Surface Methodology (RSM) was introduced in the early 1950s as a
method for experimental optimization of industrial products and processes. The
basic concepts of RSM can be understood by looking at the six pictures in
Figure 6.
We will use a process as an example but the same ideas apply to the
optimization of products. For simplicity, suppose two factors, pressure and
temperature, are believed to have an impact on the yield of a chemical process,
and that it currently is run at pressure P0 and temperature T0 as shown in
Figure 6a. Suppose further the yield is much lower than expected. Unknown to
the experimenter, however, there exists an optimal factor combination that
gives a yield of 92 percent of the theoretical optimum located as indicated by the
contour plot in Figure 6b. The challenge facing the experimenter is how to reach
the “mountain top” in as few trials as possible.
One approach would be to lay out a grid of factor combinations. For example
if each of the two factors were to be set at five levels we would need to carry out
5 × 5 = 25 trials, but with five factors the total number of combinations would
be 55 = 3,125 trials. Thus, the grid search method is not a viable experimental
strategy. A better strategy can be devised by noticing the sequential nature of
most industrial experiments. First we observe that when an initial experiment
has to be planned, the experimenters know the least about the system they are
 The use of
(a) (b)
60% experimental
Temperature

Temperature
70%
80% design
90%

(p0,t0)
92% yield
267
Pressure Pressure

(c) (d)
Temperature

Temperature
Downloaded by New York University At 11:20 09 February 2016 (PT)

Pressure Pressure

(e) (f)
Temperature

Temperature

Figure 6.
(a) to (f) show the
different phases of an
optimization study
Pressure Pressure

studying. However, at later stages they will know what has been learned from
previous experiments. Thus, as in a casino, it would seem wise to gamble with
small stakes initially until we have learned the game. Similarly, it is wise to
adopt a strategy that starts out with a small experiment and builds up the
design as information becomes available.
This strategy consists essentially of three phases (Box et al., 1978): initial
exploration, exploration along the direction of steepest ascent, and exploration
of near optimal conditions. The initial phase involves the use of a relatively
small inexpensive two-level factorial or fractional factorial designs with each
factor changed a little below and above its standard operating condition plus,
possibly one or more centre points at that condition. This is illustrated in
Figure 6c with a small 22 design for two factors plus a centre point. Those five
trials will allow the experimenter to estimate locally the slope of a small
approximating (first order or linear) plane. The vector pointing in a right angle
to the contours of this small plane indicates the direction in which the yield
locally increases the most. This is also known as the direction of steepest
ascent.
 IJQS The next phase of the RSM strategy involves trying out, step-by-step, factor
3,3 combinations along the path of steepest ascent as indicated in Figure 6c. As
long as this step-wise strategy yields increasing results, the experimenter will
continue. However, when the yield does not increase anymore, another two-level
factorial experiment is performed around the current best factor combination,
see Figure 6d. From this two-level design the direction of steepest ascent is
268 re-evaluated and the process continued.
This evaluation of the direction of steepest ascent followed by trials along
the steepest ascent may proceed through several iterations depending on the
shape of the response surface. However, the process will eventually bring the
experimenter close to an optimum, or more precisely, a stationary point as in
Figure 6e. At that point the experimenter might simply declare victory and
Downloaded by New York University At 11:20 09 February 2016 (PT)

stop. However, it is often advantageous to explore the near stationary conditions

in more detail. For example, if the region around an optimum condition is
relatively “flat” the process operators need not control the factors very closely.
In other words, the process is relatively robust to factor changes. However, in
other situations the “optimum” conditions may actually not be a single point,
but more like a mountain ridge, or in rare occasions, a saddle point.
Central composite designs can be used to explore the near optimum
conditions (Box et al., 1978). A two-dimensional version of such a design is
exemplified in Figure 6f. The information from such a design makes it possible
to fit a second order polynomial. This in turn will help determine the shape of
the response surface near the stationary point to indicate whether it is a peaked
or flat maximum or minimum, a ridge, or a saddle point. Starting with a base
design consisting of a two-level factorial or fractional factorial design we can,
when the need is perceived through curvature checks, augment the initial
design with star points to get a central composite design. Thus, when using the
response surface strategy, not just for two factors as illustrated, but in general
for any number of factors, curvature is studied when its presence has been
revealed and the designs are built up dynamically.
An example from the development of a new type of paper used for printing
currency is used to illustrate this step-wise strategy. A major problem with
ageing currency bills is the tendency of ink to fade and wear off (Barrios, 1994;
Denes et al., 1995). Thus a key quality characteristic for paper used for printing
currency is its ability to absorb and hold ink. In preliminary studies it had been
found that if the surface of paper were modified by treating it in a plasma, the
paper’s ability to absorb and hold ink could be modified. However, it was not
known exactly how and by how much the paper needed to be modified to obtain
the best results. By using a screening design they had identified three different
processing variables that had an effect on the durability of the currency paper.
These were: plasma reactor pressure, treatment time and power. The contact
angle was used as a measurement for the durability of the currency paper; the
smaller the contact angle the better.
 In order to determine the processing variable settings resulting in a mini- The use of
mum contact angle an optimization design was planned for the three selected experimental
factors. In Figure 7 both the contour plot and response surface for silicon design
tetrachloride-treated paper at a given treatment time are presented for varying
settings of pressure and power.
We can see a minimum contact angle was achieved at a pressure of about 80
and for power at about 70. However, a variation in pressure between 70 and 90, 269
and in power between 60 and 80 had a minimal effect on the quality of the paper
(measured as contact angle).
In addition, to show the usefulness of optimization design for rapid and goal-
oriented development, this example also illustrates the usefulness of response
surface methodology in identifying how sensitive the optimum results are to
Downloaded by New York University At 11:20 09 February 2016 (PT)

variations in processing conditions around the optimum. A pointed response

surface near the optimum implies the processing conditions must be carefully
controlled. However, a rather flat response surface indicates the process can
tolerate variations without a serious drop in the quality under study.

Robustness design
Robustness experiments are performed to make products that are robust to
variation in variables that under normal use cannot be controlled. Such
variables may, for example, be differences in raw material, equipment, different
operators, and variation in the treatment of the product. The idea of robustness
experiments is to identify the settings of the controllable design factors so the
uncontrollable variables have reduced influence on the response. The original
application of this idea was, although not explicitly promoted as such,
introduced by Fisher (1935). He was involved in designing experiments aimed
at identifying varieties of barley that gave reasonably good yield irrespective
of the different conditions such as soil and weather under which they were

Figure 7.
The contour plot and
response surface for
silicon tetrachloride
treated paper at
different combinations
of pressure and power
 IJQS produced. Later, the idea of robustness was pursued explicitly by Michaels
3,3 (1964) who was involved in developing household detergents that were robust
to changes in water temperature, water hardness, etc. More recently the idea
has been pursued and raised to prominence by Taguchi (1987). He
demonstrated the potential of robust design for a wide variety of industrial
applications. The idea is to ensure the manufacture of products that
270 consistently satisfy specifications by making the process insensitive or robust
to external sources of variability. (For more detailed information about robust
design, see Bisgaard, 1992b; Box et al., 1978; Kacker, 1985.)
Ideally the development of products robust to uncontrollable variables
should be performed during prototyping and preparation for manufacture in
order to prevent costly changes later on in the process. During prototyping
Downloaded by New York University At 11:20 09 February 2016 (PT)

several different prototypes should be evaluated for robustness, not only one
single prototype, so the prototype which is most reliable and robust for further
development can be chosen.
The following example illustrates the use of robust design in the preparation
for manufacture (see Figure 1). A manufacturer of food pellets for fish wanted
to study how robust the pellet quality was to variation in the raw material
(carbohydrate type and degree of grinding) at different processing conditions
(Ellekjær et al., 1996). An experiment was performed with three controllable
process variables: feeder rate (F), temperature (T) and percent water (W), and
two uncontrollable (“noise”) variables: carbohydrate type (C) and degree of
grinding (G). Pellet density was used as a measure of the quality of the pellet.
The experimental design and the pellet density for the different experimental
conditions are presented in Table V and Figure 8. The goal was to produce a
pellet with a pellet density higher than 0.86.
By simple visual inspection of the results in Table V we can see the pellet
quality is robust to variation in raw material (carbohydrate type and degree of
grinding), but varies according to the processing variables used. In Figure 8 the
average pellet quality for the different processing combinations is presented.
We can see the pellet density increased with increasing amount of water. We can
also see the effect of feeder seemed to depend on the water level used; feeder
seemed to have greater effect on pellet density at high water level than at low.
Pellets produced with high water levels and high feeder rates resulted in pellets
of the wanted quality characteristics (sample 3 and 8). The effect of temperature
seemed also to depend on the water level used. Temperature had no effect at low
water levels, whereas at high water levels the pellet density decreased with
increasing temperature. However, the temperature level used did not seem to be
critical as long as high levels for water and feeder rates were used. At these
settings a pellet of preferred density was produced using either of the
temperature levels.
These results showed using high water levels and high feeder rates could
produce a pellet of desired quality. The manufacturer can, however, choose the
temperature that is most economical. All three carbohydrates can be used as
 Carb. type: 1 1 2 2 3 3
The use of
Grinding: – + – + – + experimental
Recipe Feeder Temp. Water AVG STD design
1 – – – 0.76 0.77 0.78 0.77 0.01
2 – + + 0.86 0.84 0.83 0.85 0.02
3 + – + 0.92 0.91 0.92 0.92 0.00 271
4 + + – 0.77 0.79 0.79 0.78 0.01
5 – – + 0.86 0.85 0.88 0.86 0.02
6 – + – 0.77 0.77 0.77 0.77 0.00
7 + – – 0.78 0.78 0.79 0.78 0.00 Table V.
8 + + + 0.88 0.86 0.87 0.87 0.01 Design for both the
AVG 0.83 0.83 0.83 0.82 0.83 0.83 control variables [feeder
STD 0.08 0.06 0.06 0.05 0.06 0.06
Downloaded by New York University At 11:20 09 February 2016 (PT)

(F), temperature (T)

Note: and percent water (W)]
The design was planned as a 24–1 fractional factorial for each carbohydrate type and was and “noise” variables
performed as a 23–1 (2 × 3) split-plot design with G = FTW as the plot splitting factor. The [carbohydrate type (C)
pellet density is given for the different carbohydrate type and grinding combinations; in and degree of grinding
addition the average and standard deviation for pellet density for the different processing (G)] for the different
combinations is presented pellets

0.85 0.87

0.77 0.78 Figure 8.

+ Graphical presentation
of the experimental
design used in the fish
feed example, and the
Temp.
average pellet density
0.86 0.92 for the different
+ combinations of the
–
three control factors;
0.77 0.78 – Water feeder, temperature and
water
– Feeder +

raw materials or the manufacturer can select to choose the one that is cheapest.
The pellet quality seemed also to be insensitive to the degree of grinding, and
thus this factor need not be tightly controlled in production.
An important early application area for robust design was for the
development of cake mixes (Box and Jones, 1992). Robust product design ideas
were used in the development of cake mixes that taste reasonably well even if
the customers’ preparation and baking differ somewhat from the recommended
procedure (Box and Jones, 1992; Joglekar and May, 1990). Another example
comes from Fuji-Xerox who used robust design techniques under the
 IJQS development of their 3500 best selling copier. To make a machine that would
3,3 produce good copies of almost any size at a high rate, regardless of the paper
thickness and varying operating conditions used in different parts of the world,
they first collected information on potential operating (temperature, humidity,
dust, etc.) and market (size and shape of paper) conditions. This information
was used to design several prototypes which resulted in the development of a
272 copier that was robust to changes in the environment. An additional benefit of
this approach was it helped prevent many design changes and thus cut the
development time from an anticipated five years to two years (Zangwill, 1993).

Conclusions
The five main benefits of the use of experimental design in product
development are:
Downloaded by New York University At 11:20 09 February 2016 (PT)

(1) Simultaneous optimization of several factors: experimental design can

greatly increase the efficiency of experimentation and troubleshooting
that takes place during the development of any product. The use of
screening designs can reduce the number of runs and thereby decrease
the experimental cost, but it also ensures both rapid data collection,
reveal interaction effects and a wider inductive basis for decision
making. This was clearly shown in the emulgator example.
(2) Simultaneous cost reduction and quality improvement: It is possible to
select combinations of processing variables or design parameters that
will not only improve the quality of the product, but also reduce the cost.
Even a result that shows that a given factor has no influence on quality
is useful as it allows the product developers to choose the option that cost
the least. An example of this was the manufacture of fish feed where the
cheapest raw material among three alternatives could be chosen as they
resulted in pellets of similar quality.
(3) The product can be made robust to variation in uncontrollable (noise)
variables: The difference between success and failure for a product is
often the way the product development team deals with variables they
cannot control. One of the major strengths of robust design is its ability
to deal with uncontrollable (noise) variables such that the quality of the
product is reasonably good although the noise variables are varying.
(4) Provide a systematic approach for problem solving during the
development process: product developers do not easily get lost in
identifying variables with large effect on the quality of the end-product
and determining the optimum conditions for those variables. The
currency paper example gave an illustration of this.
(5) Different software has been developed for planning and analyzing
experiments using experimental design techniques; Guideline®+ and
Unscrambler® (www.camo.no), Minitab® (www.mintab.com), Design-
expert® (www.statease.com) and JMP (www.JMPdiscovery.com) are
examples.
 References The use of
Barrios, E. (1994), “A statistical design”, Plasma Manufacturing Bulletin, Vol. 6 No 2, pp. 7-8. experimental
Bisgaard, S. (1992a), “A conceptual framework for the use of quality concepts and statistical
methods in product design”, Journal of Engineering Design, Vol. 3 No 1, pp. 31-47.
design
Bisgaard, S. (1992b), “The design and analysis of 2 k–p × 2 q–r inner and outer array
experiments”, CQPI-report No. 90, Center for Quality and Productivity Improvement,
Madison, WI. 273
Box, G. and Jones, S.P. (1992), “Split-plot designs for robust product experimentation”, Journal of
Applied Statistics, Vol. 19 No. 1, pp. 3-26.
Box, G.E.P., Hunter, W.G. and Hunter, J.S. (1978), Statistics for Experimenters, John Wiley & Sons,
New York, NY.
Cooper, R.G. (1993), Winning at New Products. Accelerating the Process from Idea to Launch,
Addison-Wesley Publishing Company, Reading, MA.
Downloaded by New York University At 11:20 09 February 2016 (PT)

Cross, N. (1989), Engineering Design Methods, John Wiley & Sons Ltd, New York, NY.
Deming, E. (1986), Out of the Crisis, Massachusetts Institute of Technology, Center for Advanced
Engineering Study, Cambridge, MA.
Denes, F., Hua, Z.Q., Barrios, E., Evans, J. and Young, R.A. (1995), “Influence of RF-cold plasma
treatment on the surface properties of paper”, Journal of Macromolecular Science – Pure and
Applied Chemistry, submitted.
Ellekjær, M.R., Fuller, H.T. and Ladstein, K. (1997), “Analysis of unreplicated split plot
experiments with multiple responses”, Quality Engineering.
Ellekjær, M.R., Ilseng, M.A. and Næs, T. (1996), “A case study of the use of experimental design
and multivariate analysis in product improvement”, Food Quality and Preference, Vol. 7 No. 1,
pp. 29-36.
Erhorn, C. and Stark, J. (1994), Competing by Design, Oliver Wight Publications Inc., Essex
Junction.
Fisher, R.A. (1935), Design of Experiments, Oliver and Boyd, Edinburgh.
Joglekar, A.M. and May, A.T. (1990), “Accelerating product development by computer-aided
design of experiments”, Cereal Foods World, Vol. 35 No. 12, pp. 1187-200.
Juran, J.M. (1988), Juran’ Quality Control Handbook, 4th ed., McGraw-Hill Company, Inc.
Kacker, R.N. (1985), “Off-line quality control, parameter design and the Taguchi method”, Journal
of Quality Technology, Vol. 17, pp. 176-209.
Michaels, S.E. (1964), “The usefulness of experimental designs”, Applied Statistics, Vol. 13, p. 221.
Mørup, M. (1993), Design for Quality, Institute for Engineering Design, Technical University of
Denmark, Lyngby, IK publication 93.134 A.
Page, A.L. (1993), “Assessing new product development practices and performances: establishing
crucial norms”, J. Prod. Innov. Manag., Vol. 10, pp. 273-90.
Pahl, G. and Beitz, W. (1984), Engineering Design, London Design Council.
Schönkopf, S., Næs, T., Baardseth, P. and Ellekjær, M.R. (1996), “Computer-aided product
development in the food industry”, Food Technology, March, pp, 69-75.
Taguchi, G. (1987), System of Experimental Design: Engineering Methods to Optimize Quality and
Minimize Cost, UNIPUB/Kraus International, White Plains, New York, NY.
Urban, G.L. and Hauser, J.R. (1993), Design and Marketing of New Products, Prentice-Hall,
Englewood Cliffs, NJ.
Wesner, J.W., Hiatt, J.M. and Trimble, D.C. (1994), Winning with Qual ity. Applying
Qual ity Principles in Product Development, Addison-Wesley Publishing Company,
Reading, MA.
 IJQS Wheelright, S.C. (1988), “Product development and manufacturing start-up”, in Tushman, M.L.
and Moore, W.L. (Eds), Readings in the Management of Innovation, HarperBusiness.
3,3 Wheelright, S.C. and Clark, K.B. (1992), Revolutionizing Product Development, The Free Press,
New York, NY.
Zangwill, W.I. (1993), Lightning Strategies for Innovation, Lexington Books, New York, NY.

274 (Marit Risberg Ellekjær is a Senior Researcher at MATFORSK, Norwegian Food Research
Institute.
Søren Bisgaard is the Director of the Center for Quality and Productivity Improvement and a
Professor in the Department of Industrial Engineering at the University of Wisconsin-Madison.)
Downloaded by New York University At 11:20 09 February 2016 (PT)
 This article has been cited by:

1. Vinayak Kalluri, Rambabu Kodali. 2014. Analysis of new product development research: 1998-2009. Benchmarking: An
International Journal 21:4, 527-618. [Abstract] [Full Text] [PDF]
2. Jan Hron, Tomáš Macák. 2013. Application of design of experiments to welding process of food packaging. Acta Universitatis
Agriculturae et Silviculturae Mendelianae Brunensis 61, 909-915. [CrossRef]
3. Worlaluck Satianrangsarith. 2012. Design of Experiments Approach for Improving Wire Bonding Quality. International
Journal of Innovation, Management and Technology . [CrossRef]
4. Basel Younes, Alex Fotheringham. 2011. Factorial optimization of the effects of melt-spinning conditions on biodegradable as-
spun aliphatic-aromatic copolyester fibers. III. Diameter, tensile properties, and thermal shrinkage. Journal of Applied Polymer
Science 122:10.1002/app.v122.2, 1434-1449. [CrossRef]
5. B. Younes, A. Fotheringham, R. Mather. 2011. Factorial Optimisation of the Effects of Melt Spinning Conditions on
Biodegradable As-spun Aliphatic-Aromatic Co-Polyester Fibres. International Polymer Processing 26, 150-163. [CrossRef]
6. Basel Younes, Alex Fotheringham, Hassan M. EL-Dessouky, Ghassan Haddad. 2011. Factorial Optimization of the Effects
of Melt-Spinning Conditions on As-spun Aliphatic-Aromatic Copolyester Fibers I. Spin Draw Ratio, Overall Orientation
and Drawability. International Journal of Polymeric Materials 60, 316-339. [CrossRef]
7. Antonio Lanzotti, Pietro Tarantino. 2008. Kansei engineering approach for total quality design and continuous innovation.
The TQM Journal 20:4, 324-337. [Abstract] [Full Text] [PDF]
8. Hefin Rowlands, Jiju Antony. 2003. Application of design of experiments to a spot welding process. Assembly Automation
23:3, 273-279. [Abstract] [Full Text] [PDF]
Downloaded by New York University At 11:20 09 February 2016 (PT)

9. Daniela Butan, Emma O’Brien, Mark Southern, Seamus CliffordA Novel Practical Triangular Approach to Process Innovation
755-770. [CrossRef]