Documente Academic
Documente Profesional
Documente Cultură
Dr. Li Zou
1/27
A classic experiment from Ronald Fisher (1935):Lady
tasting tea
(Design) A lady declares that by tasting a cup of tea made with milk she
can discriminate whether the milk or the tea infusion was first added to
the cup. We will consider the problem of designing an experiment by
means of which this assertion can be tested. [. . . ] [It] consists in mixing
eight cups of tea, four in one way and four in the other, and presenting
them to the subject for judgment in a random order. The subject has
been told in advance of that the test will consist, namely, that she will be
asked to taste eight cups, that these shall be four of each kind [. . . ]. —
Fisher, 1935.
2/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 2 / 27
The lady in question eventually answered correctly six out of the eight
trials. The next question is that how do we analyze the data now? Before
doing so, lets summarize the results in the following contingency table.
3/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 3 / 27
(Analysis) Since the lady knew in advance these shall be four of each kind,
each cup is not independent. We cannot use Binomial test. Given the fixed
margins, we can use Fisher’s exact test in this case. Based on Fisher’s
approach, we only need to focus on the outcome of the first cell which has
a hypergeometric distribution and know how extreme it is e.g., P(X ≥ a).
Thus, if the lady is not able to discriminate whether tea or milk was poured
first (the null model), the chance of observing a result at least as favorable
towards her claim would be about 24% (p-value). If you consider 5% as an
informal rule to disprove the null hypothesis, the test is not significant.
4/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 4 / 27
An interesting takehome experiment
Some people including me claim that they can tell the difference between
Coke and Pepsi. How would you conduct an experiment to support his/her
claim?
5/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 5 / 27
Examples of Observational Studies
Remark: Most data analyses you have performed so far perhaps come
from observational studies.
6/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 6 / 27
On effects of 2,4-D using case-control study
A study in 1994 examined the risk of cancer in dogs that are exposed to
the herbicide 2,4-dichlorophenoxyacetic acid (2,4-D). The study involved
491 dogs that had developed cancer and 945 dogs as a control. Of these
two groups, researchers identified which dogs had been exposed to 2,4-D
in their owner’s yard (no control over assignments of 2,4-D treatment).
Cancer No Cancer
Treatment: 2,4-D 191 304
Control: no 2,4-D 300 641
7/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 7 / 27
This is an example of observational studies.
Investigators collected the data by searching databases at animal
hospitals. Thus, they had no control of assigning the treatment,
herbicide 2,4-D acid, to experiment units e.g., dogs.
The crude cancer rates among two groups are 0.39 for the treatment
group and 0.31 for the control group. A quick application of
one-sided Fisher’s exact test can be performed to compare the
equality of two proportions.
data = rbind(c(191, 304), c(300, 641))
fisher .test(data, alternative = ”greater ”)$”p.value”
A p-value of 0.00659 (< 0.05) indicates that the test rejects the null
hypothesis that two proportions are equal in favor of the alternative
hypothesis that the cancer rate in the treatment group is higher than
that in the control group. Thus, data supports that there is significant
association between exposure (2,4-D) and disease (Cancer).
8/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 8 / 27
Here are some restrictions of observational studies.
1 Since researchers do not have control over assigning treatments to
experimental units, the sample may be biased. For example, most of
the dogs with cancer were also from air polluted areas (dissimilarities
between treatment and control groups).
2 Hidden confounders are another problem in observational studies: a
confounder (also confounding variable) is a variable that influences
both the dependent variable and independent variable causing a
spurious association. E.g., when people study the associate between
obesity and cardiovascular disease, age is a common confounder to be
considered in the studies.
3 Even after adjusting a number of confounding variables, investigators
can only conclude an association between treatments and diseases.
Remarks: Randomized controlled experiments can avoid the above three
restrictions and conclude a causal effect.
9/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 9 / 27
After all, observational studies are a powerful tool. Probably the most
famous example of observational studies is related to the discovery of the
strong association between smoking and diseases (e.g.,lung cancer). It is
unethical to make healthy people smoke cigarettes in an experiment due to
carcinogens in cigarettes. Nobody is going to smoke for twenty years just
to please a statistician. Thus these kind of studies are necessarily
observational.
10/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 10 / 27
What does an experiment usually mean?
11/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 11 / 27
An experiment applies treatments to experimental units using a chance
mechanism and measure responses. Investigators use the responses to
learn about the treatments. In this book, we focus on comparative
experiments which compare different treatments.
We want to learn the effects of treatments. (e.g., for the first case, the
treatments are three actions used by the police)
Responses are measured and tell use how the treatment worked. (e.g., the
response could be the length of time until recurrence of assault)
12/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 12 / 27
What makes an experiment special is control. The experimenter gets to
control the assignment of treatments to units. (Control has a twofold
meaning in DOE: 1. Assignments of treatments to experimental units are
controlled; 2. A control treatment is the one that is used as basis of
comparison for other treatments.)
13/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 13 / 27
An example of using controls
Infections following surgery are a serious concern that can have a major
impact on a patient’s road to recovery. One approach to counter infection
is to kill surgical pathogens by oxidation. In one study (Greif et al.
(2000)), researchers randomly assigned 250 patients to receive 30%
inspired oxygen and 250 patients to receive 80% inspired oxygen. All
patients were undergoing surgery for colorectal resection. Of the patients
receiving 30% inspired oxygen, 28 had a surgical wound infection compared
with 13 patients who received the 80% inspired oxygen treatment.
14/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 14 / 27
Q1: Yes. We can compare the treatment group (80%) with the control
group (30%) to see the effect. In R, we can do
prop.test(c(28, 13), c(250, 250))
Q2: No. Since there was no control group- a group that received no
inspired oxygen-in this study. We do not know if 30% inspired oxygen is
better than no treatment at all.
15/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 15 / 27
Causal inferences
A lot of efforts are made in designing experiments comparing with
observational studies, what do we gain from designed experiments?
16/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 16 / 27
Considerations of a good design
17/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 17 / 27
Avoid systematic error
18/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 18 / 27
For example, a company is evaluating two different word processing
packages (A and B) for use by its clerical staff. The goal is to see how
quickly a test document can be entered correctly using two programs.
Suppose that 20 test secretaries entered the document twice using each
program once in the order A first and B second.
Based on the design, does the difference in time processing the documents
depend on the effect of word programs A and B?
19/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 19 / 27
Solution: In this design we do not know if any observed differences are due
to treatment (programs) effects or the fact that the secretary will be
familiar with the document and thus enter it faster for the second time.
20/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 20 / 27
Be precise
Experiments are precise when random error in the treatment comparisons
is small.
22/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 22 / 27
Have broad validity
Our experimental units should reflect the population about which we wish
to draw inference. If the units are actually a statistical sample from some
population of units, then the conclusions are also valid for the population.
Beyond this, we are extrapolating, and the extrapolation might or might
not be successful.
23/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 23 / 27
Some concepts
Treatments The different procedures we want to compare.
26/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 26 / 27
Treatments: Four types of protective coatings.
Replication: There are five frying pans (replications) for each treatment.
Experimental unit: Frying pan, because coatings (treatments) are
randomly assigned to the frying pans.
Measurement unit: Particular locations on the frying pan.
Total number of measurements: 4 ∗ 5 ∗ 3 = 60 measurements in this
experiment.
The experimental unit is the frying pan since the unit was randomly
assigned to a coating. The measurement unit is a location on the frying
pan
Later on, you will recognize that this experimental design is called a
completely randomized design.
27/27
Dr. Li Zou (csueb) Introduction to Design and Analysis of Experiments Jan 21, 2020 27 / 27