Quick Guide to the ERP Sample Planner

The Sample Planner is a tool to help you evaluate different approaches for conducting
the random inspections in the Environmental Results Program (ERP). Please fully read
this sheet before using the Sample Planner for the first time.

This Quick Guide Sheet Contains:

an overview of the tools within the Sample Planner,
instructions about assumptions and use, and
suggestions for other resources.

Overview of Spreadsheet Tools in the Sample Planner

The worksheets in the Sample Planner let you evaluate your approach for conducting
inspections two different ways:
you can calculate the sample size needed to meet specified data quality requirements
for a sample, or
you can evaluate the quality of your estimates based on the number of inspections
you can conduct. Each is described in the following table. (Click on the spreadsheet
name to go to that sheet.)
Spreadsheet Name

Find

Find

Find

Find

Description
Calculate Number of Inspections Needed
Calculates the number of inspections needed to
Sample Size (1-Sample)
achieve a specified margin of error and
confidence level for a single sample.
Calculates the number of inspections needed to
achieve a specified margin of error and
confidence level for the difference between
results between
Sample Size (2-Sample)
two rounds of inspections. Use this when your
most important evaluation will be of the
differences between the two rounds of
inspections.
of inspections.
Calculate Margin of Error for One or Two rounds of Inspections
Calculates the margin of error associated
Margin of Error (1-Sample)
with a specified number of inspections in
a single sample.
Calculates the margin of error associated with the
difference in
results observed between two rounds of
Margin of Error (2-Sample)
inspections of given sizes. Use this when your
most important evaluation will be of the
differences between two samples.

Instructions
Color scheme. Each sheet has a user-friendly color scheme that allows you to readily
navigate the sheet:
Blue cells provide text information for you.
Yellow cells are where you enter data.

Quick Guide to the ERP Sample Planner

Green cells provide results.
Gray cells contain assumptions and calculations, for advanced users.

Quick Guide to the ERP Sample Planner

Terminology and definitions. Statistics often uses specialized terminology. As much
as possible, this tool presents plain language along with specialized terms and
calculations. There is also a glossary on the second tab of this workbook.
Instructional comments. Be sure to read the additional information about the inputs and
results, provided in comments in the cells. Cells with comments have a small red flag
in the upper right-hand corner. Place the cursor over the cell to see the comment.
Protected cells. All the cells in this workbook are protected, except for the yellow ones
where you enter data. Consequently, you do not need to worry about accidentally
changing the formulas. If you do want to modify protected cells, please do so with caution.
You can unprotect a cell or worksheet under the Tools menu. There is no password.
Planning your sample to estimate proportions and means. Note that this tool
assumes you want to estimate a proportion. It does not let you design a sample with the
goal of estimating the mean and standard error of a continuous variable (e.g., the average
quantity of hazardous waste generated by facilities). Typically, ERP planners do not have
the necessary information in advance to undertake such designs anyway. You will still be
able to measure continuous variables in your ERP, and the ERP Results Analyzer allows
you to calculate results for continuous variables. However, you probably will not have an
advance idea of the uncertainty that will be associated with these estimates.
Ease of viewing. Use of this tool is optimized for an average desktop monitor and
Microsoft Excel setup. Some users, including those with notebook computers, may find
the tool easier to use if they change the "zoom" to less than 100% -- e.g., to show more
of the comment boxes without scrolling. Choosing the "full-screen" option will also enable
seeing more of the tool. Both of these options are accessible under the "View" menu in
the Microsoft Excel toolbar.

Other Resources
For more information on the use of statistics in ERP, please refer to the Generic Guide to
Statistical Aspects of Developing an Environmental Results Program. Please note that
the formulas used in this spreadsheet are different than (and improved from) those in the
April 25, 2003, version of the Generic Guide.
The spreadsheet ResultsAnalyzer2006.xls will help you conduct statistical tests and
calculate confidence intervals based on the results of your inspections.

Definitions of Key Terms

Population (N) - This is the total set of facilities in your universe. You may want to
determine the proportion of this group of facilities that is in compliance with a certain
regulation or makes use of a certain type of treatment technology.

Sample Size (n) - The sample size refers to the number of random inspections. A
sample that consists of the entire population is called a census.

Point Estimate of the Proportion (p)

facilities in the sample that meet your criteria (e.g., are in compliance).

Confidence Interval and Margin of Error (e)

confidence interval associated with the random sample are computed to reflect the
uncertainty associated with your point estimate of the actual population proportion or
mean.
The confidence interval gives a range of values that is believed to contain the actual
population proportion or mean with the confidence level
below). There are several different acceptable ways of constructing confidence intervals.
The standard Wald confidence interval is symmetric about the point estimate. The
interval (used in this tool for single-sample estimations) is not symmetrical except for a
point estimate of 50%, but it is often shorter (i.e., more precise). It is particularly useful
for small sample sizes and for estimating single proportions, especially when they are
outside the 30-70 percent range. For more information on this topic, please see the
journal article by Agresti and Coull, "Approximate is Better than 'Exact' for Interval
Estimation of Binomial Proportions," in The American Statistician
The margin of error is half the width of the standard
requires the user to select a confidence level. The
adding and subtracting the margin of error to the point estimate. For example, if the
margin of error of an estimated proportion is 5 percentage points, the confidence interval
will range from 5 percentage points less than the observed value to 5 percentage points
more than the observed value. In this example, if the observed proportion is 50 percent,
the confidence interval would be 45 to 55 percent. Notice that the length of the
confidence interval is twice the margin of error.
The length of the Score confidence interval is also twice the margin of error when the
observed proportion is 50% (but not for any other observation). When calculating sample
sizes required for a particular margin of error, the Sample Planner assumes a 50%
observed proportion, in order to provide the maximum sample size necessary as well as to
provide a symmetrical Score confidence interval that is twice the margin of error.
Users interested in viewing the Wald confidence interval can unprotect the spreadsheets
(see Instructions) and "unhide" hidden rows in the "Results" section of the "Find Margin of
Error (1-Sample)" worksheet in the Sample Planner. In the Results Analyzer's "Proportion
(1-Sample)" worksheet, the Wald confidence interval is shown in the "For Advanced
Users" section.

Confidence Level - The confidence level is the percentage of all possible random
samples of size n whose corresponding confidence intervals contain the actual (unknown)
population proportion or mean. (See the definition of Confidence Interval above.) For
instance, suppose the true proportion of all facilities in compliance with a particular
regulation is 62 percent. (In practice, you would only know this if all facilities were in the
sample.) A 95-percent confidence level means that approximately
samples of n facilities from the population will produce a confidence interval that includes
62% (in a confidence interval, the lower limit will be less than 62% and the upper limit will
be greater than 62%). Since you are selecting just
corresponding confidence interval may or may not contain the true proportion, but the fact
that 95% of all possible samples would contain the true proportion is expressed by
saying you have 95% confidence that your particular interval is one of these accurate
samples. You do not know for sure that this is the case, however.
To determine the necessary sample size, the user must specify the desired confidence
level and the desired margin of error.

Comparing Two Proportions - A margin of error or confidence interval can be

developed for a difference in proportions or means, as well. Because uncertainty is
associated with each of the observations, the difference between the observations also
has uncertainty associated with it. This uncertainty can be described with a margin of
error or confidence interval. For instance, if you observe 40 percent compliance in Round
1 of inspections and 60 percent compliance in Round 2 of inspections, you have observed
a difference of 20 percentage points. If you calculated a margin of error of 10 percentage
points for this example, you could say that you believe performance improved by 20
percentage points, 10 percentage points, or somewhere between 10 and 30 percentage
points.
Note that if the confidence interval contains zero, you can not be confident that any
improvement or change occurred. For instance, if you observed a difference of 20
percentage points 25 percentage points, the confidence interval of the difference would
be from - 5 to 45 percentage points. This interval includes zero, so you could not be
confident that there was a change.

total set of facilities in your universe. You may want to

s group of facilities that is in compliance with a certain
rtain type of treatment technology.

ple size refers to the number of random inspections. A

re population is called a census.

roportion (p) - This usually refers to the proportion of

et your criteria (e.g., are in compliance).

d Margin of Error (e) - The margin of error and a

with the random sample are computed to reflect the

r point estimate of the actual population proportion or

s a range of values that is believed to contain the actual

with the confidence level prescribed by the user (see
ent acceptable ways of constructing confidence intervals.
nterval is symmetric about the point estimate. The Score
gle-sample estimations) is not symmetrical except for a
often shorter (i.e., more precise). It is particularly useful
stimating single proportions, especially when they are
e. For more information on this topic, please see the
oull, "Approximate is Better than 'Exact' for Interval
ons," in The American Statistician, 1998.

e width of the standard confidence interval. It likewise

nfidence level. The Wald confidence interval is formed by
gin of error to the point estimate. For example, if the
proportion is 5 percentage points, the confidence interval
oints less than the observed value to 5 percentage points
In this example, if the observed proportion is 50 percent,
e 45 to 55 percent. Notice that the length of the Wald
margin of error.

nce interval is also twice the margin of error when the

t not for any other observation). When calculating sample
argin of error, the Sample Planner assumes a 50%
provide the maximum sample size necessary as well as to
nfidence interval that is twice the margin of error.

Wald confidence interval can unprotect the spreadsheets

hidden rows in the "Results" section of the "Find Margin of
the Sample Planner. In the Results Analyzer's "Proportion
d confidence interval is shown in the "For Advanced

nfidence level is the percentage of all possible random

ponding confidence intervals contain the actual (unknown)
(See the definition of Confidence Interval above.) For
ortion of all facilities in compliance with a particular
ctice, you would only know this if all facilities were in the
nce level means that approximately 95% of all random
population will produce a confidence interval that includes
he lower limit will be less than 62% and the upper limit will
u are selecting just one random sample, the
val may or may not contain the true proportion, but the fact
s would contain the true proportion is expressed by
ce that your particular interval is one of these accurate
ure that this is the case, however.

mple size, the user must specify the desired confidence

ions - A margin of error or confidence interval can be

oportions or means, as well. Because uncertainty is
ervations, the difference between the observations also
it. This uncertainty can be described with a margin of
r instance, if you observe 40 percent compliance in Round
compliance in Round 2 of inspections, you have observed
oints. If you calculated a margin of error of 10 percentage
uld say that you believe performance improved by 20
age points, or somewhere between 10 and 30 percentage

rval contains zero, you can not be confident that any

ed. For instance, if you observed a difference of 20
age points, the confidence interval of the difference would
ints. This interval includes zero, so you could not be

Estimate the Sample Size Required for One Round of Inspections

Go Back to Instructions / Main Menu

To Definitions of Key Terms

When to use this spreadsheet: When you want to find out how many inspections you should conduct in a round of random inspections, based
on the margin of error and confidence level you are seeking.
Enter your data in the yellow cells
The results are shown in green cells. These cells are locked.
Assumptions and calculations are shown in gray cells. These cells are locked, too.
Input/
Results

Information needed

Statistical terminology

Enter Your Information

What confidence level do you want to have?

Confidence level

95%

How many facilities is your ERP focusing on?

(Estimate if you do not know.)

1,000

Population or N

What margin of error do you want, above and below the estimate of the
proportion of facilities that have the characteristic you are measuring (e.g., are in
compliance)?

5.0%

Margin of error or
Half-width of confidence interval

277

Sample required (finite population

corrected)

Results
Number of inspections required

For Advanced Users: Assumptions and Calculations

Inspections required with an infinite population

382

Sample required for Score interval

Assumed proportion, p

0.50

Assumed proportion, p

p(1-p)

0.25

p(1-p)

Confidence level

95%

Confidence level, (1-)

Factor to achieve level of confidence required

1.960

Value of a standard normal, Z(1-/2)

Reference:
Derived from: Agresti and Coull. 1998. "Approximate is Better than 'Exact' for Interval Estimation of Binomial
Proportions." The American Statistician, v. 52, no. 2, 119-126.

Estimate Sample Sizes Required for a Specified Margin of Error for a

Difference in Proportions (Two Samples)
Go Back to Instructions / Main Menu

To Definitions of Key Terms

When to use this spreadsheet: When you want to find out how many inspections you should conduct in each round of ERP inspections, based
on the margin of error and confidence level you wish to achieve for a difference between proportions observed in two rounds of inspections.
Use this when your most important evaluation will be of the differences between two samples.
Enter your data in the yellow cells
The results are shown in green cells. These cells are locked.
Assumptions and calculations are shown in gray cells. These cells are locked, too.
Input/
Results

Information needed

Statistical terminology

Enter Your Information

What confidence level do you want to have?

Confidence level

95%

How many facilities is your ERP focusing on in Round 1?

(Estimate if you do not know.)
If it differs from Round 1, how many facilities is your ERP focusing on in
Round 2? (Estimate if you do not know.)
What is your desired margin of error? (+/-)
(You may not be able to detect differences smaller than this amount.)

1,000

Population or N1

1,000

Population or N2

5.0%

Margin of error or
Half-width of confidence interval

Number of inspections required for Round 1.

435

Sample size, n1

Number of inspections required for Round 2.

435

Sample size, n2

Results

For Advanced Users: Assumptions and Calculations

Confidence level
Factor to achieve level of confidence required
Number of inspections, before finite population adjustment

95%
1.960
768

(1-)
Value of standard normal, Z(1-/2)
m

Reference:
Derived from: Kish, Leslie. 1965. Survey Sampling. John Wiley & Sons, Inc. New York, NY. p. 41.

Estimate the Margin of Error for One Round of Inspections

Go Back to Instructions / Main Menu

To Definitions of Key Terms

When to use this spreadsheet: When you know the maximum number of inspections you can conduct in a round of random inspections, and
you want to find out the margin of error you will have.
Enter your data in the yellow cells
The results are shown in green cells. These cells are locked.
Assumptions and calculations are shown in gray cells. These cells are locked, too.
Input/
Results

Information needed

Statistical terminology

Enter Your Information

What confidence level do you want to have?

Confidence level

95%

How many facilities is your ERP focusing on?

(Estimate if you do not know.)
How many inspections will you conduct in a single round?

1,000

Population or N

100

Sample size or n

Results
Margin of error (+/-)

9.1%

Half-width of the Score confidence interval

For Advanced Users: Assumptions and Calculations

Confidence level
Factor to achieve level of confidence required

95%
1.960

(1-)
Value of standard normal, Z(1-/2)

Assumed proportion, p

0.50

Assumed proportion, p

p(1-p)

0.25

p(1-p)

Reference:
Agresti and Coull. 1998. "Approximate is Better than 'Exact' for Interval Estimation of Binomial Proportions."
The American Statistician, v. 52, no. 2, 119-126.

Estimate the Margin of Error for Differences in Proportions (Two Samples)

Go Back to Instructions / Main Menu

To Definitions of Key Terms

When to use this spreadsheet: When you know the maximum number of inspections you can conduct in each round of ERP, and want to know
the maximum margin of error to expect for a difference between proportions observed in two different rounds of inspections. Use this when
your most important evaluation will be of the differences between two samples.
Enter your data in the yellow cells
The results are shown in green cells. These cells are locked.
Assumptions and calculations are shown in gray cells. These cells are locked, too.
Input/
Results

Information needed

Statistical terminology

Enter Your Information

What confidence level do you want to have?

Confidence level

95%

How many facilities is your ERP focusing on in Round 1?

(Estimate if you do not know.)
If it differs from Round 1, how many facilities is your ERP focusing on in
Round 2? (Estimate if you do not know.)

1,000

Population or N1

1,000

Population or N2

How many inspections will you conduct in Round 1?

100

Sample size or n1

If it differs from Round 1, how many inspections will you conduct in Round 2?

100

Sample size or n2

Results
Maximum margin of error for a difference in proportions (+/-)

13.2%

Half-width of the confidence interval for a

difference in proportions

For Advanced Users: Assumptions and Calculations

Confidence level

95%

Factor to achieve level of confidence required

1.960

SE (p1-p2)

6.7%

(1-)
Value of standard normal, Z(1-/2)

Reference:
Derived from: Kish, Leslie. 1965. Survey Sampling. John Wiley & Sons, Inc. New York, NY. p. 41.