STT

Introduction 1
Method validation
Accuracy = Trueness & Precision

This file addresses the basic statistical concepts for the validation of method accuracy; accuracy is understo
trueness and precision (see [1], Terms Worksheet).

Precision
Precision (and imprecision) are qualitative concepts (see Terminology Sheet).
Experimental estimates of imprecision are the Variance or Standard Deviation of a distribution of measure
EXCEL functions: VAR; STDEV; ANOVA.
Calculations: Relative Standard Deviation in % or Coefficient of Variation (R.S.D., or CV),
100 * (STDEV/AVERAGE)

Trueness
Trueness is a qualitative concept, too. Trueness is related to the concept of systematic error.
The experimental estimate of trueness is the [instrumental] Bias, which is the "average of replicate indicatio
reference quantity value".
EXCEL function: AVERAGE.
Calculations: Difference, or %-difference.

Accuracy
As the above, accuracy is a qualitative concept.
It's experimental estimate is the Measurement Error (Error), which is the "measured quantity value minus a
value". Note: often called Total Error.
Calculations: Difference, or %-difference.

Disclaimer

EXCEL Settings
Activate the Add-Ins (Tools>Add-Ins):
Analysis ToolPak & Analysis ToolPak -VBA.

hod accuracy; accuracy is understood as joint index of

of a distribution of measurement results.

(R.S.D., or CV),

of systematic error.
s the "average of replicate indications minus a

"measured quantity value minus a reference quantity

Macro security (Tools>Macro>Security):

Medium or low.

STT

Introduction 2

Method validation - The problem of finite number of measurements

Method validation can be done only with a finite number of measurements. Consequently, all experimental d
experimental ESTIMATES have statistical CONFIDENCE INTERVALS.

Confidence intervals
STDEV: involvement of the CHI-square statistics; for imprecision.
AVERAGE: involvement of the Student's t-statistics; for trueness.
CENTILE: involvement of Normal statistics and t-statistics; for total error.
Note 1: for the total error, tolerance or prediction intervals may be of additional interest.
Note 2: method comparison studies are an important tool for estimating total error. The involved statistics (fo
here.
EXCEL functions: CHIDIST; CHIINV; TDIST; TINV; NORMDIST; NORMSDIST; NORMSINV.

Significance testing
Due to the nature of validation, we have to compare the confidence intervals of our estimates with requirem
Tests involved are the "One-sample" F-test and t-test (note: the "reference quantity value" is assumed to b
EXCEL functions: EXCEL has no in-built one-sample tests, however, they can be constructed with EXCEL u

Power analysis
A method should not get validated by chance. Therefore, we need an idea i) how many measurements we n
under assumed method performance (for example, stable CV = 7%); ii) which CV can we tolerate when the
estimating the CV (duplicates on 10 different days).
Statistics involved are the Power of the F-test and the t-test.
EXCEL functions: EXCEL has no tools for power analysis, however, the power of some tests can be estima
with the free "R"-software and G*Power software.
Tolerance interval
One worksheet shortly presents the calculation of "rest bias" from tolerance interval considerations.

Validation
Confirmation, through the provision of objective evidence, that requirements for a specific intended use or a
Systems Requirements, International Organization for Standardization (ISO), Geneva, 2000).

s. Consequently, all experimental data are only estimates of true population parameters: ALL

dditional interest.
otal error. The involved statistics (for example, regression analysis) are complex and not presented

SDIST; NORMSINV.

vals of our estimates with requirements. In statistical terms, this is done with significance tests.
ce quantity value" is assumed to be error-free).
y can be constructed with EXCEL using the above functions.

a i) how many measurements we need for validating a requirement (for example, CV limit = 15%)
which CV can we tolerate when the limit is 15% and we apply a a certain experimental design for

power of some tests can be estimated with simulation experiments. Power analysis will be done

ce interval considerations.

ents for a specific intended use or application have been fulfilled (ISO 9001, Quality Management
(ISO), Geneva, 2000).

STT

Terminology
Terminology [1]
Accuracy
Closeness of agreement between a measured quantity value and a true quantity value of the measurand.

Trueness
Closeness of agreement between the average of an infinite number of replicate measured quantity values a

Precision
Closeness of agreement between indications or measured quantity values obtained by replicate measurem
Measurement error (error)
Measured quantity value minus a reference quantity value.
Note: often called Total Error.

Systematic [measurement] error

Component of measurement error that in replicate measurements remains constant or varies in a predictab
Bias [of measurement]
Estimate of a systematic measurement error.
Instrumental bias
Average of replicate indications minus a reference quantity value.

Random [measurement] error

Component of measurement error that in replicate measurements varies in an unpredictable manner.
NOTE 2 Random measurement errors of a set of replicate measurements form a distribution that can be su
zero, and its variance.

[1] ISO/IEC Guide 99:2007. International vocabulary of metrology Basic and general concepts and assoc
Standardization: Geneva, 2007 (VIM = Vocabulaire International de Mtrologie).

[1] JCGM 200:2008. International vocabulary of metrology Basic and general concepts and associated te
(BIPM); Joint Committee for Guides in Metrology (JCGM): Paris, 2008.
Freely available at: http://www.bipm.org/en/publications/guides/vim.html

quantity value of the measurand.

eplicate measured quantity values and a reference quantity value.

es obtained by replicate measurements on the same or similar objects under specified conditions.

ns constant or varies in a predictable manner.

in an unpredictable manner.
ts form a distribution that can be summarized by its expectation, which is generally assumed to be

c and general concepts and associated terms (VIM). International Organization for

eneral concepts and associated terms (VIM). International Bureau of Weights and Measures

STT

Requirements

Requirements
We will work with the requirements for bioanalytical method validation [1]:
CVmax = 15% (20% at the limit of quantitation, LoQ).
Maximum Bias (Bmax) = 15% (20% at the limit of quantitation, LoQ).
$Maximum Total error (TEmax): 1.96 x CVmax = 29.4%
Note: In document [1], TEmax is indirectly defined by the "in-study" quality control rule: 66.7% of the results
One may slightly change that to 68.3%, because this corresponds to +/-1s.

$We use the Westgard concept for calculating CV/B ratios that satisfy a TEmax of 29.4%: TE = B + 1.96
multiplier gradually from 1.96 to 1.65 when introducing bias [8]. By doing so, we keep the results outside TE
TE = B (015) + (1.961.65) s (158.76); for B = 15, s (8.760): TE = 29.415.
Note: If we would allow Bmax and CVmax happen at the same time, TEmax would be 39.75% (=15 + 1.65

1 Guidance for Industry: Bioanalytical Method Validation, US Department of Health andHumanServices, Fo

and Research (CBER), Rockville, MD, 2001.

2 Hoffman D, Kringle R. A total error approach for the validation of quantitative analytical methods. Pharm R

3 Westgard JO, de Vos DJ, Hunt MR, Quam EF, Carey RN, Garber CC. Concepts and practices in the eva
Technol. 1978;44(8):803-13.

4 Westgard JO, de Vos DJ, Hunt MR, Quam EF, Garber CC, Carey RN. Concepts and practices in the eva
acceptability. Am J Med Technol. 1978;44(7):727-42.

5 Westgard JO, de Vos DJ, Hunt MR, Quam EF, Garber CC, Carey RN. Concepts and practices in the eva
Technol. 1978;44(6):552-71.

6 Westgard JO, de Vos DJ, Hunt MR, Quam EF, Carey RN, Garber CC. Concepts and practices in the eva
procedures. Am J Med Technol. 1978;44(5):420-30.

7 Westgard JO, de Vos DJ, Hunt MR, Quam EF, Carey RN, Garber CC. Concepts and practices in the eva
approach. Am J Med Technol. 1978;44(4):290-300.

8 Stckl D, Thienpont LM. About the z-multiplier in total error calculations. Clin Chem Lab Med 2008;46(11

ty control rule: 66.7% of the results should be within +/-15% (4 results out of 6, 4-6-15 rule) [2].

TEmax of 29.4%: TE = B + 1.96 s [3 - 7]. However, we modify that concept by changing the zso, we keep the results outside TE at 5% (moving from 2-sided 5% to 1-sided 5%).

max would be 39.75% (=15 + 1.65 x 15).

nt of Health andHumanServices, Food and Drug Administration, Center for Biologics Evaluation

itative analytical methods. Pharm Res 2007;24(6):1157-64.

Concepts and practices in the evaluation of clinical chemistry methods. V. Applications. Am J Med

Concepts and practices in the evaluation of clinical chemistry methods: IV. Decisions of

Concepts and practices in the evaluation of clinical chemistry methods. III: Statistics. Am J Med

Concepts and practices in the evaluation of clinical chemistry methods. II. Experimental

Concepts and practices in the evaluation of clinical chemistry methods. I. Background and

s. Clin Chem Lab Med 2008;46(11):1648-9.

STT

Total error

30

IBiasI (%) ..

20
[8.76;15]

10

[15;0]

0
0

10

15

CV (%)

Combinations of CV & B that fulfill TE = 29.4% (TE = Bias + z * CV)

according to TE = B (015) & z (1.961.65), s (158.8).

Note: Other fields work with TE = Bias + 1 * SD (z 10.475)

The corresponding TE curves are distinctly diffferent.
1.0

Bias (fractions of TE) ..

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

CV (fractions of TE)

0.8

0.9

1.0

Bi
0.2
0.1
0.0
0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

CV (fractions of TE)

0.8

0.9

1.0

SE/RE
0.00
0.10
0.20
0.31
0.43
0.57
0.73
0.91
1.13
1.40
1.7132
2.10
2.60
3.24
4.11
5.36
7.30
10.74
18.45
51.67
#DIV/0!

Generic
SE
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
1.96

RE
1.000
0.992
0.978
0.955
0.921
0.879
0.827
0.765
0.705
0.644
0.584
0.523
0.462
0.401
0.341
0.280
0.219
0.158
0.098
0.037
0.000

15%
SE
0.0
1.5
3.0
4.5
6.0
7.5
9.0
10.5
12.0
13.5
15.0
16.5
18.0
19.5
21.0
22.5
24.0
25.5
27.0
28.5
29.4

15
RE
15
14.89
14.67
14.32
13.82
13.18
12.40
11.48
10.58
9.67
8.76

See also
A total error approach for the validation of quantitative analytical methods
Hoffman D, Kringle R. PHARMACEUTICAL RESEARCH 2007;24:1157-64.

8.756
7.844
6.932
6.021
5.109
4.198
3.286
2.375
1.463
0.552
0

STT

Power Analysis
Finding n for a given CV, or a CV for given n
CVmax = 15%
a-error consideration
One-sample F -test (1-tailed!)
Limit
15.0
CVexp
3.40
n=3
CHI2
0.10
p Value
0.0501

One-sample F -test (1-tailed!)

Limit
15.0
CVexp
9.12
n = 10
CHI2
3.33
p Value
0.0501

Limit
CVexp
CHI2
p Value

Limit
CVexp
CHI2
p Value

15.0
6.32
0.71
0.0499

n=5

15.0
10.95
10.12
0.0500

n = 20

With 3 measurements, the maximum CV would be 3.4%.

With 20 measurements, the maximum CV would be 11%.
However, if the maximum CV is present during validation,
the chance to validate the method is only 50%!
Tolerance interval (0.95, 0.95); TE limit = 29.4% (NO bias!)
2-sided (k-limit = 1.96)
n
TE
k
s
Compare
10
29.4
3.38
8.70
roughly to
a-error
20
29.4
2.75
10.69
k-values from ISO 3207-1975

s(a)
9.11
10.94

b-error consideration = Power

We want the method validated in 95% of the cases.
We investigate the Power of the 1-sided 1-sample F -test (EXCEL has no power analysis).
G*Power (http://www.psycho.uni-duesseldorf.de/aap/projects/gpower/)
Power = 0.95
Chi-square (1-sided, a = 0.05)
n
Limit
"Power"-CV
3
15
1.96
5
15
4.11
10
15
6.65
8.69
15
20
40
15
10.29

With an actual CV of 8.69% and a limit CV of 15%, we would validate the method in 95% of the cases
(see also Worksheet Power Chi2-test)

n
3
5
10
20

o power analysis).

e the method in 95% of the cases.

df
2
4
9
19

1-sample F

STT
SD limit
n
Effect-size
Var0/Var1
1.00
1.10
1.20
1.30
1.40
1.50
1.60
1.70
1.80
1.90
2.00
2.10
2.20
2.30
2.40
2.50
2.60
2.70
2.80
2.90
3.00
3.10
3.20
3.30
3.40
3.50
3.60
3.70
3.80
3.90
4.00
4.10
4.20
4.30
4.40
4.50
4.60
4.70
4.80
4.90
5.00
5.10
5.20
5.30
5.40
5.50
5.60
5.70
5.80
5.90

15
5
Chi2-test
Power
0.050
0.071
0.095
0.121
0.148
0.176
0.204
0.233
0.261
0.288
0.315
0.340
0.365
0.389
0.412
0.434
0.456
0.476
0.495
0.513
0.531
0.548
0.564
0.579
0.593
0.607
0.621
0.633
0.645
0.657
0.668
0.678
0.688
0.698
0.707
0.716
0.724
0.732
0.740
0.747
0.755
0.761
0.768
0.774
0.780
0.786
0.792
0.797
0.802
0.807

225
10

(Var0)
20

100

Power
0.050
0.081
0.119
0.162
0.209
0.257
0.306
0.354
0.401
0.446
0.489
0.528
0.566
0.600
0.632
0.661
0.688
0.713
0.736
0.756
0.775
0.793
0.809
0.823
0.836
0.849
0.860
0.870
0.879
0.888
0.896
0.903
0.910
0.916
0.921
0.927
0.931
0.936
0.940
0.944
0.947
0.950
0.953
0.956
0.959
0.961
0.963
0.966
0.967
0.969

Power
0.050
0.096
0.157
0.229
0.308
0.389
0.467
0.540
0.607
0.666
0.718
0.763
0.801
0.833
0.860
0.883
0.902
0.918
0.931
0.943
0.952
0.959
0.966
0.971
0.976
0.979
0.983
0.985
0.987
0.989
0.991
0.992
0.993
0.994
0.995
0.996
0.996
0.997
0.997
0.998
0.998
0.998
0.998
0.999
0.999
0.999
0.999
0.999
0.999
0.999

Power
0.050
0.175
0.380
0.601
0.777
0.890
0.950
0.979
0.992
0.997
0.999
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

Var 1
225
205
188
173
161
150
141
132
125
118
113
107
102
98
94
90
87
83
80
78
75
73
70
68
66
64
63
61
59
58
56
55
54
52
51
50
49
48
47
46
45
44
43
42
42
41
40
39
39
38

6.00
6.10
6.20
6.30
6.40
6.50
6.60
6.70
6.80
6.90
7.00
7.1
7.2
7.3
7.4
7.5
7.6
7.7
7.8
7.90
8.00
8.10
8.20
8.30
8.40
8.50
8.60
8.70
8.80
8.90
9.00
9.10
9.20
9.30
9.40
9.50
9.60
9.70
9.80
9.90
10.00
10.10
10.20
10.30
10.40
10.50
10.60
10.70
10.80
10.90
11.00
11.10
11.20
11.30
11.40
11.50

0.812
0.817
0.821
0.826
0.830
0.834
0.838
0.841
0.845
0.849
0.852
0.855
0.858
0.861
0.864
0.867
0.870
0.873
0.875
0.878
0.880
0.883
0.885
0.887
0.890
0.892
0.894
0.896
0.898
0.900
0.901
0.903
0.905
0.907
0.908
0.910
0.912
0.913
0.915
0.916
0.917
0.919
0.920
0.922
0.923
0.924
0.925
0.926
0.928
0.929
0.930
0.931
0.932
0.933
0.934
0.935

0.971
0.973
0.974
0.975
0.977
0.978
0.979
0.980
0.981
0.982
0.983
0.984
0.985
0.985
0.986
0.987
0.987
0.988
0.989
0.989
0.990
0.990
0.990
0.991
0.991
0.992
0.992
0.992
0.993
0.993
0.993
0.994
0.994
0.994
0.994
0.994
0.995
0.995
0.995
0.995
0.995
0.996
0.996
0.996
0.996
0.996
0.996
0.996
0.997
0.997
0.997
0.997
0.997
0.997
0.997
0.997

0.999
0.999
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

38
37
36
36
35
35
34
34
33
33
32
32
31
31
30
30
30
29
29
28
28
28
27
27
27
26
26
26
26
25
25
25
24
24
24
24
23
23
23
23
23
22
22
22
22
21
21
21
21
21
20
20
20
20
20
20

11.60
11.70
11.80
11.90
12.00
12.10
12.20
12.30
12.40
12.50
12.60
12.70
12.80
12.90
13.00
13.10
13.20
13.30
13.40
13.50
13.60
13.70
13.80
13.90
14.00
14.10
14.20
14.30
14.40
14.50
14.60
14.70
14.80
14.90
15.00

0.936
0.937
0.938
0.939
0.940
0.941
0.941
0.942
0.943
0.944
0.945
0.945
0.946
0.947
0.948
0.948
0.949
0.950
0.950
0.951
0.952
0.952
0.953
0.953
0.954
0.955
0.955
0.956
0.956
0.957
0.957
0.958
0.958
0.959
0.959

0.997
0.998
0.998
0.998
0.998
0.998
0.998
0.998
0.998
0.998
0.998
0.998
0.998
0.998
0.998
0.998
0.998
0.999
0.999
0.999
0.999
0.999
0.999
0.999
0.999
0.999
0.999
0.999
0.999
0.999
0.999
0.999
0.999
0.999
0.999

1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

19
19
19
19
19
19
18
18
18
18
18
18
18
17
17
17
17
17
17
17
17
16
16
16
16
16
16
16
16
16
15
15
15
15
15

1-sided
G*Power software (http://www.psycho.uni-duesseldorf.de/aap/projects/gpower/).

SD1
15.0
14.3
13.7
13.2
12.7
12.2
11.9
11.5
11.2
10.9
10.6
10.4
10.1
9.9
9.7
9.5
9.3
9.1
9.0
8.8
8.7
8.5
8.4
8.3
8.1
8.0
7.9
7.8
7.7
7.6
7.5
7.4
7.3
7.2
7.2
7.1
7.0
6.9
6.8
6.8
6.7
6.6
6.6
6.5
6.5
6.4
6.3
6.3
6.2
6.2

1.0
0.9

0.8
0.7
0.6
0.5
0.4
0.3
0.2

0.1
0.0
16.0

14.0

12.0

10.0

8.0
SD

6.0

4.0

2.0

0.0

6.1
6.1
6.0
6.0
5.9
5.9
5.8
5.8
5.8
5.7
5.7
5.6
5.6
5.6
5.5
5.5
5.4
5.4
5.4
5.3
5.3
5.3
5.2
5.2
5.2
5.1
5.1
5.1
5.1
5.0
5.0
5.0
4.9
4.9
4.9
4.9
4.8
4.8
4.8
4.8
4.7
4.7
4.7
4.7
4.7
4.6
4.6
4.6
4.6
4.5
4.5
4.5
4.5
4.5
4.4
4.4

4.4
4.4
4.4
4.3
4.3
4.3
4.3
4.3
4.3
4.2
4.2
4.2
4.2
4.2
4.2
4.1
4.1
4.1
4.1
4.1
4.1
4.1
4.0
4.0
4.0
4.0
4.0
4.0
4.0
3.9
3.9
3.9
3.9
3.9
3.9

1.0
0.9

0.8
0.6
0.5
0.4
0.3
0.2

0.1
0.0
0.0

Power

0.7

STT

Power Analysis
Finding Bias for a given n , with CV: see Worksheet Power CV
a and b considerations (maximum bias = 15%)
n = 10
n = 20
CV
6.65
8.69
CI-a
15 - CI-a
CI-b
df
9
3.85
11.1
7.52
19
3.36
11.6
6.64

15 - CI-b
7.5
8.4

Exact values by power of the non-c

"R" software: http://www.r-project.o
Piface: http://www.cs.uiowa.edu/~rl
http://www.psycho.uni-duesseldorf.

With 20 measurements, we can tolerate a bias of 11.6% (a-error) and 8.4% (b-error).

Exact values by power of the non-central one-sample t- test (95% power)

R" software: http://www.r-project.org/
Piface: http://www.cs.uiowa.edu/~rlenth/Power/
http://www.psycho.uni-duesseldorf.de/aap/projects/gpower/

1-sample t

STT
SD from power calculations (limit 15)
SD
4.11
6.65
n
5
10
Effect-size
t-test
Diff/SD
Power
Power
0.00
0.050
0.050
0.01
0.052
0.053
0.02
0.054
0.056
0.03
0.056
0.060
0.04
0.058
0.063
0.05
0.061
0.067
0.06
0.063
0.071
0.07
0.065
0.075
0.08
0.068
0.079
0.09
0.070
0.084
0.10
0.073
0.088
0.11
0.075
0.093
0.12
0.078
0.098
0.13
0.081
0.103
0.14
0.084
0.109
0.15
0.087
0.114
0.16
0.090
0.120
0.17
0.093
0.126
0.18
0.096
0.132
0.19
0.099
0.138
0.20
0.102
0.145
0.21
0.106
0.152
0.22
0.109
0.159
0.23
0.113
0.166
0.24
0.116
0.173
0.25
0.120
0.181
0.26
0.124
0.188
0.27
0.128
0.196
0.28
0.132
0.205
0.29
0.136
0.213
0.30
0.140
0.222
0.31
0.144
0.230
0.32
0.148
0.239
0.33
0.153
0.249
0.34
0.157
0.258
0.35
0.162
0.267
0.36
0.166
0.277
0.37
0.171
0.287
0.38
0.176
0.297
0.39
0.181
0.307
0.40
0.185
0.317
0.41
0.190
0.328
0.42
0.196
0.339
0.43
0.201
0.349
0.44
0.206
0.360

8.69
20

11.9
100

Power
0.050
0.055
0.060
0.065
0.070
0.076
0.083
0.090
0.097
0.104
0.112
0.121
0.130
0.139
0.149
0.159
0.170
0.181
0.193
0.205
0.217
0.230
0.243
0.257
0.271
0.286
0.300
0.315
0.331
0.347
0.363
0.379
0.396
0.412
0.429
0.446
0.463
0.480
0.497
0.515
0.532
0.549
0.566
0.583
0.600

Power
0.050
0.061
0.074
0.089
0.106
0.125
0.147
0.171
0.198
0.226
0.257
0.290
0.325
0.362
0.400
0.438
0.478
0.517
0.557
0.596
0.634
0.670
0.705
0.739
0.770
0.799
0.826
0.850
0.872
0.892
0.909
0.924
0.937
0.949
0.958
0.966
0.973
0.979
0.983
0.987
0.990
0.992
0.994
0.996
0.997

Difference
0.000
0.041
0.082
0.123
0.164
0.206
0.247
0.288
0.329
0.370
0.411
0.452
0.493
0.534
0.575
0.617
0.658
0.699
0.740
0.781
0.822
0.863
0.904
0.945
0.986
1.028
1.069
1.110
1.151
1.192
1.233
1.274
1.315
1.356
1.397
1.439
1.480
1.521
1.562
1.603
1.644
1.685
1.726
1.767
1.808

0.45
0.46
0.47
0.48
0.49
0.50
0.51
0.52
0.53
0.54
0.55
0.56
0.57
0.58
0.59
0.60
0.61
0.62
0.63
0.64
0.65
0.66
0.67
0.68
0.69
0.70
0.71
0.72
0.73
0.74
0.75
0.76
0.77
0.78
0.79
0.80
0.81
0.82
0.83
0.84
0.85
0.86
0.87
0.88
0.89
0.90
0.91
0.92
0.93
0.94
0.95
0.96

0.211
0.217
0.222
0.228
0.233
0.239
0.245
0.251
0.256
0.262
0.268
0.275
0.281
0.287
0.293
0.300
0.306
0.313
0.319
0.326
0.332
0.339
0.346
0.352
0.359
0.366
0.373
0.380
0.387
0.394
0.401
0.408
0.415
0.422
0.429
0.436
0.444
0.451
0.458
0.465
0.472
0.480
0.487
0.494
0.501
0.508
0.516
0.523
0.530
0.537
0.544
0.551

0.371
0.382
0.393
0.405
0.416
0.427
0.439
0.450
0.462
0.473
0.485
0.497
0.508
0.520
0.531
0.543
0.554
0.566
0.577
0.589
0.600
0.611
0.622
0.633
0.644
0.655
0.665
0.676
0.686
0.697
0.707
0.716
0.726
0.736
0.745
0.754
0.763
0.772
0.781
0.789
0.798
0.806
0.814
0.821
0.829
0.836
0.843
0.850
0.857
0.863
0.869
0.875

0.616
0.632
0.649
0.664
0.680
0.695
0.710
0.725
0.739
0.753
0.766
0.779
0.792
0.804
0.815
0.827
0.837
0.848
0.858
0.867
0.876
0.885
0.893
0.901
0.908
0.915
0.921
0.927
0.933
0.939
0.944
0.948
0.953
0.957
0.961
0.964
0.967
0.970
0.973
0.976
0.978
0.980
0.982
0.984
0.986
0.987
0.989
0.990
0.991
0.992
0.993
0.994

0.998
0.998
0.999
0.999
0.999
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

1.850
1.891
1.932
1.973
2.014
2.055
2.096
2.137
2.178
2.219
2.261
2.302
2.343
2.384
2.425
2.466
2.507
2.548
2.589
2.630
2.672
2.713
2.754
2.795
2.836
2.877
2.918
2.959
3.000
3.041
3.083
3.124
3.165
3.206
3.247
3.288
3.329
3.370
3.411
3.452
3.494
3.535
3.576
3.617
3.658
3.699
3.740
3.781
3.822
3.863
3.905
3.946

0.97
0.98
0.99
1.00
1.01
1.02
1.03
1.04
1.05
1.06
1.07
1.08
1.09
1.10
1.11
1.12
1.13
1.14
1.15
1.16
1.17
1.18
1.19
1.20
1.21
1.22
1.23
1.24
1.25
1.26
1.27
1.28
1.29
1.30
1.31
1.32
1.33
1.34
1.35
1.36
1.37
1.38
1.39
1.40
1.41
1.42
1.43
1.44
1.45
1.46
1.47
1.48

0.559
0.566
0.573
0.580
0.587
0.594
0.601
0.608
0.614
0.621
0.628
0.635
0.642
0.648
0.655
0.661
0.668
0.674
0.681
0.687
0.693
0.700
0.706
0.712
0.718
0.724
0.730
0.736
0.741
0.747
0.753
0.758
0.764
0.769
0.775
0.780
0.785
0.790
0.795
0.800
0.805
0.810
0.815
0.819
0.824
0.829
0.833
0.837
0.842
0.846
0.850
0.854

0.881
0.887
0.892
0.898
0.903
0.907
0.912
0.917
0.921
0.925
0.929
0.933
0.937
0.940
0.944
0.947
0.950
0.953
0.956
0.958
0.961
0.963
0.965
0.967
0.970
0.971
0.973
0.975
0.977
0.978
0.980
0.981
0.982
0.984
0.985
0.986
0.987
0.988
0.989
0.989
0.990
0.991
0.992
0.992
0.993
0.993
0.994
0.994
0.995
0.995
0.996
0.996

0.994
0.995
0.996
0.996
0.997
0.997
0.997
0.998
0.998
0.998
0.998
0.999
0.999
0.999
0.999
0.999
0.999
0.999
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

3.987
4.028
4.069
4.110
4.151
4.192
4.233
4.274
4.316
4.357
4.398
4.439
4.480
4.521
4.562
4.603
4.644
4.685
4.727
4.768
4.809
4.850
4.891
4.932
4.973
5.014
5.055
5.096
5.138
5.179
5.220
5.261
5.302
5.343
5.384
5.425
5.466
5.507
5.549
5.590
5.631
5.672
5.713
5.754
5.795
5.836
5.877
5.918
5.960
6.001
6.042
6.083

1.49
1.50
1.51
1.52
1.53
1.54
1.55
1.56
1.57
1.58
1.59
1.60
1.61
1.62
1.63
1.64
1.65
1.66
1.67
1.68
1.69
1.70
1.71
1.72
1.73
1.74
1.75
1.76
1.77
1.78
1.79
1.80
1.81
1.82
1.83
1.84
1.85
1.86
1.87
1.88
1.89
1.90
1.91
1.92
1.93
1.94
1.95
1.96
1.97
1.98
1.99
2.00

0.858
0.862
0.866
0.870
0.873
0.877
0.880
0.884
0.887
0.891
0.894
0.897
0.900
0.903
0.906
0.909
0.912
0.914
0.917
0.920
0.922
0.925
0.927
0.930
0.932
0.934
0.936
0.939
0.941
0.943
0.945
0.947
0.948
0.950
0.952
0.954
0.955
0.957
0.958
0.960
0.961
0.963
0.964
0.966
0.967
0.968
0.969
0.971
0.972
0.973
0.974
0.975

0.996
0.997
0.997
0.997
0.997
0.998
0.998
0.998
0.998
0.998
0.998
0.999
0.999
0.999
0.999
0.999
0.999
0.999
0.999
0.999
0.999
0.999
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

6.124
6.165
6.206
6.247
6.288
6.329
6.371
6.412
6.453
6.494
6.535
6.576
6.617
6.658
6.699
6.740
6.782
6.823
6.864
6.905
6.946
6.987
7.028
7.069
7.110
7.151
7.193
7.234
7.275
7.316
7.357
7.398
7.439
7.480
7.521
7.562
7.604
7.645
7.686
7.727
7.768
7.809
7.850
7.891
7.932
7.973
8.015
8.056
8.097
8.138
8.179
8.220

1-sided
G*Power software (http://www.psycho.uni-duesseldorf.de/aap/projects/gpower/).
20

100

Difference
0.000
0.067
0.133
0.200
0.266
0.333
0.399
0.466
0.532
0.599
0.665
0.732
0.798
0.865
0.931
0.998
1.064
1.131
1.197
1.264
1.330
1.397
1.463
1.530
1.596
1.663
1.729
1.796
1.862
1.929
1.995
2.062
2.128
2.195
2.261
2.328
2.394
2.461
2.527
2.594
2.660
2.727
2.793
2.860
2.926

Difference
0.000
0.087
0.174
0.261
0.348
0.435
0.521
0.608
0.695
0.782
0.869
0.956
1.043
1.130
1.217
1.304
1.390
1.477
1.564
1.651
1.738
1.825
1.912
1.999
2.086
2.173
2.259
2.346
2.433
2.520
2.607
2.694
2.781
2.868
2.955
3.042
3.128
3.215
3.302
3.389
3.476
3.563
3.650
3.737
3.824

Difference
0.000
0.119
0.238
0.357
0.476
0.595
0.714
0.833
0.952
1.071
1.190
1.309
1.428
1.547
1.666
1.785
1.904
2.023
2.142
2.261
2.380
2.499
2.618
2.737
2.856
2.975
3.094
3.213
3.332
3.451
3.570
3.689
3.808
3.927
4.046
4.165
4.284
4.403
4.522
4.641
4.760
4.879
4.998
5.117
5.236

1.0
0.9
0.8
0.7
Power

10

0.6
0.5
0.4
0.3
0.2
0.1
0.0
0.0

2.0

4.0
Difference

6.0

2.993
3.059
3.126
3.192
3.259
3.325
3.392
3.458
3.525
3.591
3.658
3.724
3.791
3.857
3.924
3.990
4.057
4.123
4.190
4.256
4.323
4.389
4.456
4.522
4.589
4.655
4.722
4.788
4.855
4.921
4.988
5.054
5.121
5.187
5.254
5.320
5.387
5.453
5.520
5.586
5.653
5.719
5.786
5.852
5.919
5.985
6.052
6.118
6.185
6.251
6.318
6.384

3.911
3.997
4.084
4.171
4.258
4.345
4.432
4.519
4.606
4.693
4.780
4.866
4.953
5.040
5.127
5.214
5.301
5.388
5.475
5.562
5.649
5.735
5.822
5.909
5.996
6.083
6.170
6.257
6.344
6.431
6.518
6.604
6.691
6.778
6.865
6.952
7.039
7.126
7.213
7.300
7.387
7.473
7.560
7.647
7.734
7.821
7.908
7.995
8.082
8.169
8.256
8.342

5.355
5.474
5.593
5.712
5.831
5.950
6.069
6.188
6.307
6.426
6.545
6.664
6.783
6.902
7.021
7.140
7.259
7.378
7.497
7.616
7.735
7.854
7.973
8.092
8.211
8.330
8.449
8.568
8.687
8.806
8.925
9.044
9.163
9.282
9.401
9.520
9.639
9.758
9.877
9.996
10.115
10.234
10.353
10.472
10.591
10.710
10.829
10.948
11.067
11.186
11.305
11.424

6.451
6.517
6.584
6.650
6.717
6.783
6.850
6.916
6.983
7.049
7.116
7.182
7.249
7.315
7.382
7.448
7.515
7.581
7.648
7.714
7.781
7.847
7.914
7.980
8.047
8.113
8.180
8.246
8.313
8.379
8.446
8.512
8.579
8.645
8.712
8.778
8.845
8.911
8.978
9.044
9.111
9.177
9.244
9.310
9.377
9.443
9.510
9.576
9.643
9.709
9.776
9.842

8.429
8.516
8.603
8.690
8.777
8.864
8.951
9.038
9.125
9.211
9.298
9.385
9.472
9.559
9.646
9.733
9.820
9.907
9.994
10.080
10.167
10.254
10.341
10.428
10.515
10.602
10.689
10.776
10.863
10.949
11.036
11.123
11.210
11.297
11.384
11.471
11.558
11.645
11.732
11.818
11.905
11.992
12.079
12.166
12.253
12.340
12.427
12.514
12.601
12.687
12.774
12.861

11.543
11.662
11.781
11.900
12.019
12.138
12.257
12.376
12.495
12.614
12.733
12.852
12.971
13.090
13.209
13.328
13.447
13.566
13.685
13.804
13.923
14.042
14.161
14.280
14.399
14.518
14.637
14.756
14.875
14.994
15.113
15.232
15.351
15.470
15.589
15.708
15.827
15.946
16.065
16.184
16.303
16.422
16.541
16.660
16.779
16.898
17.017
17.136
17.255
17.374
17.493
17.612

9.909
9.975
10.042
10.108
10.175
10.241
10.308
10.374
10.441
10.507
10.574
10.640
10.707
10.773
10.840
10.906
10.973
11.039
11.106
11.172
11.239
11.305
11.372
11.438
11.505
11.571
11.638
11.704
11.771
11.837
11.904
11.970
12.037
12.103
12.170
12.236
12.303
12.369
12.436
12.502
12.569
12.635
12.702
12.768
12.835
12.901
12.968
13.034
13.101
13.167
13.234
13.300

12.948
13.035
13.122
13.209
13.296
13.383
13.470
13.556
13.643
13.730
13.817
13.904
13.991
14.078
14.165
14.252
14.339
14.425
14.512
14.599
14.686
14.773
14.860
14.947
15.034
15.121
15.208
15.294
15.381
15.468
15.555
15.642
15.729
15.816
15.903
15.990
16.077
16.163
16.250
16.337
16.424
16.511
16.598
16.685
16.772
16.859
16.946
17.032
17.119
17.206
17.293
17.380

17.731
17.850
17.969
18.088
18.207
18.326
18.445
18.564
18.683
18.802
18.921
19.040
19.159
19.278
19.397
19.516
19.635
19.754
19.873
19.992
20.111
20.230
20.349
20.468
20.587
20.706
20.825
20.944
21.063
21.182
21.301
21.420
21.539
21.658
21.777
21.896
22.015
22.134
22.253
22.372
22.491
22.610
22.729
22.848
22.967
23.086
23.205
23.324
23.443
23.562
23.681
23.800

uesseldorf.de/aap/projects/gpower/).
Different n
Fixed SD

8.69

(n = 20)

1.0
0.9
0.8

Power

0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
8.0

10.0

0.0

5.0

10.0
Difference

15.0

20.0

STT

Tolerance interval
Finding Bias for a given n , Tolerance interval considerations
Tolerance interval (0.95, 0.95); TE limit = 29.4% ($)
1-sided
1-sided
n
Beta CV
k
kxs
10
6.65
2.91
19.4
20
8.69
2.40
20.9
k-values from ISO 3207-1975
$29.4 = 1.96*15% CV
With 20 measurements, we can tolerate a bias of 8.5%.

Bias
10.0
8.5

1-sided b-content interval (0.95, 0.95) (for ANOVA); TE limit = 29.4%, values from Data-Worksheet
This example is based on the Worksheets Precision and Data (ANOVA calculations)
Hoffman D, Kringle R. Pharm Res 2007;24:1157-64.
s t2
73.8
H1
1.707
Help numbers
dfB
9
H2
1.538
12.606
dfE
10
MSB
76.30
1.026
Ne
18.86
MSE
71.29
Z0.95
1.645
J (rep/run)
2
2
CHI (0.95,9)
3.325
J-1
1
Not relevant for this application
CHI2(0.95,10)
3.940
I (runs)
10
Actual mean
102
Target
Bias
1-sided b-content (Mean +/-)
21.3
100
8.1
With 20 measurements, we can tolerate a bias of 8.1%.
Note 1
The "rest bias" is calculated by use of the TE-limit of 29.4%.

Note 2
With n = 20, all 3 approaches deliver a bias limit of approx. 8%
when we use the "b"-CV for calculation.

, values from Data-Worksheet

VA calculations)
Note, simplified calculations, better values in:
-Expectation and -Content Tolerance Limits for Balanced One-Way ANOVA Random Model
Robert W. Mee, Technometrics, Vol. 26, No. 3 (1984), pp. 251-254

r this application

STT

Precision

Requirement
We will work with the CV requirement for bioanalytical method validation [1]: CVmax = 15% (20% at the lim
understood as total CV, consisting of a within-day (within-run) and a between-day (between
work with the standard deviation (s) at a level of 100 (= m). This corresponds then directly to the CV.

Experimental design
The experimental design for estimating imprecision should strive for balanced degrees of freedom (df) for w
Further, the number of measurement results should be sufficiently high for obtaining reasonable confidence
design:
Days = 10 (= k); Replicates = 2 (per day = n); Total = 20 (the guidance documents require 5 [1] or 3 [2] mea
The data are simulated with the EXCEL-function "Random Number Generation" (Tools>Data Analysis) in s
of approximately 8.69. Three data sets are simulated:
sb = 0, sw = 8.69, m = 100 (directly 2 x 10)
sb = 6.14, sw = 6.14, m = 100 (first the 10 between, then the replicates: 10 times, "between values",
sb = 7.8, sw = 3.8, m = 100 (see case 2)
Depending on the ratio sb/sw, the df's for st approach the total df's (maximum 19) or the between df's (minim
calculated by ANOVA (dynamically programmed), the df's with the Satterthwaite approximation =[(n

1 Guidance for Industry: Bioanalytical Method Validation, US Department of Health andHumanServices, Fo

Evaluation and Research (CBER), Rockville, MD, 2001.

2 ICH Topic Q2 (R1), Validation of Analytical Procedures: Text and Methodology, International Conference
Requirements for Registration of Pharmaceuticals for Human Use, Geneva, 2005.

3 Stckl D, D'Hondt H, Thienpont LM. Method validation across the disciplines-Critical investigation of majo
protocols. J Chromatogr B Analyt Technol Biomed Life Sci 2008;Dec 31[Epub ahead of print].

[1]: CVmax = 15% (20% at the limit of quantitation, LoQ). This CVmax is
day (between-run) component. For the ease of the EXCEL-simulations, we will
onds then directly to the CV.

anced degrees of freedom (df) for within- (sw) and between-day (sb) estimates [3].
or obtaining reasonable confidence intervals. We choose for following experimental

ocuments require 5 [1] or 3 [2] measurements, only).

eration" (Tools>Data Analysis) in such a way to obtain a total standard deviation (st)

0 times, "between values", sw and n = 2)

mum 19) or the between df's (minimum 9). The imprecision components are
rthwaite approximation =[(n -1)xMSw+MSb]^2/{[(n - 1)xMSw^2]/k+MSb^2/(k-1)}.

nt of Health andHumanServices, Food and Drug Administration, Center for Biologics

hodology, International Conference on Harmonization (ICH) of Technical

Critical investigation of major validation criteria and associated experimental

Epub ahead of print].

STT

Data
ANOVA (equal sample sizes!)
Replicate
Day
A
B
1
109.8
99.1
2
103.0
105.2
3
91.7
104.5
4
114.6
92.6
5
99.4
95.1
6
108.3
113.0
7
114.0
106.4
8
98.1
107.0
9
92.0
107.0
10
98.0
82.0
AVERAGETotal
102.040
COUNTRep
2

Source of Variation
Between Groups
Within Groups
Satterthwaite
Total

SS
686.6880
712.8600
1399.55

ANOVA (equal sample sizes!)

Replicate
Day
A
B
1
106.8
100.5
2
95.4
103.6
3
102.0
84.3
4
107.2
122.3
5
104.3
103.0
6
95.5
97.2
7
115.9
108.6
8
103.7
99.7
9
93.6
91.0
10
101.4
102.6
AVERAGETotal
101.930
COUNTRep
2

Source of Variation
Between Groups
Within Groups
Satterthwaite
Total

SS
967.3920
365.1500
1332.542

ANOVA (equal sample sizes!)

Replicate

VAR
57.245
2.42
81.92
242
9.245
11.045
28.88
39.605
112.5
128

df
9
10
18.86
19

VAR
19.845
33.62
156.645
114.005
0.845
1.445
26.645
8
3.38
0.72

df
9
10
15
19

Daily
Mean
104.45
104.1
98.1
103.6
97.25
110.65
110.2
102.55
99.5
90

MS
76.299
71.286

Daily
Mean
103.65
99.5
93.15
114.75
103.65
96.35
112.25
101.7
92.3
102

MS
107.488
36.515

Daily

Day
1
2
3
4
5
6
7
8
9
10
AVERAGETotal
COUNTRep

Source of Variation
Between Groups
Within Groups
Satterthwaite
Total

A
106.2
85.8
101.2
101.0
103.1
109.6
83.6
100.6
101.7
99.6

B
101.7
91.0
108.0
108.2
109.0
118.0
88.0
99.4
104.9
103.6
101.210
2

VAR
10.125
13.52
23.12
25.92
17.405
35.28
9.68
0.72
5.12
8

Mean
103.95
88.4
104.6
104.6
106.05
113.8
85.8
100
103.3
101.6

SS
1239.948
148.890

df
9
10
11
19

MS
137.772
14.889

1388.838

Help 1
5.81
4.24
15.52
2.43
22.94
74.13
66.59
0.26
6.45
144.96

Help 2
57.245
2.42
81.92
242
9.245
11.045
28.88
39.605
112.5
128

8.44 s w
Set s b to 0 when a negative SQRT results!
1.58 s b
s
8.59 t
Standard error: s/sqrt(n)
1.928 s t, Satterthwaite+1
1.953 s b and s t

Help 1
2.96
5.90
77.09
164.35
2.96
31.14
106.50
0.05
92.74
0.00

Help 2
19.845
33.62
156.645
114.005
0.845
1.445
26.645
8
3.38
0.72

6.04 s w
5.96 s b
8.49 s t
Standard error: s/sqrt(n)
2.146 s t, Satterthwaite+1
2.318 s b and s t

Help 1
7.51
164.10
11.49
11.49
23.43
158.51
237.47
1.46
4.37
0.15

Help 2
10.125
13.52
23.12
25.92
17.405
35.28
9.68
0.72
5.12
8

3.86 s w
7.84 s b
8.74 s t
Standard error: s/sqrt(n)
2.529 s t, Satterthwaite+1
2.625 s b and s t