APPENDIX A

Statistical Tables

This Appendix contains a selection of statistical tables for use in solving the
problems in the text. The tables are abbreviated forms of larger tables found in
statistical books, such as National Bureau of Standards Handbook 91, referred to in
several places throughout the text.

Table A.1. Use of Range to Estimate Standard Deviation

Number of Factor
Sets of Degrees of Number of replicates in set
Replicates Freedom 2 3 4 5 6
d 2⋅ 1.41 1.91 2.24 2.48 2.67
1 df 1.00 1.98 2.93 3.83 4.68

d 2⋅ 1.23 1.77 2.12 2.38 2.58

3 df 2.83 5.86 8.44 11.1 13.6

d 2⋅ 1.19 1.74 2.10 2.36 2.56

5 df 4.59 9.31 13.9 18.4 22.6

d 2⋅ 1.16 1.72 2.08 2.34 2.55

10 df 8.99 18.4 27.6 36.5 44.9

d 2⋅ 1.15 1.71 2.07 2.34 2.54

15 df 13.4 27.5 41.3 54.6 67.2

d 2⋅ 1.14 1.70 2.07 2.33 2.54

20 df 17.8 36.5 55.0 72.7 89.6

∞ d2⋅ 1.13 1.69 2.06 2.33 2.53

Intermediate values for d 2⋅ and df may be obtained by interpolation, or from the reference
from which this table was adapted. Adapted from Lloyd S. Nelson. J. Qual. Tech. 7 No. 1.
January 1975. ©American Society for Quality Control, Used by permission.

233

© 2004 by CRC Press LLC

 234 STATISTICAL TECHNIQUES FOR DATA ANALYSIS

Table A.2. Z-Factors for Confidence Intervals for the Normal Distribution
Confidence Level, % Z-Factor
One-Sided Two-Sided
Interval Interval
50 0.68
67 1.00
75 1.15
90 1.282 1.645
95 1.645 1.960
95.28 2.000
99 2.326 2.575
99.5 2.576
99.74 3.000
99.9 3.090
99.9934 4.000
99.99995 5.000
100 – 10−9 6.000

© 2004 by CRC Press LLC

 STATISTICAL TABLES 235

Table A.3. Student t Variate

* 80% 90% 95% 98% 99% 99.73%
Z=3
** 90% 95% 97.5% 99% 99.5% 99.85%

Df t.90 t.95 t.975 t.99 t.995 t.9985

1 3.078 6.314 12.706 31.821 63.657 235.80
2 1.886 2.920 4.303 6.965 9.925 19.207
3 1.638 2.353 3.182 4.541 5.841 9.219
4 1.533 2.132 2.776 3.747 4.604 6.620
5 1.476 2.015 2.571 3.365 4.032 5.507

6 1.440 1.943 2.447 3.143 3.707 4.904

7 1.415 1.895 2.365 2.998 3.499 4.530
8 1.397 1.860 2.306 2.896 3.355 4.277
9 1.383 1.833 2.362 2.821 3.250 4.094
10 1.372 1.812 2.228 2.764 3.169 3.975

11 1.363 1.796 2.201 2.718 3.106 3.850

12 1.356 1.782 2.179 2.681 3.055 3.764
13 1.350 1.771 2.160 2.650 3.012 3.694
14 1.345 1.761 2.145 2.624 2.977 3.636
15 1.341 1.753 2.131 2.602 2.947 3.586

16 1.337 1.746 2.120 2.583 2.921 3.544

17 1.333 1.740 2.110 2.567 2.898 3.507
18 1.330 1.734 2.101 2.552 2.878 3.475
19 1.328 1.729 2.093 2.539 2.861 3.447
20 1.325 1.725 2.086 2.528 2.845 3.422

25 1.316 1.708 2.060 2.485 2.787 3.330

30 1.310 1.697 2.042 2.457 2.750 3.270

40 1.303 1.684 2.021 2.423 2.704 3.199

60 1.296 1.671 2.000 2.390 2.660 3.130

∞ 1.282 1.645 1.960 2.326 2.576 3.000

*Columns to be used in calculating corresponding two-sided confidence interval. Excerpted from

“Experimental Statistics” NBS Handbook 91. Last column from B. J. Joiner, J. Research NBS.
**Columns to be used for a corresponding one-sided test.

© 2004 by CRC Press LLC

 236 STATISTICAL TECHNIQUES FOR DATA ANALYSIS

Table A.4. Factors for Computing Two-Sided Confidence Intervals for σ

Degrees α = .05 (95% α = .01 (99% α = .001 (99.9%
of confidence) confidence) confidence)
Freedom BU BL BU BL BU BL
1 17.79 .358 86.31 .297 844.4 .248
2 4.86 .458 10.70 .388 33.3 .329
3 3.18 .518 5.45 .445 11.6 .382
4 2.57 .559 3.89 .486 6.94 .422
5 2.25 .590 3.18 .518 5.08 .453
6 2.05 .614 2.76 .544 4.13 .478
7 1.92 .634 2.50 .565 3.55 .500
8 1.82 .651 2.31 .583 3.17 .519
9 1.75 .666 2.17 .599 2.89 .535
10 1.69 .678 2.06 .612 2.69 .549

15 1.51 .724 1.76 .663 2.14 .603

20 1.42 .754 1.61 .697 1.89 .640

25 1.36 .775 1.52 .721 1.74 .667

30 1.32 .791 1.46 .740 1.64 .688

35 1.29 .804 1.41 .755 1.58 .705

40 1.27 .815 1.38 .768 1.52 .720

45 1.25 .824 1.35 .779 1.48 .732

50 1.23 .831 1.33 .788 1.45 .743

Excerpted from “Experimental Statistics”, NBS Handbook 91.

© 2004 by CRC Press LLC

 STATISTICAL TABLES 237

Table A.5. Factors for Computing Two-Sided Tolerance Intervals for a Normal
Distribution

τ = 0.95 τ = 0.99
n/p .90 .95 .99 .999 .90 .95 .99 .999
2 32.02 37.67 48.43 60.57 160.19 188.49 242.30 303.05
3 8.38 9.92 12.86 16.21 18.93 22.40 29.06 36.62
4 5.37 6.37 8.30 10.50 9.40 11.15 14.53 18.38
5 4.28 5.08 6.63 8.42 6.61 7.86 10.26 13.02
6 3.71 4.41 5.78 7.34 5.34 6.34 8.30 10.55
7 3.37 4.01 5.25 6.68 4.61 5.49 7.19 9.14
8 3.14 3.73 4.89 6.23 4.15 4.94 6.47 8.23
9 2.97 3.53 4.63 5.90 3.82 4.55 5.97 7.60
10 2.84 3.38 4.43 5.65 3.58 4.26 5.59 7.13

15 2.48 2.95 3.88 4.95 2.94 3.51 4.60 5.88

20 2.31 2.75 3.62 4.61 2.66 3.17 4.16 5.31

25 2.21 2.63 3.46 4.41 2.49 2.97 3.90 4.98

p = proportion of population to be covered (e.g., .90 = 90%).
τ = probability of inclusion (e.g., .95 = 95%).
Excerpted from “Experimental Statistics”, NBS Handbook 91.

© 2004 by CRC Press LLC

 238 STATISTICAL TECHNIQUES FOR DATA ANALYSIS

Table A.6. Critical Values for the F Test, F0.975

dfN
dfD 1 2 4 6 8 10 15 20 30 40

1 648 800 900 937 957 969 983 993 1001 1006
2 38.5 39.0 39.2 39.3 39.4 39.4 39.4 39.4 39.5 39.5
4 12.2 10.6 9.6 9.2 9.0 8.8 8.7 8.6 8.5 8.4
6 8.8 7.3 6.2 5.8 5.6 5.5 5.3 5.2 5.1 5.0
8 7.6 6.1 5.0 4.6 4.4 4.3 4.1 4.0 3.9 3.8
10 6.9 5.5 4.5 4.1 3.8 3.7 3.5 3.4 3.3 3.3
15 6.2 4.8 3.8 3.4 3.2 3.1 2.9 2.8 2.6 2.6
20 5.9 4.5 3.5 3.1 2.9 2.8 2.6 2.5 2.4 2.3
30 5.6 4.2 3.2 2.9 2.6 2.5 2.3 2.2 2.1 2.0
40 5.4 4.0 3.1 2.7 2.5 2.4 2.2 2.1 1.9 1.9
60 5.3 3.9 3.0 2.6 2.4 2.3 2.1 1.9 1.8 1.7
120 5.2 3.8 2.9 2.5 2.3 2.2 1.9 1.8 1.7 1.6
∞ 5.0 3.7 2.8 2.4 2.2 2.1 1.8 1.7 1.6 1.5

For use for a one-tailed test of equality of standard deviation estimates at 2.5% level of confi-
dence, or for a two-tailed test at 5% level of confidence. dfN and dfD refer to degrees of free-
dom of variances of numerator and denominator of F test, respectively. Excerpted from “Ex-
perimental Statistics”, NBS Handbook 91.

© 2004 by CRC Press LLC

 STATISTICAL TABLES 239

Table A.7. Values for Use in the Dixon Test for Outliers
Risk of False Rejection
Number of
Statistic Observations, n 0.5% 1% 5% 10%
3 .994 .988 .941 .886
4 .926 .889 .765 .679
τ10 5 .821 .780 .642 .557
6 .740 .698 .560 .482
7 .680 .637 .507 .434

8 .725 .683 .554 .479

τ11 9 .677 .635 .512 .441
10 .639 .597 .477 .409

11 .713 .679 .576 .517

τ21 12 .675 .642 .546 .490
13 .649 .615 .521 .467

14 .674 .641 .546 .492

15 .647 .616 .525 .472
16 .624 .595 .507 .454
τ22 17 .605 .577 .490 .438
18 .589 .561 .475 .424
19 .575 .547 .462 .412
20 .562 .535 .450 .401
Tabulated values obtained from reference NBS Handbook 91. Original reference: W. J. Dixon,
“Processing Data Outliers”, Biometrics BIOMA, March, 1953, Vol. 9, No. 1, pp 74–89.

© 2004 by CRC Press LLC

 240 STATISTICAL TECHNIQUES FOR DATA ANALYSIS

Table A.8. Values for Use in the Grubbs Test for Outliers
Risk of False Rejection
Number of 0.1% 0.5% 1% 5% 10%
Data Points Critical Values
3 1.155 1.155 1.155 1.153 1.148
4 1.496 1.496 1.492 1.463 1.425
5 1.780 1.764 1.749 1.672 1.602
6 2.011 1.973 1.944 1.822 1.729
7 2.201 2.139 2.097 1.938 1.828
8 2.358 2.274 2.221 2.032 1.909
9 2.492 2.387 2.323 2.110 1.977
10 2.606 2.482 2.410 2.176 2.036

15 2.997 2.806 2.705 2.409 2.247

20 3.230 3.001 2.884 2.557 2.385

25 3.389 3.135 3.009 2.663 2.486

50 3.789 3.483 3.336 2.956 2.768

100 4.084 3.754 3.600 3.207 3.017

Tabulated values obtained in part from ASTM E-178 which should be consulted for more
extensive tables. Original reference: F. E. Grubbs and G. Beck, “Extension of Sample Sizes and
Percentage Points for Significance Tests of Outlying Observations”, Technometrics, TCMTA,
Vol. 14, No. 4, Nov. 1972, pp 847–54.

© 2004 by CRC Press LLC

 STATISTICAL TABLES 241

Table A.9. Values for Use in the Youden Test to Identify Consistent Outlying
Performance Approximate 5% Two-Tail Limits for Ranking Scores*
Number of
Number of Materials
Parti-
cipants 3 4 5 6 7 8 9 10 11 12 13 14 15

3 4 5 7 8 10 12 13 15 17 19 20 22
12 15 17 20 22 24 27 29 31 33 36 38

4 4 6 8 10 12 14 16 18 20 22 24 26
16 19 22 25 28 31 34 37 40 43 46 49

5 5 7 9 11 13 16 18 21 23 26 28 31
19 23 27 31 35 38 42 45 49 52 56 59

6 3 5 7 10 12 15 18 21 23 26 29 32 35
18 23 28 32 37 41 45 49 54 58 62 66 70

7 3 5 8 11 14 17 20 23 26 29 32 36 39
21 27 32 37 42 47 52 57 62 67 72 76 81

8 3 6 9 12 15 18 22 25 29 32 36 39 43
24 30 36 42 48 54 59 65 70 76 81 87 92

9 3 6 9 13 16 20 24 27 31 35 39 43 47
27 34 41 47 54 60 66 73 79 85 91 97 103

10 4 7 10 14 17 21 26 30 34 38 43 47 51
29 37 45 52 60 67 73 80 87 94 100 107 114

11 4 7 11 15 19 23 27 32 36 41 46 51 55
32 41 49 57 65 73 81 88 96 103 110 117 125

12 4 7 11 15 20 24 29 34 39 44 49 54 59
35 45 54 63 71 80 88 96 104 112 120 128 136

13 4 8 12 16 21 26 31 36 42 47 52 58 63
38 48 58 68 77 86 95 104 112 121 130 138 147

14 4 8 12 17 22 27 33 38 44 50 56 61 67
41 52 63 73 83 93 102 112 121 130 139 149 158

15 4 8 13 18 23 29 35 41 47 53 59 65 71
44 56 67 78 89 99 109 119 129 139 149 159 169
*Approximate, because of rounding. From W.J. Youden. p. 149 of NBS Special Publication 300,
Vol. 1

© 2004 by CRC Press LLC

 242 STATISTICAL TECHNIQUES FOR DATA ANALYSIS

Table A.10. Values for Use in the Cochran Test for Extreme Values for Variance
(5% Risk of Wrong Decision)
Numbers of Number of Replicate Values Used to Compute
Variances Each Variance
Compared 2 3 4 5 6 7 10 ∞
2 .9985 .9750 .9392 .9057 .8772 .8534 .8010 .5000
3 .9969 .8709 .7977 .7457 .7071 .6771 .6167 .3333
4 .9065 .7679 .6841 .6287 .5895 .5598 .5017 .2500
5 .8412 .6838 .5981 .5441 .5065 .4783 .4214 .2000
6 .7808 .6161 .5321 .4803 .4447 .4184 .3682 .1667
7 .7271 .5612 .4800 .4307 .3974 .3726 .3259 .1429
10 .6020 .4450 .3733 .3311 .3029 .2823 .2439 .1000
20 .3894 .2705 .2205 .1921 .1735 .1602 .1357 .0500
30 .2929 .1980 .1593 .1377 .1237 .1137 .0958 .0333
40 .2370 .1576 .1259 .1082 .0968 .0887 .0745 .0250
60 .1737 .1131 .0895 .0765 .0682 .0623 .0520 .0167
Extract from Table 14.1, Eisenhart, Hastay, and Wallis, Selected Techniques of Statistical
Analysis, McGraw-Hill Book Co. (1947). Each variance must be estimated with the same number
of degrees of freedom. Calculate s2 (largest)/Σ s2i . If ratio exceeds tabulated value, assume
largest value is extreme with 95% confidence (5% risk of wrong decision).

© 2004 by CRC Press LLC

 STATISTICAL TABLES 243

Table A.11. A Short Table of Random Numbers

84 81 91 06 12 11 83 06 10 34 23 29 00 64 02 40
03 45 86 26 38 48 95 32 05 35 84 07 39 35 10 38
07 28 77 46 12 64 45 16 83 53 60 92 20 91 90 97
63 45 60 89 58 63 83 56 80 54 84 46 74 23 60 65
43 11 37 49 95 08 49 10 47 94 92 39 30 03 47 02

76 15 84 79 23 21 86 11 60 28 03 52 46 32 06 27
04 03 08 25 83 32 44 94 53 88 60 08 76 82 04 71
89 56 57 39 95 37 24 65 22 14 98 17 94 98 65 42
80 05 61 14 67 01 34 06 71 09 36 06 84 50 53 36
71 44 13 03 43 65 06 70 39 80 65 33 78 79 75 51

52 04 65 03 16 07 68 14 97 17 57 59 93 81 08 65
02 55 02 62 36 92 40 47 28 54 55 71 82 29 32 85
04 55 10 83 50 73 39 80 84 60 99 95 88 99 98 50
95 86 97 72 93 74 72 26 30 96 84 01 66 00 72 83
89 39 22 82 05 02 30 84 98 92 46 47 90 34 80 01

85 58 68 28 86 31 34 54 59 97 03 90 48 98 90 09
99 31 09 47 96 81 26 06 17 29 63 46 27 65 01 13
76 07 69 89 27 70 76 08 23 06 17 73 02 67 69 38
01 86 37 92 06 46 66 85 16 60 52 69 77 67 72 66
36 18 34 99 29 68 96 05 59 11 90 89 66 17 28 02

97 83 05 39 60 67 45 54 55 01 60 70 95 72 59 94
51 20 31 58 93 26 48 84 83 38 84 39 93 00 86 81
12 20 88 06 82 14 18 97 61 44 93 75 32 18 79 67
76 14 61 54 07 65 87 50 00 37 61 42 85 03 65 06
93 54 70 44 15 07 55 49 29 97 58 37 13 04 72 99

63 69 70 66 12 85 99 31 59 41 87 05 62 02 44 74
31 10 56 98 92 38 71 51 79 93 38 67 28 26 49 01
03 17 76 87 23 64 36 89 01 51 75 96 47 78 71 53
98 17 65 89 52 19 13 94 63 62 57 47 87 79 17 02
49 00 59 40 56 10 94 77 60 24 27 11 93 91 51 24

21 83 49 05 47 29 67 88 46 94 41 10 17 85 02 36
62 92 13 02 64 33 12 83 29 61 29 59 57 70 64 80
13 73 76 47 12 42 74 66 91 82 86 71 66 76 50 39
50 09 97 21 15 79 18 13 49 76 59 54 31 55 17 36
35 58 59 60 27 43 71 03 89 74 42 96 77 19 50 22

70 01 37 01 96 64 76 74 10 53 86 33 89 46 58 91
58 45 68 43 11 05 26 58 64 72 79 91 54 78 88 26
69 30 79 36 18 47 91 25 34 54 77 24 62 13 60 88
28 89 94 85 90 09 13 55 19 24 95 26 76 57 45 33
50 26 02 28 11 39 48 44 17 23 69 94 84 23 22 49

64 53 30 65 99 05 32 13 46 47 35 54 68 46 31 41
57 29 23 22 8 20 27 04 09 65 45 56 58 68 24 75
56 15 11 92 54 06 32 09 23 72 02 90 80 86 61 79

© 2004 by CRC Press LLC