11/6/2014

COMP 233/2
Probability and Statistics
for Computer Science
Week 10 (slides curtsey Dr. T. Fevens)
S. P. Mudur
Parameter estimation
Point estimates
Interval estimates

Reading: §7.1-7.3

Parameters and Statistics

• A parameter is a population measure (e.g. mean ,

variance 2).
• A statistic is a sample function (e.g. sample mean
X , sample variance S2).
• Hence, statistics may be regarded as random
variables.
• Statistics are used to estimate parameters and
are called estimators.
COMP233 Week 10 2

COMP233 Week 10 1
 11/6/2014

Estimators

• We will consider two types of estimators:

1. A point estimate (or maximum likelihood
estimator) of a parameter is a single
numerical value of a respective estimator.
2. An interval estimate of a parameter is an
interval in which we estimate that the
parameter lies.
 The width of the interval depends on how
much confidence we can attach to the
interval estimate.

COMP233 Week 10 3

“Good” Estimator

• Let  be a parameter and An a statistic

estimating  (based on a sample of size n).
• An is a “good” estimator of , if
1. It is unbiased : E[An] = 

2. It is consistent : lim  An  0;
n 

3. It is relatively efficient : It has the

smallest variance.

COMP233 Week 10 4

COMP233 Week 10 2
 11/6/2014

Estimating the
Mean and Variance
• Given a population of size N, we need to find a good
estimate for its mean  and variance .
• With data for all N, we can calculate the exact
values for  and .
• When the number N is too big, so we pick a sample of
reasonable (?) size n.
• Find the sample mean and sample variance. So the
question:
– How good is X as an estimate of  ?
– How good is S2 as an estimate of ?
COMP233 Week 10 5

Point Estimates in a Normal

Population
• Let X1, X2, . . ., Xn be a sequence of independent,
normal random variables, each having unknown
mean  and unknown variance 2.
• Then the point estimates for  and 2 are given,
respectively, by
n n

X
i 1
i (X
i 1
i  X )2
and
n n

COMP233 Week 10 6

COMP233 Week 10 3
 11/6/2014

Comment

• Note that the point estimator of the variance σ 2

differs from the sample variance
n

(X i  X )2
S2  i 1
n 1
in that the denominator is n rather than n − 1.
However, for n of reasonable size, these two
estimators of σ 2 will be approximately equal.

COMP233 Week 10 7

Point Estimators for

non-Normal Populations
• Let X1, X2, . . ., Xn be a sequence of independent,
random variables, each having unknown mean  and
unknown variance 2.
• For many populations (e.g., Normal, Bernoulli,
Poisson), the point estimator for the mean is ܺത.
But this is not always the case.
• E.g., for the Uniform Distribution the point
estimator for the mean (maximum likelihood) is
max( X 1 , X 2 , , X n )
2
COMP233 Week 10 8

COMP233 Week 10 4
 11/6/2014

Confidence Interval for a

Normal Mean (Variance
Known)

Reading: §7.3

Motivation

• The point estimator for μ for a normal population

is ܺത. However, we don’t expect that the sample
mean ܺത will exactly equal μ, but rather that it will
“be close.”
• Hence, rather than a point estimate, it is
sometimes more valuable to be able to specify an
interval for which we have a certain degree of
confidence that μ lies within.
• To obtain such an interval estimator, we make use
of the probability distribution of the point
estimator.
COMP233 Week 10 10

COMP233 Week 10 5
 11/6/2014

Interval Estimates: Terms

• An interval estimate of a parameter is an interval

within which the parameter is estimated to exist.
• The confidence level of an interval estimate is the
probability that the interval contains the
parameter.
• Notation: An interval estimate with a confidence
level 1-, is referred to as a 1- confidence
interval.

COMP233 Week 10 11

Interval Estimates on the

Population Mean
• An interval estimate on the mean is an interval
centered at the sample mean:

(X , X   )
 is the maximum error of estimation.

• That is  ( X   , X   )
• How confident we are in this statement depends
on (1 – )  the confidence level of the interval.

COMP233 Week 10 12

COMP233 Week 10 6
 11/6/2014

The Error and the

Confidence Level
• Recall that 1    P{ X      X   }
• It is clear that 1- grows as  increases.
– You have a better chance of hitting the
population mean if you widen the interval
around the sample mean.
• We would like to know the exact relation between
 and .
– For example, what would the error be if you
would like to be 99% confident in your interval
estimate of  ?
COMP233 Week 10 13

Note

X     X 
     X  
      X    
    X   
    X   
Thus, 1    P{    X     }

COMP233 Week 10 14

COMP233 Week 10 7
 11/6/2014

Observations

• If the original variable X is normally distributed,

then the sample mean is normally distributed with
mean  and variance 2/n.
• We’re interested in P{x1  X  x2 }
where x1 =  and x2 = .
• The corresponding z-values are
x1   (    )   n n
z1     ; z2  
/ n / n  

COMP233 Week 10 15

The Relation between  and 

n
• Thus, 1    2 P{0  Z   }

• That is, 1- is twice the area under the
SND-curve between 0 and  n /  .
• Hence,  n /  = z/2 – the
100(1-)-percentage point of the standard ND
variable. 
n


Area  1  
16
COMP233 Week 10

COMP233 Week 10 8
 11/6/2014

The Central Limit Theorem

(Recall)

• Let X1, X2, . . ., Xn be a sequence of independent

identically distributed (i.i.d.) random variables,
each having mean  and variance 2.

• Then, for large values of n, the distribution of

X = X1 + X2 + . . . + Xn
is approximately normal
with mean n and variance n2.

COMP233 Week 10 17

Implications of the
Central Limit Theorem
• For large n,
– The distribution of the sum of independent
identically distributed random variables is
normal although the variables themselves
need not be normally distributed.
– The distribution of the sample means is
approximately normal, with mean  and
variance 2/n.
• In many practical examples a sample of size 30
or more will be sufficient for the normal
approximation to work well. In some cases the
Central Limit Theorem will work even if n<30.
COMP233 Week 10 18

COMP233 Week 10 9
 11/6/2014

Example

• The president of a large university

wishes to estimate the average age
of students presently enrolled.
• From past studies, the standard deviation is known
to be 2 years. A sample of 50 students is selected,
and the sample mean is found to be 23.2 years.
• Find the 95% confidence interval of the population
mean .

COMP233 Week 10 19

Solution

• We need to find  such that

P{ X      X + }
= P{23.2      23.2 + }
= 0.95 = 1 − α
• Hence,   1  0.95  0.05
• Thus, we need to find z (a.k.a z/2) such that
P{Z  z}  1   / 2  0.975
/2 /2

20 COMP233 Week 10
/2

COMP233 Week 10 10
 11/6/2014

Solution

• From the standard normal distribution table,

we get
n
z / 2    1.96

50
Thus,   1.96
2
2
   1.96  0.6
50

COMP233 Week 10 21

Solution

• Hence, the 95% confidence interval of the

population mean is

( X  , X +  )
= (23.2  0.6, 23.2 + 0.6)
= (22.6, 23.8)

COMP233 Week 10 22

COMP233 Week 10 11
 11/6/2014

Summary

• Thus, “with 95 percent confidence” we assert that

the true mean lies within 1.96σ/ n of the
observed sample mean.
• The interval
   
 X  1.96 , X  1.96 
 n n
is called a 95 percent confidence interval estimate
of μ.

COMP233 Week 10 23

100(1 − α) percent two-sided

confidence intervals
• In general, a 100(1 − α) percent two-sided
confidence interval for μ is
   
 X  z / 2 , X  z / 2 
 n n
where X is the observed sample mean.
• E.g., a 99 percent confidence interval estimate of
μ is
   
 X  2.58 , X  2.58 
 n n

COMP233 Week 10 24

COMP233 Week 10 12
 11/6/2014

Example

• A college president wishes to estimate

the average age of students presently
enrolled.
• How large a sample is necessary if:
• The president would like to be 99% confident that
the estimate should be accurate within 1 year.
• From a previous study the standard deviation of
the ages is known to be 3 years.

COMP233 Week 10 25

Solution

• Here we are given the following:

1- = 0.99
=3
=1
• We would like to know the sample size, n, such
that
P{Z  z / 2 }  1   / 2  0.995,

where z / 2   n /   n / 3 .

COMP233 Week 10 26

COMP233 Week 10 13
 11/6/2014

Solution

• From the table z = 2.58.

n
• Thus,  2.58, or n = [(2.58)(3)]2 = 59.9.
3
• Which is rounded up to 60.

COMP233 Week 10 27

One-Sided Confidence
Intervals
X 
• Knowing that Z  n is a standard normal
random variable,  along with the identities
P{Z > zα} = α and P{Z < −zα} = α
results in one-sided confidence intervals of any
desired level of confidence. Specifically, we obtain
that   
  
 X  z ,   and   , X  z 
 n   n
are, respectively, 100(1 − α) percent one-sided
upper and 100(1 − α) percent one-sided lower
confidence intervals for μ.
COMP233 Week 10 28

COMP233 Week 10 14
 11/6/2014

Example

• Suppose that when a signal having value μ is

transmitted from location A the value received at
location B is normally distributed with mean μ and
variance 4.
• That is, if μ is sent, then the value received is μ +
N where N, representing noise, is normally
distributed with mean 0 and variance 4.
• To reduce error, the same value is sent 9 times.

COMP233 Week 10 29

Example, cont.
• Suppose the successive values received are
5, 8.5, 12, 15, 7, 9, 7.5, 6.5, 10.5
• Construct the following confidence intervals:
a) A 95 percent two-sided confidence interval
estimate of μ
b) A 99 percent two-sided confidence interval
estimate of μ
c) A 95 percent one-sided upper confidence
estimate of μ
d) And a 95 percent one-sided lower confidence
estimate of μ .
COMP233 Week 10 30

COMP233 Week 10 15
 11/6/2014

Solution

a) Since 81
X  9
9
It follows, under the assumption that the values
received are independent, that a 95 percent
confidence interval for μ is
 2 2 
 X  1.96 , X  1.96   (7.69, 10.31)
 9 9 
Hence, we are “95 percent confident” that the
true message value μ lies between 7.69 and 10.31.

COMP233 Week 10 31

Solution

b) Since 81
X  9
9
It follows, since z0.005 = 2.58, that a 99 percent
confidence interval for μ is
 2  2 
 X  2.58 , X  2.58    (7.28, 10 .72 )
 3  3 
Hence, we are “99 percent confident” that the
true message value μ lies between 7.28 and 10.72.

COMP233 Week 10 32

COMP233 Week 10 16
 11/6/2014

Solution
0.95 0.05

c) Since P{Z  1.645}  0.95 0.05

then z0.05 = 1.645, and the 95 percent one-sided

upper confidence interval for μ is
 2 
 X  1.645  ,    (7.903,  )
 3 
d) and the 95 percent one-sided lower confidence
interval for μ is
  2 
   , X  1.645     (  , 10 .097 )
  3 
COMP233 Week 10 33

Confidence Interval for a

Normal Mean (Variance
Unknown)

Reading: §7.3.1

COMP233 Week 10 17
 11/6/2014

Confidence Intervals on .

• Suppose that 1- = P{−z < Z < z }.

• Then the 1- two-sided confidence interval for 
is given by
   
1    P  X  z / 2    X  z / 2 
 n n

Area  1  

A Problem with
Estimating the Mean
• The above procedure only works if
 is known, and ܺ௜ are normally distributed
1.
or n is at least 30, or
2. Variance unknown, and n is at least 40 (we use
S instead of  in the previous slide).
• But what if  is unknown and n < 40?
– Which is often the case.

COMP233 Week 10 36

COMP233 Week 10 18
 11/6/2014

Search for a Suitable Statistic

• If σ is known, then a suitable statistic is

X 
Z
/ n
• But, if  is unknown, then suppose that we use S
instead of σ, and we get a statistic as follows.
X 
S/ n
• Now, we need to know the distribution of this
statistic…
COMP233 Week 10 37

Reminder: The Distribution of S2

• The random variable

(n  1) S 2
  n21
 2

is chi-square distributed with n-1 degrees of

freedom.

COMP233 Week 10 38

COMP233 Week 10 19
 11/6/2014

Reminder

• Hence,
S2  n21

 2
n 1
• Or,

S  n21

 n 1
COMP233 Week 10 39

Rewriting new statistic…

• An elementary transformation of the new statistic

in the case of unknown σ implies that

X 
X  / n Z
   Tn 1
S/ n S2  n21
2 n 1

COMP233 Week 10 40

COMP233 Week 10 20
 11/6/2014

Conclusion

• Hence, a suitable statistic to estimate the

population mean has a t-distribution with n -1
degrees of freedom.

X 
Tn 1 
S/ n

COMP233 Week 10 41

Confidence intervals for 

(unknown σ)
• Suppose 1 -  = P{-t/2,n-1, < Tn-1 < t/2,n-1 }.

Area  1  

COMP233 Week 10 42

COMP233 Week 10 21
 11/6/2014

Confidence intervals for .

• Then the two-sided (1--confidence interval on 

is given by

 S S 
1    P  X  t / 2,n 1    X  t / 2,n 1 
 n n
• I.e.,
1    P{ X      X   }
where S
  t / 2,n 1
n

COMP233 Week 10 43

Example

• Ten (10) randomly selected automobiles were

stopped, and the tread depth of the right front
tire was measured. The sample mean was 0.32 inch,
and the sample standard deviation was 0.08 inch.
Find the 95% confidence interval of the mean
depth. Assume that the variable is approximately
normally distributed.

COMP233 Week 10 44

COMP233 Week 10 22
 11/6/2014

Solution

• Here  is unknown, and n (= 10) is small (< 40).

• Use S (= 0.08) instead of .
• Use the table for T instead of Z.
•  = 0.05, d.f. = n  1 = 9.
• From the table,

n 10
t / 2, n 1  t0.025, 9  2.262    .
S 0.08

COMP233 Week 10 45

Solution

• Thus,
S 0.08
  t / 2, n 1  2.262   0.057.
n 10

• And the 95% confidence interval for  is

( X  ε, X + ε)
=(0.32  0.057, 0.32 + 0.057)
= (0.263, 0.377)

COMP233 Week 10 46

COMP233 Week 10 23
 11/6/2014

Example

• The data represent a sample of the number of

home fires started by candles for the past
several years. Find the 99% confidence interval
for the mean number of home fires started by
candles each year.

5460 5900 6090 6310

7160 8440 9930

COMP233 Week 10 47

Solution

• Assume that the number of home fires started by

candles each year to be approximately ND.
• Find the sample mean and sample standard
deviation.

X  7041.4
S  1610.3

COMP233 Week 10 48

COMP233 Week 10 24
 11/6/2014

Solution

•  = 0.01, d.f. = n  1 = 6.
• From the table,

n 7
t / 2, n 1  t0.005,6  3.707   
S 1610.3
• Thus,  = 2256.2.
• The 99% confidence interval is
( X  ε, X + ε)
= (7041.4  2256.2, 7041.4 + 2256.2)
= (4685.2, 9297.6)
COMP233 Week 10 49

Summary: 100(1 − α) percent

two-sided confidence intervals for μ
• In general, with unknown variance, i.e. σ2 is not known,
a 100(1 − α) percent two-sided confidence interval for
μ is
 S S 
 X  t / 2, n 1 , X  t / 2, n 1 
 n n

where X is the observed sample mean, and S is the

observed sample standard deviation.

COMP233 Week 10 50

COMP233 Week 10 25
 11/6/2014

One-Sided Confidence Intervals

• With the identities

P{T > tα,n-1} = α and P{T < −tα,n-1} = α
we can also define one-sided confidence intervals
for μ of any desired level of confidence:
 S   S 
 X  t ,n 1 ,   and   , X  t , n 1 
 n   n
are, respectively, 100(1 − α) percent one-sided
upper and 100(1 − α) percent one-sided lower
confidence intervals for μ.

COMP233 Week 10 51

Confidence Intervals for the

Variance of a Normal
Distribution

Reading: §7.3.2

COMP233 Week 10 26
 11/6/2014

Estimating the Variance

• Another parameter which often needs to be

estimated is the variance .
• Its unbiased estimator is the sample variance S2 .
• In order to construct an interval estimate on the
population variance we shall require a more
detailed analysis of S2.

COMP233 Week 10 53

Analysis of Sample Variance

• Let X1,…, Xn be independent identically

distributed Normal RV. We can see that

n n

(X
k 1
k
2
 X)  (X
k 1
k   ) 2  n( X   ) 2 ,

so
n n 2 2
1  Xk     X   
2

k 1
(Xk  X )  2

k 1 
   
   
 n 


COMP233 Week 10 54

COMP233 Week 10 27
 11/6/2014

Analysis of Sample Variance

• Standardization of a Normal Distribution implies

that
n
n 1 1
 2
2
S 
 2 (X
k 1
k  X )2

n
 Z
k 1
2
k Z2

COMP233 Week 10 55

Reminder: Distribution of the

Sample Variance
• We have stated earlier that the random variable

S2
(n  1)   n21
 2

is chi-square distributed with n-1 degrees of

freedom (not n).

COMP233 Week 10 56

COMP233 Week 10 28
 11/6/2014

Confidence Intervals on 2
• Suppose 1 -  = P {2left,n-1 ≤ 2 ≤ 2right,n-1}, where
2left,n-1 and 2right,n-1 are percentage points
corresponding to 

2 distribution
with 4 d.f.
Area   / 2

Area   / 2
Area  1  

2left,n-1 2right,n-1
COMP233 Week 10 57

Note
2 2 2
left, n-1   n 1   right, n-1

2 (n  1) S 2 2
 left, n-1  2
  right, n-1

1 2 1
 2
 
left, n-1 (n  1) S 2 2
 right, n-1

1 2 1
 2
 
 right, n-1 (n  1) S 2 2
left, n-1

(n  1) S 2 (n  1) S 2
2
2  2
 right, n-1 left, n-1

COMP233 Week 10 58

COMP233 Week 10 29
 11/6/2014

Confidence Intervals on 2

• Thus, a two-sided (1-)-confidence interval on 2

is given by
 (n  1) S 2 (n  1) S 2 
1    P 2   2
2

 /2,n-1 1/2,n-1 
• Hence a 100(1 − α) percent confidence interval
for σ2 is
 (n  1) S 2 (n  1) S 2 
 2   , 2 

 2
 /2,n-1  1/2, n-1 

COMP233 Week 10 59

One-Sided Confidence Intervals

on 2
• Similarly, we can also define one-sided confidence
intervals for σ2 of any desired level of confidence:
 (n  1) S 2   2
 ,   and  0, (n  1) S 
 2   2 
  , n-1   1 , n-1 
are, respectively, 100(1 − α) percent one-sided
upper and 100(1 − α) percent one-sided lower
confidence intervals for σ 2.

COMP233 Week 10 60

COMP233 Week 10 30
 11/6/2014

Example

• Find the 95% confidence interval on the variance

and the standard deviation of the nicotine content
of cigarettes manufactured.
A sample of 20 cigarettes has a sample standard
deviation of 1.6 milligrams. The nicotine content is
assumed to be normally distributed.

COMP233 Week 10 61

Solution

• d.f. = n  1 = 19
•  = 0.05
• Two relevant areas:
/2 = 0.025 (for 2right,n-1), and
1  /2 = 0.975 (for 2left,n-1).
• From the table,
2right,n-1 = 32.852, and
2left,n-1 = 8.907

COMP233 Week 10 62

COMP233 Week 10 31
 11/6/2014

Solution

• Thus, the 95% confidence interval for 2 is

 (n  1) S 2 (n  1) S 2 
 , 2 
  right,
2
left, n-1 
 n-1

 19(1.6) 2 19(1.6) 2 
  ,   1.5, 5.5
 32. 852 8 . 907 
And, the 95% confidence interval for 
  1.5, 
5.5  1.2, 2.3

COMP233 Week 10 63

Example 2

• Find the 90% confidence interval on the variance

and the standard deviation of the grades in a
data structures exam.
• The data represent a sample, and the grades are
assumed to be normally distributed.

59 54 53 52 51
39 49 46 49 48

COMP233 Week 10 64

COMP233 Week 10 32
 11/6/2014

Solution

• Find S 2 = 28.2.
• # of d.f. = n  1 = 9.
•  = 0.1.
• Two relevant areas:
/2 = 0.05 (for 2right,n-1), and
1  /2 = 0.95 (for 2left,n-1).
• From the table,
2right,n-1 = 16.919, and
2left,n-1 = 3.325.

COMP233 Week 10 65

Solution

• Thus, the 90% confidence interval for 2 is

 (n  1) S 2 (n  1) S 2 
 , 2 
  right,
2
 
 n-1 left, n-1 

 9(28.2) 9(28.2) 
 ,   15,76.3
 16 . 919 3 . 325 
And, the 90% confidence interval for 
  15, 
76.3  3.87,8.73

COMP233 Week 10 66

COMP233 Week 10 33
 11/6/2014

Example 3

• A standardized procedure is expected to produce

washers with very small deviation in their
thicknesses. Suppose that 10 such washers were
chosen and measured. If the thicknesses of these
washers were, in inches,
.123 .133 .124 .125 .126
.128 .120 .124 .130 .126
what is a 95 percent one-sided lower confidence
interval for the standard deviation of the
thickness of a washer produced by this procedure?

COMP233 Week 10 67

Solution

Area  1  
• Find S2 = 1.366x10-5.
• # of d.f. = n  1 = 9.
•  = 0.05.
• The relevant area: 2left,n-1
1- = 0.95 (for2left,n-1 since we want the one
sided lower confidence interval, so the lower
bound is 0 and the upper bound is a function
of1/2left,n-1).
• From the table,
2left,n-1 = 21-,n-1 = 20.95, 9 = 3.325.

COMP233 Week 10 68

COMP233 Week 10 34
 11/6/2014

Solution

• Thus, the 95% one-sided lower confidence interval

for 2 is
 
 (n  1) S 2   9(1.366  10 5 ) 
 0, 2    0,
 left, n-1   3.325 

 0, 36.974  10  6 
  

• And, the 95% one-sided lower confidence interval

for  is




   0, 36.974  10 6   0, 6.081  10 3


COMP233 Week 10 69

100(1-α) Confidence Intervals

X 1 ,, X n ~ N (  ,  2 )

 X  X  /( n  1)
n n
X   X i / n,
2
S i
i 1 i 1

Assumption Parameter Confidence Interval


σ2 known μ X  z / 2
n
S
σ2 unknown μ X  t / 2,n-1
n
 (n  1) S 2 (n  1) S 2 
μ unknown σ2  , 2 
 2 
 /2, n-1 1/2, n-1 
COMP233 Week 10 70

COMP233 Week 10 35
 11/6/2014

Example

• A manufacturer claims that television plasma

screens have an average lifetime of 6.1 years.
• We want to test this claim, and either accept or
reject it.
• Suppose for certainty that a random sample
produced a sample mean of 5.9 years.

– How should we approach this problem?

COMP233 Week 10 71

Future Plans

• Next week we are going to see how confidence

intervals techniques allow one to give more precise
answers about population parameters.
• This is only possible if there is a conjecture
formulated on the basis of additional
considerations.
• The statement is checked on consistence with a
sample data.
• So the next lecture is on
TESTING OF STATISTICAL HYPOTHESES.

COMP233 Week 10 72

COMP233 Week 10 36
 11/6/2014

References/Resources Used

• Lecture Slides for MATH 401 of Dr. Oleksiy Us,

Department of Mathematics, German University of
Cairo. [PPT]

COMP233 Week 1 73

COMP233 Week 10 37