Math 2274 Lecture 20 Distributions of Sample Statistics, Chapter 6 (Lecture notes)

In reality, it may be too time-consuming and/or expensive to take measurements/observations

on every single element in a population (census). This introduces the concept of a random
sample which is a representative sample of a fixed number of elements from the population.

Definition 1: A simple random sample (s.r.s.) of size n from a finite population of size N is a
subset of n elements chosen from the entire population in such a way that each subset of n
elements in the population has the same chance of being chosen.

For a large population, the measurements made on sample elements are approximated using
independent and identically (i.i.d) distributed random variables.

Definition 2: A r.s. of size n from a population with distribution F is a sequence X1, X2, ...,Xn of
i.i.d random variables such that Xi ~ F for i = 1,2,...,n.

In order to describe the characteristics of a population in terms of the values X1, X2, ...,Xn from a
r.s., there exists a number of ways of summarizing this information. Quantities whose values
depend on the outcome of the r.s., that is, they are functions of the r.s., are called statistics e.g.
sample mean (), sample variance (S2) , sample median (), trimmed mean, quartiles etc.

Definition 3: Let X = (X1, X2, ...,Xn) be a r.s. of size n from a population. A random variable that
is a function of X is called a statistic.

Since a statistic is itself a random variable, it has a distribution, mean and variance.

Sample Mean Let X1, X2, ...,Xn be i.i.d F (each Xi has distribution of F and Xis are
independent for i = 1,...,n), where F is the distribution of the population. The sample mean is
1
defined as = =1 .

Theorem 1: Let X1, X2, ...,Xn be an i.i.d r.s. from a population with finite mean and finite
variance 2. Then,
2

i) E( ) = ii) Var( ) =

Proofs: See pg. 137, Lecture notes.

Note: a) E(Xi) = , Var(Xi) = 2 for i = 1,...,n.

b)The sample mean has an average value which is approximately the population mean.

c) The variance of the sample mean decreases as the sample size increases.

Using the theory of linear combinations of random variables, the following theorem arises.
 2
Theorem 2: If X1, X2, ...,Xn are i.i.d N(, 2) then ~N(, ) .

Proof: From previous theorem, X1, X2, ...,Xn are i.i.d and normal. Hence, every linear
combination of X1, X2, ...,Xn is also normal etc.

Sample Variance The sample variance, S2 measures the spread of the observations in the
sample. It is representative of the variance of the distribution or population.

Definition 4: Let X1, X2, ...,Xn be i.i.d random variables from a population with mean and
finite variance 2. The sample variance, Sn2 is defined to be
1
Sn2= 1 =1( )2

1 1
Question: Show that Sn2= 1 =1( )2 1 (=1 2 2 )

Theorem 3: Let X1, X2, ...,Xn be i.i.d random variables from a a population with mean and
finite variance 2. Then E(Sn2) = 2.

Proof: See pg. 139, Lecture notes.

Convergence in distribution Chapter 7, Lecture notes.

A sequence of random variables X1, X2, ...,Xi, ..., such that the distribution function of Xi is
Fi (i=1,2,...) converges in distribution to a random variable X with df. F if Fi(x) F(x), .

Central Limit Theorem (C.L.T):

Let X1, X2, ...,Xi, ..., Xn be i.i.d random variables with mean and variance 2. Then,

N(0,1). Therefore, for large n, N(0,1).

Corollary 4: Let X1, X2, ...,Xi, ..., Xn be i.i.d random variables with mean and variance 2. Then,
if n is sufficiently large:
2
) (, ) ii) =1 (, 2 )

Corollary 5: Let X be such that X ~ Binomial (n,p), where 0<p<1. Then, if n is sufficiently large
X N(np, np(1-p).

Proof: See pg. 143, Lecture notes.