Sunteți pe pagina 1din 3

Sean Li CS 4850 Notes Spring 2013 Mathematical Foundations for the Information Age Lecture 1 1/21/13 Professor: John

hn Hopcroft 5144 Upson Hall, jeh@cs.cornell.edu. Homework: Two problems every lecture. 1. Select one of 2.1, 2.2, 2.3, or 2.4. 2. Central limit theorem If random variable does not have a nite variance, how do you estimate the mean? High Dimensional Data. Nonintuitive in dimensions higher than 3. Example. Distance between pairs of random points in n dimensions.
n

d(x, y) =
i=1

(xi yi )2 .

Law of Large Numbers. The average of n independent samples is close to the expected value. Central Limit Theorem. Let x be a random variable with nite variance, and let P (x) be the probability. Note that the expected value E(x) is

E(x) =

xP (x)dx.

The variance 2 = E[(x m)2 ], where m is the mean. Suppose we have a sample x1 , . . . , xn . Then x1 + + xn 2 P m 2. n n Markovs Lemma. Let x be a nonnegative random variable. Then P (x a) E(x) . a

Proof. We have
a

E(x) =
0

xP (x)dx =
0

xP (x)dx +
a

xP (x)dx.

This implies E(x)


a

xP (x)dx a
a

P (x)dx,

from which the lemma immediately follows. Corollary. P (x cE(x)) = E(x) 1 = . cE(x) c 1 . a2

Chebyshevs Inequality. P (|x m| a)

Proof. Squaring removes the absolute value, so we have P (|x m| a) = P [(x m)2 a2 2 ] by Markovs lemma. Proof of Central Limit Theorem. Recall that E(x1 + + xn ) = E(x1 ) + + E(xn ), which does not require independence. Now assume x has mean 0 (can be shifted), then 2 (cx) = c2 E(x2 ) = c2 2 (x), and 2 (x + y) = E[(x + y)2 ] = E(x2 ) + 2E(xy) + E(y 2 ). Assuming independence, we have E(x2 ) + 2E(xy) + E(y 2 ) = E(x2 ) + 2E(x)E(y) + E(y 2 ) = 2 (x) + 2 (y) as E(x) = E(y) = 0. This can be generalized to n variables that are pairwise independent. Now by Chebyshevs inequality, P x1 + + xn m n 2( 1 2 = a
x2 ++x2 n 1 ) n , 2

1 E[(x m)2 ] = 2 22 a a

where = a, so a = . But 2(
x2 ++x2 n 1 ) n 2

1 2 (x2 + + x2 ) 1 n 2 (x) 2 (x) 1 n = 2 = , 2 2 n2 n n2

which proves the theorem.


Page 2

High Dimensional Unit Spheres. The volume of an n-dimensional sphere converges to 0. What about n-dimensional Gaussian distributions? Since volume of the sphere goes to 0, almost all the mass is on an annulus of radius d. Almost all the surface area is on the equator, so we have that the distance between two random points is near 2d.

Page 3

S-ar putea să vă placă și