PROBABILITY

Contents
1. Integration - The abstract integral
1.1. Lp -Spaces
1.2. Modes of Convergence
1.3. Limit Theorems
1.4. The Radon-Nikodyn Theorem
2. Product Spaces
2.1. Finite products

1
1
2
4
6
7
7

1. Integration - The abstract integral

1.1. Lp -Spaces. (, F, P) is a probability space. We can define for all p (0, )
and all random variables X : R,
(1)

kXkp = (E{|X|p })1/p

provided that the integral exists-i.e that the |X|p is P-integrable.

Definition 1.1. The space Lp (P ) is the collection of all random variables X :
R that are p times P -integrable. More precisly, these are the random variable
X such that |X|p < .
Remark 1.2. More generally, if (, F, ) is a -finite measure space, then Lp ()
will denote the collection of all measurable functions f : R such that kf k < .
Occasionally we write respectively kf kLp () and Lp (, F, ) in place of kf kp and
Lp () in order to emphasise that the underlying measure space if (, F, ).
Theorem 1.3. Properties of Lp spaces
(1) Lp () is a linear space. That is, kaf kp = |a| kf kp for all a R and
f Lp (), and f + g Lp () if f, g Lp ().
(2) Holder inequality - If p > 1 and p1 + q 1 = 1, then kf gk1 kf kp kgkq for
all f Lp (), g Lq ().
(3) Mikowskis Inequality. If p 1, then kf + gk kf kp + kgkp for all f, g
Lp ().
Definition 1.4. Cauchy-Schwartz-Bunyakovsky Inequality
For all f, g L2 ()
1

PROBABILITY

Z
|

f gd| kf k2 kgk2

Definition 1.5. A function : R R is covex if

(2)

(x + (1 )y) (x) + (1 )(y)

Jensen Inequality. Suppose is a probability measure. If : R R is

convex and (f ) and f are integrable, then
Z
Z
(f )d ( f d)
Example 1.6. Since
(x) =R |x| is convex. Jensens inequality extendsR the triR
angleR inequality: |f |d | f d|. Another example is the inequality ef d
exp( f d), valid because (x) = ex is convex.
Proposition 1.7. If () < and r > p > 1, then Lr () Lp (). In fact,
(3)

kf kp [()] p r kf kr

Fix some p 1 and define d(f, g) = kf gkp for all f, g Lp (). According to
Minkowskis inequality, d has the following properties
(1) d(f, f ) = 0;
(2) d(f, g) d(f, h) + d(h, g); and
(3) d(f, g) = d(g, f ).
In other words, if it were the case that d(f, g) = 0 f = g, then d(, ) would
metrize Lp (). Unfortunately, the latter property does not hold in general. For
an example consider g = f 1A where A 6= and (Ac ) = 0. Evidently then g 6= f
but d(f, g) = 0.
Nonetheless, if we can identify the elements of Lp () that are equal to each other
outside a null set, then the resulting collection of equivalence classes - endowed
with the usual quotient topology and Borel -algebra is indeed a metric space. It
is also complete; i.e., every Cauchy sequence converges.
Theorem 1.8. Let (, F, ) denote a -finite measure space. For any f, g
Lp (), write f g iff f = g a.e. That is, ({ : f () 6= g()}) = 0. Then
is an equivalent relation on Lp (). Let [f ] denote the -orbit of f ; i.e, f [f ] iff
f g. Let Lp () = {[f ] : f Lp ()} and define k[f ]kp = kf kp . Then, Lp () is a
complete normed linear space. Moreover L2 () is a Hilbert space.
1.2. Modes of Convergence. Throughout, (, F, ) is a measure space, and
f, f1 , f2 , : R.
Definition 1.9. We say that fn converges to f -almost everywhere (written
a.e., a.e.[] or even a.e.) if

PROBABILITY

(4)

({ : limsup|fn () f ()| > 0}) = 0

n

We write {f A} for { : f () A} and {f A} for ({f A}). In

this way, fn converge to f a.e iff {fn 6 f } = 0. When (, (F ), P) is a probability
space and X, X1 , X2 , . . . are random variables on this space, we say instead that
Xn converges to X almost surely (written a.s.).
Definition 1.10. We say that fn f in Lp () if lim kfn f kp = 0. Also,
n

fn f in measure if lim {|fn f | } = 0 for all  > 0. If X, X1 , X2 , . . . are

n

random variables on the probability space (, F, P), the we say that Xn converges to
X in probability when Xn X in P -measure; that is, if limn P {|Xn X| } = 0
P
for all  > 0. We write this as Xn X.
Theorem 1.11. Either a.e. convergence or Lp -convergence implies convergence in
measure. Conversely, if sup|fj | 0 in measure, then fn 0 almost everywhere.
jn

Before presenting the proof, we need some concepts.

Definition 1.12. Markovs inequality If f L1 (), then for all > 0,
(5)

1
{|f | }

Z
|f |d
{f }

kf k1

R
R
Proof. Set = {|f | } and note that |f |d d = (). This yields
the first inequality. The second is obvious.

We can apply the preceding to the function |f |p to deduce
Definition 1.13. Chebyshevs inequality For all p, > 0 and f Lp (),
(6)

1
{|f | } p

Z
{|f |}

|f |p d

kf kpp
p

Proof. Theorem 1.11

By Chebyshev inequality, Lp ()-convergence implies convergence in measure. In
order to prove that a.e.-convergence implies convergence in measure we first need
to understand a.e.-convergence a little better.

Note that fn f a.e. if and only if (

N =1 n=N {|fn f | }) = 0 for all
 > 0. Since is continuos,
(7)

fn f a.e. lim (
n=N {|fn f | }) = 0
N

PROBABILITY

Because {|fN f | } (nN {|fn f | }, if fN f a.e. then fN f in

measure. Finally, if sup|fj | 0 in measure then
jn

(8)

(
N =1 n=N {sup|fj | }) = lim ({sup|fj | }) = 0
N

jn

jN

Thus, the lim sup|fm | lim supsup|fj | <  a.e. []. If N () denotes the set of
m

jn

s for which this inequality fails, then Q+ N () is a null set of which lim|fm | < 
m

for every rational  > 0 i.e., off Q+ we have lim|fm | = 0.

m

1.3. Limit Theorems.

R
(1) integration is a positive operation (i.e. if f 0 then f d 0);
(2) it is a linear operation
Theorem 1.14. Bounded Convergence.
Suppose f1 , f2 , . . . are measurable functions on (, F) such Rthat supnR| fn | is
bounded by a constant K. If fn f in measure [], then lim fn d = f d.
n

Lemma 1.15. Fatous Lemma. If {fi }

i=1 is a collection of non negative integrable functions on (, F, ), then
Z
Z
lim inf fn d lim inf fn d
n

Proof. Let gn = inf fj and observe that gn f = lim infk fk asn . In

n

jn

particular, for any constant K > 0, (f K gn K) is a bounded measurable

function that converges to 0 as n . Because gn fn , the bounded convergence
theorem implies that
Z
Z
Z
lim inf fn d lim )gn K)d = (f K)d.
n

Therefore, it suffices to prove that

Z
(9)

lim

Z
(f K)d =

f d.

For all  > 0 we can find a simple function S such that:

(1) 0 S f ;
(2) Rthere exists
R C > 0 such that
R S() C; R
R
R
(3) Sd f d .N ow (f K)d (S K)d = Sd f d 
if K > C. This proves (9)


PROBABILITY

Theorem 1.16. Monotone Convergence Theorem

Suppose {fn }
n=1 is a sequence of non-negative integrable functions on (, F, )
such that fn (x) fn+1 (x) for all n 1 and x , and f (x) = lim fn (x) exists
n
R
R
for all x . Then lim fn d = f d.
n
R
R
Proof. by monotonicity L = lim fn d exists and is f d. Apply Fatous
n
Lemmato deduce the complementary inequality.

Theorem 1.17. The Dominated Convergence Theorem. Suppose {fi }
i=1
is a sequence of measurable functions on (, F) such that sup | fm | is integrable
m
R
R
[d]. Then, lim fn d = lim fn d provided that f (x) = lim fn (x) exists for
n
n
n
all x .
Proof. Thanks to Fatous Lemma f L1 (). Also, F = sup | fi | L1 () by
i1

assumpiton. Apply Fatous Lemma

to the non-negative function gn = 2F |
R
fn f | to deduce that lim | fn f | d = 0. The dominate convergence
n
theorem follows from this and the bound
Z
(10)

Z
fn d

Z
f d |

| fn f | d,

which is merely the triangle inequality for integrals.


Proof. Theorem 1.8
The fact that Lp (), and hence Lp (), is a linear space has been established in
Theorem (1.3). From the above we know that d(f, g) = kf gkp is a norm as
soon as we prove that d(f, g) = 0 [f ] [g]. In order to establish completeness
p
suppose that {fn }
n=1 is a Cauchy sequence in L (). It sufficies to show that fn
p

converges in L (). Recall that {fn }n=1 is Cauchy means that lim kfn f kp
m,n

k
0. Thus, we can
Pfind a subsequence {nk }k = 1 such that kfnk +1 f nk kp 2 .
Consequently, k kfnk +1 fnk kp <
Thanks to Minkoskis inequlity and the
P.

monotone
convergence
theorem,
k
k=1 | (fnk +1 fnk | kp < . In particular,
P
fnk ) converges -almost everywhere.
k (fnk +1 P
If f = k (fnk +1 fnk ) then f Lp () by Fatous Lemma. By the triangle
inequality for Lp -norms,

(11)

kf fnk kp

X
j=k+1

as k .
Minkoskis inequality implies that

kfnk +1 fnk kp 0

PROBABILITY

(12)

kf fN kp kf fnk kp + kfnk fN kp

Therefore we can let N and k tend to to see that fn f in Lp ().


By the Holder inequality hf, gi =
is a Hilbert space.

f gd is an inner product. Therefore, L2 ()

1.4. The Radon-Nikodyn Theorem. Given two measures andRv, When can
we find a function such that for all measurable sets A, v(A) = A d. If

is Lebesgue measure, then the

R function denotes a probability density function,
and the prescription v(A) = A d defines the probability measure v.
Definition 1.18. Given two measures and v on (, F), we say that v is absolutely continuous with respect to (written v << ) if v(A) = 0 for all A F
such that (A) = 0.
1
R Suppose that (, F, ) is a finite measure space and f L (). Then v(A) =
f d defines a finite measure, and v << .
A

Theorem 1.19. Radon-Nikodyn Theorem If v << are two

R finite measures
R
on (, F), then there exists a non-negative L1 () such that f dv = f d
for all bounded measurable functions f : R. Furthermore, is unique up to
a -null set.
Remark 1.20. Frequently we write : dv/d. The function is called the
Radon-Nikodyn derivative of v with respect to . For all bounded measurable f :
R
Z
Z
(13)
f dv = f (dv/d)d
1
Remark 1.21.
R Suppose (, F, ) is a measure space, f L () is non negative
and v(E) = E f d for all E F. Then v is a measure also (dv/d) = f a.e. [].
Furthermore, if kf k1 = 1 then v is probability measure with density function f .

PROBABILITY

2. Product Spaces
If A1 and A2 are sets, then their product A1 A2 is defined to be the collection of
all ordered pairs (a1 , a2 ) where a1 A1 and a2 A2 . We can have infinite-product
spaces of the type A1 A2
2.1. Finite products. Suppose (1 , F1 , 1 ) and (2 , F2 , 2 ) are two finite measure spaces. The is a natural -algebra F1 F2 and a measure 1 2 that
corresponds to the product set 1 2 .
First consider the collection
A0 = {A1 A2 : A1 F1 , A2 F2 }

(14)

We add to this all finite disjoint unions of elements of F0 and call the resulting
collection A.
Lemma 2.1. The collection A is an algebra, and (A) = (A0 ).
Definition 2.2. We write F1 F2 in place of (A)
Define on A0 as:
A1 F1 , A2 F2
Pn
i
i
If A1 , A2 , A0 are disjoint, then we define (
i=1 A ) =
i=1 (A ). This
j
constructs on the algebra A in a well-defined manner. Suppose ni=1 Ai = m
j=1 B
i
j
n
i
n
m
where the A s are disjoint and the B s are also disjoint. Then i=1 A = i=1 j=1
(Ai B j ) is a disjoint union of nm sets. Therefore,
n X
m
X
j
(16)
(ni=1 Ai ) =
(Ai B j ) = (m
j=1 B ),

(15)

(A1 A2 ) = (A1 )2 (A2 )

i=1 j=1

by symmetry.
Theorem 2.3. There exists a unique measure 1 2 on (1 2 , F1 F2 ) such
that 1 2 = on A.
Definition 2.4. The measure mu1 2 is called the product measure of 1 and
2 ; the space 1 2 is the corresponding product space, and F1 F2 is the
product -algebra. The measure space (1 2 , F1 F2 , 1 2 ) is the product
measure space.
Remark 2.5. By induction, we can construct a product measure space (, F, )
based on any finite number of measure space i , Fi , i , i = 1, . . . , n : Define =
1 2 n , F = F1 F2 Fn and = 1 2 n .
Proof. Theorem 2.3