Use of moment generating

functions

Definition
Let X denote a random variable with probability
density function f(x) if continuous (probability mass
function p(x) if discrete)
Then
mX(t) = the moment generating function of X

E etX

etx f x dx if X is continuous

e p x

tx

if X is discrete

The distribution of a random variable X is described by

either
1. The density function f(x) if X continuous (probability mass function p(x) if
X discrete), or
2. The cumulative distribution function F(x), or
3. The moment generating function mX(t)

Properties
1. mX(0) = 1
2. mXk 0 k th derivative of mX t at t 0.

k E X

k E X
3.

x f x dx
x p x
k

X continuous
X discrete

k k
2 2 3 3
mX t 1 1t t t L t L .
2!
3!
k!

4. Let X be a random variable with moment

generating function mX(t). Let Y = bX + a
Then mY(t) = mbX + a(t)
= E(e [bX + a]t) = eatmX (bt)
5. Let X and Y be two independent random variables
with moment generating function mX(t) and mY(t) .

Then mX+Y(t) = mX (t) mY (t)

6. Let X and Y be two random variables with

moment generating function mX(t) and mY(t)
and two distribution functions FX(x) and
FY(y) respectively.
Let mX (t) = mY (t) then FX(x) = FY(x).
This ensures that the distribution of a random
variable can be identified by its moment
generating function

M. G. F.s - Continuous distributions

M. G. F.s - Discrete distributions

Moment generating function of the

gamma distribution

mX t E etX

tx
e
f x dx

where

1 x
x e

f x

x0
x0

e f x dx

mX t E e

tX

tx

1 x
e
x e dx

0

1 t x

x e
dx

0
tx

using

or

b a a 1 bx
0 a x e dx 1

a
a 1 bx
0 x e dx ba

then

1 t x
mX t
x e
dx

Moment generating function of the

Standard Normal distribution

mX t E etX
where

tx
e
f x dx

1
f x
e
2

x2

thus

mX t

tx

1
e
2

x2

dx

1
e
2

x2
tx
2

dx

We will use

mX t

t2
2

1
e
2

1
e
2 b

1
e
2

x2
tx
2

x 2 2 tx

1
e
2

2
xa

2 b2

dx 1

dx

dx

x 2 2 tx t 2

t2
2

e dx e

t2
2

1
e
2

2
x t

dx

Note:

2
3
4
x
x
x
e x 1 x L
2! 3! 4!
2

t t

t2
2
2 2
t

2
mX t e 1

L
2
2!
3!
2

2m

t
t
t
t
1 2 3 L m L
2 2 2! 2 3!
2 m!
Also

2 2 3 3
mX t 1 1t t t L
2!
3!

Note:

2
3
4
x
x
x
e x 1 x L
2! 3! 4!
2

t t

t2
2
2 2
t

2
mX t e 1

L
2
2!
3!
2

Also

2m

t
t
t
t
1 2 3 L m L
2 2 2! 2 3!
2 m!
2 2 3 3
mX t 1 1t t t L
2! 3!

k k th moment

x k f x dx

Equating coefficients of tk, we get

k 0 if k is odd and
2 m
1
for k 2m then m
2 m ! 2m !
hence 1 0, 2 1, 3 0, 4 3

Using of moment generating

functions to find the distribution of
functions of Random Variables

Example
Suppose that X has a normal distribution with
mean and standard deviation .
Find the distribution of Y = aX + b
Solution:
2t 2
t
mX t e 2
2
2 at

maX b t e mX at e e
bt

bt

at

2 a 2t 2
a b t
2

= the moment generating function of the normal

distribution with mean a + b and variance a22.

Thus Y = aX + b has a normal distribution with

mean a + b and variance a22.
Special Case: the z transformation

X 1

Z
X
aX b

1

Z a b
0

2
1
2
2 2
Z a 2 1

Thus Z has a standard normal distribution .

Example
Suppose that X and Y are independent each having a
normal distribution with means X and Y , standard
deviations X and Y
Find the distribution of S = X + Y

Solution:
mX t e
mY t e

X2 t 2
X t
2
Y2 t 2
Y t
2

Now

mX Y t mX t mY t e

X2 t 2
X t
2

Y2 t 2
Y t
2

or

m X Y t e

X X

2
X

Y2 t 2
2

= the moment generating function of the

normal distribution with mean X + Y and

2
2
variance X Y

Thus Y = X + Y has a normal distribution

2
2
with mean X + Y and variance X Y

Example
Suppose that X and Y are independent each having a
normal distribution with means X and Y , standard
deviations X and Y
Find the distribution of L = aX + bY

Solution:
mX t e

X t

X2 t 2

mY t e

Now

Y2 t 2
Y t
2

maX bY t maX t mbY t mX at mY bt

e

X2 at
X at
2

Y2 bt
Y bt
2

or

t
2

maX bY t e

a X b X

2
X

b 2 Y2 t 2
2

= the moment generating function of the

normal distribution with mean aX + bY

2 2
2 2
and variance a X b Y

Thus Y = aX + bY has a normal

distribution with mean aX + BY and
2 2
2 2
variance a X b Y

Special Case:
a = +1 and b = -1.
Thus Y = X - Y has a normal distribution
with mean X - Y and variance

X 1 Y X Y
2

Example (Extension to n independent RVs)

Suppose that X1, X2, , Xn are independent each having a
normal distribution with means i, standard deviations i
(for i = 1, 2, , n)
Find the distribution of L = a1X1 + a1X2 + + anXn

Solution:
mX i t e
Now

i2t 2
i t
2

(for i = 1, 2, , n)

ma1 X1 L an X n t ma1 X1 t L man X n t

mX1 a1t L mX n ant
e

12 a1t
1 a1t
2

L e

n2 an t
n an t
2

or

ma1 X1 L an X n t e

a11 ... an n

2 2
2 2
1 1 ... an n

= the moment generating function of the

normal distribution with mean a11 ... an n
2 2
2 2
and variance a1 1 ... an n
Thus Y = a1X1 + + anXn has a normal
distribution with mean a11 + + ann and
2 2
2 2
a1 1 ... an n
variance

Special case:
1
a1 a2 L an
n
1 2 L n

12 12 L 12 2
In this case X1, X2, , Xn is a sample from a
normal distribution with mean , and standard
deviations and
1
L X1 X 2 L X n
n

X the sample mean

Thus

Y x a1 x1 ... an xn

n x

x1 ... 1

has a normal distribution with mean

x a11 ... an n
1 ... 1
n
n
and variance

x2 a12 12 ... an2 n2

2
...
n

1
1
2
n
n
n

2
n

Summary
If x1, x2, , xn is a sample from a normal
distribution with mean , and standard
deviations then x the sample mean
has a normal distribution with mean

x
and variance

n
2
x

standard deviation x
n

Sampling distribution
of x

Population

The Central Limit theorem

If x1, x2, , xn is a sample from a distribution
with mean , and standard deviations then
if n is large x the sample mean
has a normal distribution with mean

x
and variance

n
2
x

standard deviation x
n

Proof: (use moment generating functions)

We will use the following fact:
Let
m1(t), m2(t),
denote a sequence of moment generating functions
corresponding to the sequence of distribution
functions:
F1(x) , F2(x),
Let m(t) be a moment generating function
corresponding to the distribution function F(x) then
if
lim mi t m t for all t in an interval about 0.
i

then lim Fi x F x for all x.

i

Let x1, x2, denote a sequence of independent

random variables coming from a distribution with
moment generating function m(t) and distribution
function F(x).
Let Sn = x1 + x2 + + xn then
mSn t mx1 x2 L xn t =mx1 t mx2 t L mxn t
= m t

x1 x2 L xn S n
now x

n
n
t
t
or mx t m 1 t mSn m
Sn
n
n
n

x
n
n
Let z

then mz t e

and ln mz t

nt
mx

nt
m

t
n

t n ln m

t
t2
Let u
or n
and n 2 2
u
u
n
t

Then ln mz t

t
n

t n ln m

n
t 2
t2
2 2 2 ln m u
u u
t 2 ln m u u
2

u2

Now lim ln mz t
n

lim ln
u 0

mz t

ln m u u
t2
2 lim
u 0
u2

t2
2 lim
u 0

m u

m u
2u

using L'Hopital's rule

m u m u m u
2

t
2 lim
u 0

m u
2

using L'Hopital's ru

m u m u m u
m u

t
2 lim
u 0

t m 0 m 0
2

using L'Hopital's rule again

t E x E xi
2

2
2

2
i

thus lim ln mz t
n

t2

2
2

and lim mz t e
n
2

t2
2

Now m t e

t2
2

Is the moment generating function of the standard

normal distribution
Thus the limiting distribution of z is the standard
normal distribution
i.e. lim Fz x
n

1
e
2

u2

du

Q.E.D.