Module 14

Expectation of a Random Variable

() Module 14 Expectation of a Random Variable 1 / 19

 X : a given r.v. with d.f. FX (·) and p.m.f./p.d.f. fX (·).
Definition 1:
(a) Let X be a discrete r.v. with support SX and p.m.f. fX (·).
We say that the expected value of X (denoted by E (X ))
exists and equals
X
E (X ) = x fX (x),
x∈SX
P
provided x∈SX |x|fX (x) < ∞.
(b) Let X be an A.C. r.v. with p.d.f. fX (·). We say that the
expected value of X (denoted by E (X )) exists and equals
Z ∞
E (X ) = x fX (x)dx,
−∞
R∞
provided −∞ |x|fX (x)dx < ∞.

() Module 14 Expectation of a Random Variable 1 / 19

Result 1:
Let X be a discrete or A.C. r.v. Then
Z ∞ Z 0
E (X ) = P({X > y })dy − P({X < y })dy ,
0 −∞

provided the expectation exists.

Proof: For A.C. case (the proof for discrete case follows similarly).
Z ∞ Z 0 Z ∞
E (X ) = x fX (x)dx = x fX (x)dx + x fX (x)dx
−∞ −∞ 0
Z 0 Z 0 Z ∞Z x
= − fX (x)dydx + fX (x)dydx
−∞ x 0 0
Z 0 Z y Z ∞Z ∞
= − fX (x)dxdy + fX (x)dxdy
−∞ −∞ 0 y
Z 0 Z 0
= − P({X < y })dy + P({X > y })dy .
−∞ ∞

() Module 14 Expectation of a Random Variable 2 / 19

Corollary 1:

(a) Let X be a discrete or an A.C. r.v. with P({X ≥ 0}) = 1.

Then
Z ∞ Z ∞
E (X ) = P({X > y }) = (1 − FX (y ))dy .
0 0

(b) If X is discrete with P({X ∈ {0, 1, 2, ....}}) = 1, then

∞
X
E (X ) = P({X ≥ k}).
k=1

() Module 14 Expectation of a Random Variable 3 / 19

Result 2:
Let X be a discrete or an A.C. r.v. with p.m.f./p.d.f. fX (·) and support
SX . Let Y = g (X ), for some function g : R → R. Then
 P
 x∈SX g (x)fX (x), if X is discrete
E (Y ) = E (g (X )) = .
 R∞
−∞ g (x)fX (x)dx, if X is A.C.
Proof: (For discrete case.) Let fY (·) be the p.m.f. of Y and let
SY = g (SX ) = {g (x) : x ∈ SX }. Then, clearly, SY is the support of Y and
X
E (Y ) = y fY (y )
y ∈SY
X
= y P({h(X ) = y })
y ∈SY
X X
= y fX (x)
y ∈SY x∈SX
h(x)=y

() Module 14 Expectation of a Random Variable 4 / 19

 X X
= y fX (x)
y ∈SY x∈SX
h(x)=y
X X
= h(x)fX (x)
y ∈SY x∈SX
h(x)=y
X
= h(x)fX (x).
x∈SX

() Module 14 Expectation of a Random Variable 5 / 19

Example 1:

Let X be a r.v. with p.m.f.

 cp
 xp , if x = 1, 2, . . .
fX (x) = ,
0, otherwise


where p > 1 is a given constant. Then SX = {1, 2, . . .} and

X
E (|X |r ) = |x|r fX (x)
x∈SX
∞
X 1
= cp
x p−r
x=1

is finite iff r < p − 1. Thus E (X r ) exists iff r < p − 1.

() Module 14 Expectation of a Random Variable 6 / 19

In particular E(X) exists iff p > 2. For p > 2
X
E (X ) = x fX (x)
x∈SX
∞
X 1
= cp .
x p−1
x=1

() Module 14 Expectation of a Random Variable 7 / 19

Example 2:
Let X be a r.v. with p.d.f.
1 1
fX (x) = · , −∞ < x < ∞.
π 1 + x2
Then
Z ∞
E (|X |) = |x|fX (x)dx
−∞
∞
|x|
Z
1
= dx
π −∞ 1 + x2
Z ∞
2 x
= dx
π 0 1 + x2

= ∞.

Thus E (X ) does not exists.

() Module 14 Expectation of a Random Variable 8 / 19
Example 3:

Let X be a r.v. with p.d.f.


 2x, if 0 < x < 1
fX (x) = .
0, otherwise


Then
Z ∞
E (X 3 ) = x 3 fX (x)dx
−∞
Z 1
= 2 x 4 dx
0

2
= .
5

() Module 14 Expectation of a Random Variable 9 / 19

Result 3:

Let X be a discrete or an A.C. r.v. and let gi : R → R, i = 1, ..., k, be

such that E (gi (X )) exists.
(a) If P({g1 (X ) ≤ g2 (X )}) = 1 then E (g1 (X )) ≤ E (g2 (X )). In
particular if P({a ≤ X ≤ b}) = 1, for some real constants a
and b, then a ≤ E (X ) ≤ b.
(b) If P({X ≥ 0}) = 1 and E (X ) = 0 then P({X = 0}) = 1.
(c) If E (X ) exists then |E (X )| ≤ E (|X |).
(d) For real constants c1 , . . . , ck
k k
!
X X
E ci gi (X ) = ci E (gi (X )) .
i=1 i=1

() Module 14 Expectation of a Random Variable 10 / 19

Proof :
(a) (For discrete case.) Let A = {x ∈ R : g1 (x) ≤ g2 (x)}. Then,
fX (x) = 0, ∀ x ∈ Ac (P({g1 (X ) ≤ g2 (X )}) = 1).
X
E (g1 (X )) = g1 (x)fX (x)
x∈SX
X X
= g1 (x)fX (x) + g1 (x)fX (x)
x∈SX ∩A x∈SX ∩Ac
X
= g1 (x)fX (x)
x∈SX ∩A
X
≤ g2 (x)fX (x)
x∈SX ∩A
X X
= g2 (x)fX (x) + g2 (x)fX (x)
x∈SX ∩A x∈SX ∩Ac
X
= g2 (x)fX (x) = E (g2 (X )).
x∈SX
() Module 14 Expectation of a Random Variable 11 / 19
 (b) (Discrete case.) Since P({X ≥ 0}) = 1, we have
SX ⊆ [0, ∞). Thus, for n = 1, 2, . . .,
X
E (X ) = x fX (x)
x∈SX
X X
= x fX (x) + x fX (x)
x∈SX ∩[0, n1 ) x∈SX ∩[ n1 ,∞)
X
≥ x fX (x)
x∈SX ∩[ n1 ,∞)
1 X
≥ fX (x)
n
x∈SX ∩[ n1 ,∞)
 
1 1
= P X ≥
n n
 
1
⇒ P X ≥ ≤ nE (X ) = 0, ∀ n = 1, 2, . . .
n
() Module 14 Expectation of a Random Variable 12 / 19
  
1
⇒ P X ≥ = 0, ∀ n = 1, 2, . . .
n
 
1
⇒ lim P X ≥ = 0
n→∞ n
∞  !
[ 1
⇒ P X ≥ = 0
n
n=1

⇒ P({X > 0}) = 0

⇒ P({X = 0}) = 1.

() Module 14 Expectation of a Random Variable 13 / 19

 (c) Follow from (a) on using the fact that

−|X | ≤ X ≤ |X |.

(d) (Discrete case.) For simplicity of notations we prove the

result for k = 2.
X
E (c1 g1 (X ) + c2 g2 (X )) = (c1 g1 (x) + c2 g2 (x))fX (x)
x∈SX
X
= c1 g1 (x)fX (x) +
x∈SX
X
c2 g2 (x)fX (x)
x∈SX
= c1 E (g1 (X )) + c2 E (g2 (X )).

Remark 1: If E (X ) exists then using (c) above it follows that

|E (X )| < ∞ (i.e., E (X ) is finite).
() Module 14 Expectation of a Random Variable 14 / 19
Some special Expectations

X a r.v.;
g : R → R: a given function;
Then Y = g (X ) is a r.v. and E (g (X )) = expected value of g (X ).
Some special expectations are:
(i) µ01 = µ = E (X ) = mean of (distribution of) X;
(ii) For r ∈ {1, 2, ...} , µ0r = E (X r ) = r -th moment of X about origin;
(iii) For r ∈ {1, 2, ....} , E (|X |r ) = r -th absolute moment of X about
origin;
(iv) For r ∈ {1, 2, ...} , µr = E ((X − µ)r ) = r -th moment of X about its
mean (or r -th central moment);
(v) µ2 = σ 2 = E ((X − µ)2 ) = Variance of X (written as Var(x)).

() Module 14 Expectation of a Random Variable 15 / 19

Remark 1:

(a)

Var(X ) = E ((X − µ)2 )

= E (X 2 − 2µX + µ2 )
= E (X 2 ) − 2µE (X ) + µ2
= E (X 2 ) − µ2
= E (X 2 ) − (E (X ))2 .

(b) Since Var(X) = E ((X − E (X ))2 ) ≥ 0, we have

E (X 2 ) ≥ (E (X ))2 ,

for any r.v. X .

(c) Var(X ) = 0 ⇒ P({X = E (X )}) = 1.

() Module 14 Expectation of a Random Variable 16 / 19

Take Home Problems

1 Let X be a r.v. with p.d.f.


c (x + 1), if − 1 < x < 1
fX (x) = ,
0, otherwise

where c is a real constant.

(a) Find the value of c;
(b) Find the mean and variance of X .
2 . Let X be a r.v. with p.d.f.
1

 x2
, if x > 1
fX (x) = .
0, otherwise


Show that E (X ) does not exists.

() Module 14 Expectation of a Random Variable 17 / 19

Abstract of Next Module

In next module we will introduce a transform

 
MX (t) = E e tX , t ∈ R,

that can be used to generate moments (µ0r = E (X r ), r = 1, 2, . . .) of a

r.v. X . This transform is called the moment generating function (m.g.f.)
of X . We will study various properties of m.g.f.

() Module 14 Expectation of a Random Variable 18 / 19

 Thank you for your patience

() Module 14 Expectation of a Random Variable 19 / 19