The converse, however, is not always true.

Cov(X, Y ) can be 0 for variables that are not independent.

For an example where the covariance is 0 but
X and Y arent independent, let there be three
outcomes, (1, 1), (0, 2), and (1, 1), all with the
same probability 31 . Theyre clearly not independent since the value of X determines the value of
Y . Note that X = 0 and Y = 0, so

Covariance and Correlation

Math 217 Probability and Statistics
Prof. D. Joyce, Fall 2014
Covariance. Let X and Y be joint random variables. Their covariance Cov(X, Y ) is defined by

Cov(X, Y ) = E((X X )(Y Y ))

= E(XY )
= 13 (1) + 31 0 + 13 1 = 0

Cov(X, Y ) = E((X X )(Y Y )).

Notice that the variance of X is just the covariance

Weve already seen that when X and Y are inof X with itself
dependent, the variance of their sum is the sum of
their variances. Theres a general formula to deal
Var(X) = E((X X )2 ) = Cov(X, X)
with their sum when they arent independent. A
covariance term appears in that formula.
Analogous to the identity for variance
Var(X + Y ) = Var(X) + Var(Y ) + 2 Cov(X, Y )
Var(X) = E(X 2 ) 2X
Heres the proof
there is an identity for covariance
Var(X + Y )
Cov(X) = E(XY ) X Y
= E((X + Y )2 ) E(X + Y )2
= E(X 2 + 2XY + Y 2 ) (X + Y )2
Heres the proof:
= E(X 2 ) + 2E(XY ) + E(Y 2 )
2X 2X Y 2Y
Cov(X, Y )
= E(X 2 ) 2X + 2(E(XY ) X Y )
= E((X X )(Y Y ))
+ E(Y 2 ) 2Y
= E(XY X Y XY + X Y )
= Var(X) + 2 Cov(X, Y ) + Var(Y )
= E(XY ) X E(Y ) E(X)Y + X Y
= E(XY ) X Y

Bilinearity of covariance. Covariance is linear

Covariance can be positive, zero, or negative. in each coordinate. That means two things. First,
Positive indicates that theres an overall tendency you can pass constants through either coordinate:
that when one variable increases, so doe the other,
Cov(aX, Y ) = a Cov(X, Y ) = Cov(X, aY ).
while negative indicates an overall tendency that
when one increases the other decreases.
Second, it preserves sums in each coordinate:
If X and Y are independent variables, then their
Cov(X1 + X2 , Y ) = Cov(X1 , Y ) + Cov(X2 , Y )
covariance is 0:
and

Cov(X, Y ) = E(XY ) X Y
= E(X)E(Y ) X Y = 0

Cov(X, Y1 + Y2 ) = Cov(X, Y1 ) + Cov(X, Y2 ).

1

2
Heres a proof of the first equation in the first But Var(X) = X
and Var(Y ) = Var(Y ) = Y2 ,
condition:
so that equals


Cov(aX, Y ) = E((aX E(aX))(Y E(Y )))
X Y
,
2 + 2 Cov
X Y
= E(a(X E(X))(Y E(Y )))

= aE((X E(X))(Y E(Y )))

= a Cov(X, Y )

By the bilinearity of covariance, that equals

2

The proof of the second condition is also straightforward.

2
Cov(X, Y ) = 2 2XY )
x Y

and weve shown that

Correlation. The correlation XY of two joint
0 2(1 XY .
variables X and Y is a normalized version of their
covariance. Its defined by the equation
Next, divide by 2 move one term to the other side
of the inequality to get
Cov(X, Y )
XY =
.
X Y
XY 1,
Note that independent variables have 0 correlaso
tion as well as 0 covariance.
1 XY 1.
By dividing by the product X Y of the standard deviations, the correlation becomes bounded
This exercise should remind you of the same
between plus and minus 1.
kind of thing that goes on in linear algebra. In
1 XY 1.
fact, it is the same thing exactly. Take a set of
real-valued random variables, not necessarily indeThere are various ways you can prove that in- pendent. Their linear combinations form a vector
equality. Heres one. Well start by proving
space. Their covariance is the inner product (also


called the dot product or scalar product) of two
Y
X
= 2(1 XY ).

0 Var
vectors in that space.
X
Y
X Y = Cov(X, Y )
There are actually two equations there, and we can
prove them at the same time.
2
First note the 0 parts follow from the fact The norm kXk of X is the square root of kXk
variance is nonnegative. Next use the property defined by
proved above about the variance of a sum.
2
kXk2 = X X = Cov(X, X) = V (X) = X


X
Y

Var
and, so, the angle between X and Y is defined by

 X Y 



X
Y
X Y
Cov(X, Y )
X Y
= Var
+ Var
+ 2 Cov
,
= XY
cos =
=
X
Y
X Y
kXk kY k
X Y
Now use the fact that Var(cX) = c2 Var(X) to that is, is the arccosine of the correlation .
XY
rewrite that as

 Math 217 Home Page at
1
X
1
Y
Var(X) + 2 Var(Y ) + 2 Cov
,
http://math.clarku.edu/~djoyce/ma217/
2
X
Y
X
Y
2