ECON3209: L ECTURE 1

I NTRODUCTION TO P ROBABILITY
Professor Alan Woodland
School of Economics
UNSW

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

1 / 28

I NTRODUCTION TO P ROBABILITY

The theory of probability is often attributed to Pascal and Fermat ( 1650).

It is remarkable that a science which began with consideration of games of

chance should have become the most important object of human knowledge....The
most important questions of life are indeed, for the most part, really only problems
of probability.... The theory of probabilities is at bottom nothing but common sense
reduced to calculus. Laplace, Theorie Analytique des Probabilites, 1820

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

2 / 28

FATE LAUGHS AT P ROBABILITIES .. B ULWER

It is 1941 and the Germans are bombing Moscow. Most people in Moscow flee to
the underground bomb shelters at night, except for a famous Russian statistician
who tells a friend that he is going to sleep in his own bed, saying that There is
only one of me, among five million other people in Moscow. What are the chances
Ill get hit?
He survives the first night, but the next evening he shows up at the shelter. His
friend asks why he has changed his mind. Well, says the statistician, there are
five million people in this city, and one elephant in the Moscow Zoo. Last night,
THEY GOT THE ELEPHANT!

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

3 / 28

D O WE N EED P ROBABILITY T HEORY ?

Probability theory and statistics are useful in dealing with the following types of
situations:
Games of chance: throwing dice, shuffling cards, drawing balls out of urns.
Quality control in production: you take a sample from a shipment, count how many
defectives.
Actuarial Problems: the length of life anticipated for a person who has just applied
for life insurance.
Scientific Eperiments: you count the number of mice which contract cancer when
a group of mice is exposed to cigarette smoke.
Quantum mechanics: the proportion of photons absorbed by a polarization filter
Statistical mechanics: the velocity distribution of molecules in a gas at a given
pressure and temperature.
Economics: the probability of future wage rate outcomes (high, low, medium) over
an individuals lifetime.
Econometrics: the probability distribution for disturbances in a regression model;
the probability of correct forecasts of a recession.

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

4 / 28

P ROBABILITY: C ONCEPT AND D EFINITIONS

Probability Theory is a mathematical representation of random phenomena. A
probability is a numerical measurement of uncertainty.

D EFINITION (C LASSICAL )
If there are N equally likely possibilities, of which one must occur and n are
regarded as favorable, or as success, then the probability of a success is given
by the ratio Nn .
Example: A box contains 4 blue balls and 6 red balls. Randomly pick a ball from
the box, what is the probability it is blue?
Caveat: ideal situation with very limited scope

D EFINITION (R ELATIVE F REQUENCY )

Probability of an event is the proportion of the time that events of the same kind
will occur in the long run.
Repeat a game a large number of times under the same conditions. The
probability of winning is approximately equal to the number of wins in the repeated
trials.
Consistent with Classical approach
Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

5 / 28

P ROBABILITY: C ONCEPT AND D EFINITIONS

D EFINITION (S UBJECTIVE A PPROACH )

A degree of belief in a proposition.
Subjective evaluations.
Educated guess.

D EFINITION (A XIOMATIC A PPROACH )

Kolmogorov developed the first rigorous approach to probability ( 1930). He built
up probability theory from a few fundamental axioms.
Probabilities are mathematical objects that behave according to well-defined rules.
Any of the previous concepts can then be used in applications, as long as they
obey the rules.

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

6 / 28

S AMPLE S PACE AND E VENTS

D EFINITION
With every situation (random experiment) with uncertain outcome we associate a
sample space, noted S, which represents the set of all possible outcomes
(described by the characteristics which we are interested in)
Each outcome is an element or sample point of the sample space.
Discrete sample space: contains a finite number of elements, or an infinite but
countable number of outcomes.
Continuous sample space: elements are not countable. Eg. measurements of
physical properties (time,pressure,length..)
Events are associated with subsets of the sample space.
combinations of two or more events are also events
from given observable events we can derive new observable events by
set theoretical operations.

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

7 / 28

S ET T HEORETICAL O PERATIONS

There are two ways to to denote a set: either by giving a rule, or by listing the
elements. Here are the formal definitions of set theoretic operations. Letters A and
B, etc denote events from the same sample space S, and I is an arbitrary index set. x
stands for an element and x A means that x is an element of A:
1

A B {x : x A x B}: Containment.

A B = {x : x A and x B}: Intersection.

iI Ai = {x : x Ai for all i I}.

A B = {x : x A or x B}: Union.

iI Ai ={x: there exists an i such that x Ai }.

AC = {x : x
/ A but x S} (Event Complement)

BASIC L AWS
Associative Law: A B = B A and A B = B A
Distributive Law:(A B) C = A (B C) and (A B) C = A (B C)
DeMorgans Law: (A B)C = AC B C and (A B)C = AC B C

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

8 / 28

P ROOF OF D E M ORGAN S L AW

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

10 / 28

A XIOMS OF P ROBABILITY
Consider an experiment whose sample space is S and let E be an event in the sample
space S.
Probabilities are values of a set function.
Probability measure: assigns real numbers to the subsets of sample space S.
For each event E of the sample space S, we assume that a number P(E) is defined
and satisfies the following three axioms:

A XIOMS OF P ROBABILITY
1

0 P(E) 1.

P(S) = 1.

For any sequence of n mutually exclusive events E1 , E2 , ..., En :

P
P (ni=1 ) = ni=1 P(Ei )

P ROPOSITIONS
1

P E C = 1 P (E).

If E F , then P(E) P(F ).

P(E F ) = P(E) + P(F ) P(E F ).

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

11 / 28

C OUNTING RULES FOR P ROBABILITY

We will be working in a finite probability space in which all simple events have equal
probabilities. To compute the probability of a certain event, one must count the
elements of the set which this event represents. In other words, we count the number
of different ways to achieve a certain outcome.

S UM RULE
If a first task can be done in m ways and a second task can be done in n ways, and if
these tasks cannot be done at the same time, then there are n + m ways to do either
task. Example: A library has 10 stat textbooks and 15 biology textbooks. How many
textbooks can a student choose from if he is interested in learning about either stat or
biology?

E XAMPLE
How many textbooks can a student choose from if he is interested in learning about
stat and biology?

M ULTIPLICATION P RINCIPLE
If two successive choices are to be made with n1 ways in the first stage, and n2 ways in
the second stage, then the total number of successive choices is n1 n1 .
Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

12 / 28

C OUNTING RULES FOR P ROBABILITY

E XAMPLE
You throw two dice.
Let A = sum of numbers shown is five or less
Let B = both of the numbers shown are five or less
S:

11
21
31
41
51
61

12
22
32
42
52
62

13
23
33
43
53
63

14
24
34
44
54
64

15
25
35
45
55
65

16
26
36
46
56
66

6 6 = 36 possible
outcomes.

11 12 13 14
22 23
A = 21
,
31 32
41
.
P(A) = 10
36

B=

11
21
31
41
51

P(B)

12 13
22 23
32 33
42 43
52 53
= 25
.
36

14
24
34
44
54

15
25
35 ,
45
55

GENERALIZED PRODUCT PRINCIPLE

If r successive choices are to be made with exactly nj at each stage j, then the total
number of outcomes is n1 n1 ... nr = rj=1 nj .

E XAMPLE
You throw two dice and you flip two coins. What is the sample space?
Use Tree diagram.
Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

13 / 28

C OUNTING RULES FOR P ROBABILITY - S OLUTION FOR E XAMPLE

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

14 / 28

P ERMUTATIONS AND C OMBINATIONS

D EFINITION (P ERMUTATION )
A permutation of a set is its arrangement in a certain order (an ordered arrangement).
The number of permutations is the number of ways a set can be written without
repeating its its elements. From the multiplication principle, the number of permutations
of a set of n elements is n (n 1) (n 2)... 2 1 = n! (n factorial). By definition,
0! = 1.

D EFINITION (O RDERED K - TUPLETS )

If one does not arrange the whole set, but interested in the number of k-tuples made up
of distinct elements of the set of n, then the number of possibilities is
n!
.
n (n 1) (n 2)... (n k + 2) (n k + 1) = (nk
)!
We use the notation Pn,k for the number of permutations of n elements taken k at a
time.

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

15 / 28

P ERMUTATIONS AND C OMBINATIONS

E XAMPLE (T HE BIRTHDAY PROBLEM , DG P.30)

Determine the probability p, that at least two people in a group of k people
(2 k 365) will have the same birthday, that is, will have been born on the same day
of the same month but not necessarily in the same year. (Can you make a guess for
k = 30)?

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

16 / 28

P ERMUTATIONS AND C OMBINATIONS - B IRTHDAY P ROBLEM S OLUTION

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

18 / 28

P ERMUTATIONS AND C OMBINATIONS

Suppose that the order of the elements in the k-tuples is not important, how many
k -element (1 k n) subset does a set of n distinct element have?

RULE OF C OMBINATIONS
The number of k -element subsets of an n-element set is :
P
n!
Cn,k = kn,k
= ( kn ) == (nk
. It is pronounced as: n choose k .
!
)!k !

T HEOREM (B INOMIAL C OEFFICIENTS )

For all numbers x and y and each positive integer n,
n

(x + y ) =

n
X

Cn,k x k y nk

k=0

PASCAL T RIANGLE
(00)
(10)

1
1
1
1
1
1

3
4

1
2

1
3

6
10

10

(20)

1
4

3
0

(40)

(50)
Professor Alan Woodland (ECON3209 )

(21)
3
1

()

1
5

(11)

()

(33)

()

(41)
(51)

(22)
3
2

(42)
(52)

(43)
(53)

(44)
(54)

(55)

c
School
of Economics, UNSW

Probability Theory

19 / 28

P ERMUTATIONS AND C OMBINATIONS

D EFINITION (M ULTINOMIAL C OEFFICIENTS )

The multinomial coefficient is:
Cn,n1 ,n2 ,...,nk =

n
n1 , n2 , ..., nk

n!
.
n1 !n2 !...nk !

. This can be interpreted as the number of combinations of n objects divided into k

groups comprising n1 , n2 , ..., nk objects respectively.

T HEOREM (M ULTINOMIAL T HEOREM )

For all numbers x1 , x2 ,..,xk and each positive integer n,
(x1 + x2 + ... + xk )n =

n
n1 , n2 , ... nk

x1 1 x2 2 ...xk k =

n!
n
n
n
x1 1 x2 2 ...xk k
n1 !n2 !...nk !

where the summation extends over all possible combinations of nonnegative integers
n1 , n2 , ...nk such that n1 + n2 + ... + nk = n

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

20 / 28

C ONDITIONAL P ROBABILITY AND I NDEPENDENCE

D EFINITION
If we know that the event B has already occurred, then we know that the outcome of the
experiment is one of those included in B. To evaluate the probability of an event A, we
must consider the set of those outcomes in B which also result in the occurrence of A.
P(A|B) =

P(A B)
P(B)

Draw a Venn Diagram

M ULTIPLICATION RULE AND I NDEPENDENCE

The multiplication rule is a result of the conditional probability rule:
P(A B) = P(A|B) P(B) = P(B|A) P(A).
Two events A and B are independent if and only if
P(A|B) = P(A) and P(B|A) = P(B).
Independence of A and B implies that P(A B) = P(A) P(B).

C AUTION :
What is the difference between Independent events and Exclusive events?
Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

21 / 28

I NDEPENDENCE
E XAMPLE
You have an urn with 5 white and 5 red balls. You take 2 balls without replacement. A is
the event that the first ball is white, and B that the second ball is white.
What is the probability that the first ball is white?
What is the probability that the second ball is white?
Are these two events independent?
Are these two events disjoint?
What is the probability that both balls have the same color?

E XAMPLE (E XAMPLE S OLUTION )

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

23 / 28

BAYES T HEOREM

T HEOREM
In its simplest form Bayes theorem reads:
P(A|B) =

P(B A)
P(B|A)P(A) + P(B|AC )P(AC )

T HEOREM (G ENERALIZATION OF BAYES S

FORMULA )
If B1 , B2 , ..., Bk constitute a partition of the
sample space S and P(Bi ) 6= 0, for
i = 1, 2, ..., k; then for any event A in S such
that P(A =
6 0),
P(Bj |A) = Pk

P(Bj A)

i=1

Professor Alan Woodland (ECON3209 )

Probability Theory

P(A|Bi )P(Bi )

c
School
of Economics, UNSW

24 / 28

BAYES T HEOREM
E XAMPLE
AIDS diagnostic tests are usually over 99.9% accurate on those who do not have AIDS
(i.e.,only 0.1% false positives) , and 100% accurate on those who have AIDS (i.e.,no
false negatives at all). A test is called positive if it indicates that the subject has AIDS.
Assuming that 0.5% of the population actually have AIDS, compute the probability that
a particular individual has AIDS, given that he or she has tested positive.

E XAMPLE (A NSWER )
Advice: Never trust your intuition. Always follow the rule
Let A be the event that the subject has AIDS, and let P be the event that the subject
tested positive. The information given to us is the following:
P(A) = 0.5% = 0.005 P(AC ) = 1 0.005 = 0.995
P(P|A) = 100% = 1
P(P|AC ) = 0.1% = 0.001
We want P(A|P)
P(A|P) =
P(A|P) =

P(P|A)P(A)
P(P|A)P(A)+P(P|AC )P(AC )
10.005
10.005+0.0010.995

Professor Alan Woodland (ECON3209 )

5000
5995

= 0.8340.

Probability Theory

c
School
of Economics, UNSW

25 / 28

F UN T IME

Three roommates slept through their midterm statistics exam on Monday morning.
Since they had returned together by car from the same hometown late Sunday
evening, they decided on a great little falsehood. The three met with the instructor
Monday afternoon and told him that an ill-timed flat tyre had delayed their arrival
until noon.The instructor, while somewhat skeptical, agreed to give them a
makeup exam on Tuesday.
When they arrived the instructor issued them the same makeup exam and
ushered each to a different classroom. The first student sat down and noticed
immediately the instructions indicated that the exam would be divided into Parts I
and II weighted 10% and 90% respectively. Thinking nothing of this disparity, he
proceeded to answer the questions in Part I. These he found rather easy and
moved confidently to Part II on the next page. Suddenly his eyes grew large and
his face paled. Part II consisted of one short and pointed question.......
Which tyre was it?

Professor Alan Woodland (ECON3209 )

Probability Theory

c
School
of Economics, UNSW

27 / 28

c
School
of Economics, UNSW

28 / 28

F URTHER R EADING

A. DeGroot.
Chapter 1, Chapter 2.
For special note:
Ch.1 Sampling with Replacement.
Ch.1 Sampling without Replacement.
Ch.1 Blood Types, DG pp.34.
Ch.1 Statistical Swindles, DG pp.51.
Ch.2 Clinical Trial.
Ch.2 The Game of Craps.
B. Miller and Miller.
Chapter 1, Chapter 2.
For special note:
Ch. 1 Combinatorial Methods.
Ch. 1 Binomial Coefficients.

Professor Alan Woodland (ECON3209 )

Probability Theory