B39AX

Engineering Mathematics and Statistics

(1st half)
Dr Xu Wang

Institute of Phonics and Quantum Sciences (IPAQS)

EPS, Heriot Watt University, UK

e-mail: X.Wang@hw.ac.uk
Website: http://home.eps.hw.ac.uk/~xw66/

1
1

INTRODUCTION TO THE COURSE (1st half)

Part 1: Probability
Part 2: Statistics
Probability axioms
Conditional probability Descriptive statistics
Bayes’ law
Permutation Normal distribution
Combination
Sampling
Random variables
(discrete, continues)
PMF, p.d.f, c.d.f.
Hypothesis test
Expectation, variance,
standard deviation,

2nd half Vector Calculus By Dr. Alex Belyev

1
 What are Probability and
Statistics?
Probability is the chance that something will happen
(is the measure of the likelihood that an event will occur)

Uncertainty, Randomness
Gambling: Poker, Dice, Roulette, Lottery, Majiang, …
Physics: Philosophy
* Classical Mechanics (Isaac Newton)
* Electromagnetism (James Maxwell)
* Thermodynamics and statistical mechanics (laws of thermodynamics, Ludwig Boltzmann) 3
* Quantum Mechanics (Niels Bohr, uncertainty principle, Copenhagen interpretation, etc.)

Statistics is a branch of mathematics dealing with the collection,

analysis, interpretation, presentation, and organization of data.
In statistics, a population is a set of similar items or events which is of
interest for some question or experiment

Two main statistical methods:

Descriptive statistics, which summarize data from a sample using indexes
such as the mean or standard deviation. Descriptive statistics are most often
concerned with two sets of properties of a distribution (sample or population):
central tendency (or location) seeks to characterize the distribution's central or
typical value, while dispersion (or variability) characterizes the extent to which
members of the distribution depart from its center and each other
Inferential statistics, which draw conclusions from data that are subject to
random variation (e.g., observational errors, sampling variation).

Mathematical statistics is the application of mathematics to statistics.

Inferences on mathematical statistics are made under the framework
of probability theory, which deals with the analysis of random
phenomena.
4

2
 Course arrangement
Recommended Text:
Probability and Random variables: a Beginner’s
Guide
by David Stirzaker, Cambridge University Press,
2007 (5th printing)

Assessments:
1.There will be one Class test in the 1st half 15%
(one in the 2nd half for 15%)

2.Final Exam (2 Hours duration) at the end of the

semester- 70% 5

Topic 1: Probabilities, Permutations,

Combinations and Bayes rule
• Experiments, Outcomes, Events,
Sample Spaces, & Probabilities
• Contingency Tables, Venn Diagrams,
and Trees
• Probability: Rules and Application
• Permutations and Combinations
• Bayes Rule
6

3
 Definitions
An experiment is a situation involving chance or probability
that leads to results called outcomes.

An outcome is the result of a single trial of an experiment.

A sample point is the most basic outcome of an

experiment

The sample space is the collection of all possible outcomes

An event is one or more outcomes of an experiment.

Probability is a measure of how likely an event

7 is.

Outcome Examples
Experiment Sample Space
Toss a Coin Head, Tail
Toss 2 Coins HH, HT, TH, TT
Select 1 Card, kind 2♥, 2♦, ..., A♠ (52)
Select 1 Card, suit Heart, Club etc. (4)
Play a Football Game Win, Lose, Draw
Observe Gender Male, Female

4
 Outcomes
1. Mutually Exclusive
2 Outcomes cannot occur at the same time
cannot draw both a Heart and a Club
2. Collectively Exhaustive
1 Outcome in sample space must occur
Club, Diamond, Heart or Spade must be drawn

5
 Events
1. Any Collection of Sample Points
2. Simple Event
Outcome With 1 Characteristic
3. Compound Event
Collection of Outcomes or Simple Events
2 or More Characteristics

Event Outcomes
Experiment:
Toss 2 Coins 1 Head & 1 Tail HT, TH
Heads on 1st Coin HH, HT
At Least 1 Head HH, HT, TH
Heads on Both HH
11

Forming Compound Events

1. Intersection
Outcomes in Both Events A and B
(i.e., A ∩ B) ‘a six and a heart’ on a card draw
2. Union
Outcomes in Either Event A or B or
(i.e., A ∪ B) ‘a six or a heart’ on a card draw
Commutativity
Associativity
Distributive laws
DeMorgan’s Law
12

6
 Event Intersection: Venn
Diagram
Experiment: Draw 1 Card. Denomination,
Colour & Suit.

Black Event
Sample Black:
Space: 2B♣, ...,
2R♥, 2R♦, AB♠
2B♣, ..., AB♠
Ace S
Event Ace: Joint Event (Ace ∩ Black):
AR♥, AR♦, AB♣, AB♠ AB♣ , AB♠ 13

Event Intersection:
Contingency Table
Experiment: Draw 1 Card. Denomination,
Colour & Suit.
Color Simple
Sample Type Red Black
Total Event
Space (S): Ace:
Ace Ace & Ace & Ace
2R♥, 2R♦, AR♥,
2B♣, ..., AB♠
Red Black
AR♦,
Non-Ace Non & Non & Non-
AB♣ ,
Joint Event Red Black Ace AB♠
Ace AND Total Red Black S
Black:
AB♣ , AB♠ Simple Event Black: 2B♣, ..., AB♠
14

7
 Event Union :
Venn Diagram
Experiment: Draw 1 Card. Denomination,
Colour & Suit.
Black Event
Sample Black:
Space: 2B♣ ,
2R♥, 2R♦, 2B♠, ...,
2B♣, ..., AB♠ AB♠
Ace S
Event Ace: Event (Ace ∪ Black):
AR♥, AR♦, AB♣, AB♠ AR♥, ..., AB♠, 2B15
♣, ..., KB♠

Event Union : Contingency

Table
Experiment: Draw 1 Card. Denomination,
Colour & Suit.
Color Simple
Sample Type Red Black
Total Event
Space (S): Ace:
Ace Ace & Ace & Ace
2R♥, 2R♦, AR♥,
2B♣, ..., AB♠
Red Black
AR♦,
Non-Ace Non & Non & Non-
AB♣ ,
Joint Event Red Black Ace AB♠
Ace OR Total Red Black S
Black:
AR♥, ..., AB♠, 2B♣, ..., KB♠ Simple Event Black:
2B♣, ..., AB16
♠

8
 Special Events

1. Null Event
Club & Diamond on a single card
2. Complement of Event
For Event A, All Events Not In A: AC
3. Mutually Exclusive Events
Events Do Not Occur Simultaneously

Complement of Event
Example
Experiment: Draw 1 Card. Denomination,
Colour & Suit.

Black
Sample
Space:
2R♥, 2R♦,
2B♣, ..., AB♠
S
Event Black: Complement of Event Black,
2B♣, 2B♠, ..., AB♠ Black ’: 2R♥, 2R♦, ..., AR♥, AR♦
18

9
 Mutually Exclusive Events
Example
Experiment: Draw 1 Card. Denomination,
Colour & Suit.

Sample
♥ Outcomes
in Event
Space: Heart:
2♥ , 2♦ , 2♥ , 3♥ ,

♠
2♣, ..., A♠ 4♥, ..., A♥
S
Event Spade:
2♠, 3♠, 4♠, ..., A♠ Events ♠ &♥ Mutually Exclusive
19

Assigning Event Probabilities

1. a priori, classical method. This is based on
prior knowledge of the process. For example,
consider throwing a pair of dice and finding
the total thrown. Assume these are ‘fair’ dice.
P(Event) = X / T
X = No. of Event Outcomes, e.g. throwing a ‘6’
T = Total Outcomes in Sample Space, all throws
Likelihood of each of T Outcomes is based on
equal likelihood of each dice landing showing any
number {1…6}
So, P(2) = 1/36 P(7) = 6/36

10
Assigning Event Probabilities
2. empirical, classical method. This is based on
experiment, collecting real data

P(Event) = X / T, when experiment is repeated T times, and

the event in question is observed X times.
This also called the relative frequency method
So for example, we could conduct an experiment to draw
cards, with replacement) from a pack 1000 times, say, and
count the number of times we drew a heart, hopefully close
to 250!

Finding probabilities by
experiment
Empirical data: what’s the probability
of getting a head on the toss of a
Total Heads / single fair coin?
Number of Tosses
1.00 How many trials do we need to run to
become confident? That comes later!
0.75

0.50

0.25

0.00
0 25 50 75 100 125
Number of Tosses
22

11
Assigning Event Probabilities
3. Subjective method. This is based on experience,
knowledge of the situation, commonly used in expert
systems for medical diagnosis, or playing the stock
market, for example.

In general, each elicitation of knowledge is unique, i.e. can’t

be repeated, and we will get different probabilities from
different people.

To represent uncertainty, many people have come up with

new ways to represent ‘probabilities’ using methods like
fuzzy logic, Dempster-Shafer logic which in general have
some measure of belief associated with any estimate of
the probability of an event (yes, no, maybe).

12
13
Compound Events: Contingency Table

Event
Event B1 B2 Total
A1 P(A1 ∩ B1) P(A1 ∩ B2) P(A1)
A2 P(A2 ∩ B1) P(A2 ∩ B2) P(A2)
Total P(B1) P(B2) 1

Joint Probability Marginal (Simple) Probability

14
 Contingency Table
Example
Draw 1 Card. Denomination and Colour

Color
Type Red Black Total
Ace 2/52 2/52 4/52
Non-Ace 24/52 24/52 48/52 P(Ace)

Total 26/52 26/52 52/52

P(Red) P(Ace AND Red)

29

Additive Rule

1. Used to Get Compound Probabilities

for Union of Events
2. P(A OR B) = P(A ∪ B)
= P(A) + P(B) - P(A ∩ B)
3. For Mutually Exclusive Events:
P(A OR B) = P(A ∪ B) = P(A) + P(B)

30

15
 Additive Rule Example

Draw 1 Card. Denomination and Colour

Color
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52

P(Ace OR Black) = P(Ace) + P(Black) - P(Ace ∩ Black)

4 26 2 28
= + − =
52 52 52 52
31

32

16
 Conditional Probability
1. Event Probability Given that Another
Event Occurred
2. Revise Original Sample Space to
Account for New Information
Eliminates Certain Outcomes

3. P(A | B) = P(A and B)

P(B)
P(A and B)= P(A | B) *P(B)
33

Conditional Probability
Using Venn Diagram
Black ‘Happens’:
Eliminates All
Black
Other Outcomes

Ace
Black
S (S)
Event (Ace AND Black)

34

17
 Conditional Probability
Using Contingency Table
Draw 1 Card. Denomination and Colour

Color
Type Red Black Total Revised
Sample
Ace 2 2 4 Space
Non-Ace 24 24 48
Total 26 26 52
P(Ace AND Black) 2 / 52 2
P(Ace | Black) = = =
P(Black) 2635/ 52 26

36

18
 Statistical Independence
1. An event occurrence does Not affect the
probability of another event
Toss 1 Coin Twice . There is no causality
2. Tests for independence (Definition)
P(A | B) = P(A)
P(A and B) = P(A)*P(B)

Prove:
Dependent!
If A and B are independence,
Then A and Bc are independent A B

Tree Diagram

Experiment: Select 2 Pens from 20 Pens:

14 Blue & 6 Red. Don’t Replace.
P(R|R) = 5/19 R
P(R) = 6/20
R
P(B|R) = 14/19 B
Dependent!
P(R|B) = 6/19 R
B
P(B) = 14/20
P(B|B) = 13/19
38
B

19
39

40

20
 41

Multiplicative Rule
1. Used to Get Compound Probabilities
for Intersection of Events
Called Joint Events
2. P(A and B) = P(A ∩ B)
= P(A)*P(B|A)
= P(B)*P(A|B)
3. For Independent Events:
P(A and B) = P(A ∩ B) = P(A)*P(B)

42

21
43

44

22
 Bayes’ Theorem (Rule, Law)
Bayes’ Theorem: Let events A1,…,Ak form a partition of
the space S such that Pr(Aj) > 0 for all j and let B be
any event such that Pr(B) > 0. Then for i = 1,..,k:
Pr ( Ai ) Pr ( B | Ai )
Pr ( Ai | B ) =
∑k Pr (Ak ) Pr (B|Ak )

Proof:
Pr( Ai ∩ B) Pr( Ai ) Pr(B | Ai)
Pr( Ai | B) = =
Pr(B) ∑k Pr(Ak ) Pr(B | Ak )
Bayes’ Theorem is just a simple rule for computing the conditional
probability of events Ai given B from the conditional probability of B given
each event Ai and the unconditional probability of each Ai

Interpretation of Bayes’ Theorem

Pr( B | Ai ) = The conditional

Pr(Ai) = Prior distribution for the Ai. It probability of B given Ai. It
summarizes your beliefs about the summarizes the likelihood of event
probability of event Ai before Ai or B are B given Ai.
observed.

Pr ( Ai ) Pr ( B | Ai )
Pr ( Ai | B ) =
∑k Pr (Ak ) Pr (B|Ak )

Pr( Ai | B ) = The posterior ∑k Pr( Ak ) Pr( B | Ak ) = The normalizing constant.

distribution of Ai given B. It
represents the probability of
event Ai after Ai has B has been
observed.
This is equal to the sum of the quantities in the
numerator for all events Ak. Thus, P( Ai | B )
represents the likelihood of event Ai relative to all
other elements of the partition of the sample
space.

46

23
 47

Example: Monty Hall problem (Three door problem)

Suppose you're on a game show, and you're given the choice of three doors:
Behind one door is a car; behind the others, goats.
You pick a door, say No. 1, and the host, who knows what's behind the doors,
opens another door, say No. 3, which has a goat. He then says to you, "Do you
want to pick door No. 2?"
Is it to your advantage to switch your choice?

Possibility of winning a car

Stay? 1/3
Switch? 2/3
In search of a new car, the player picks a door, say 1. The game host then opens one of the other doors, say 3, to rveal a goat and offers to let the player pick door 2 instead of door 1.

48

24
 Permutations and
Combinations
Both are ways to count the possibilities
The difference between them is whether order
matters or not
Consider a poker hand:
A♦, 5♥, 7♣, 10♠, K♠
Is that the same hand as:
K♠, 10♠, 7♣, 5♥, A♦
Does the order the cards are handed out matter?
If yes, then we are dealing with permutations
If no, then we are dealing with combinations

49

Permutations
A permutation is an ordered arrangement of the
elements of some set S
Let S = {a, b, c}
c, b, a is a permutation of S
b, c, a is a different permutation of S
An r-permutation is an ordered arrangement of r
elements of the set
A♦, 5♥, 7♣, 10♠, K♠ is a 5-permutation of the set of
52 cards
The notation for the number of r-permutations:
P(n,r) or nPr
The poker hand is one of P(52,5) permutations
50

25
 Permutations
Number of poker hands (5 cards):
P(52,5) = 52*51*50*49*48 = 311,875,200
Number of (initial) blackjack hands (2 cards):
P(52,2) = 52*51 = 2,652
r-permutation notation: P(n,r)
The poker hand is one of P(52,5) permutations

P(n, r ) = n(n − 1)(n − 2)...(n − r + 1)

n!
= Factorial
(n − r )!
n
= ∏i
i = n − r +1

51

Another Permutation Example

How many permutations of {a, b, c, d, e, f, g} end

with a?
Note that the set has 7 elements

The last character must be a

The rest can be in any order
Thus, we want a 6-permutation on the set {b, c, d,
e, f, g}
P(6,6) = 6! = 720

Why is it not P(7,6)? 52

26
 Combinations

What if order doesn’t matter?

In poker, the following two hands are equivalent:
A♦, 5♥, 7♣, 10♠, K♠
K♠, 10♠, 7♣, 5♥, A♦

The number of r-combinations, C(n,r) or nCr, of a

set with n elements, where n is non-negative
and 0≤r≤n is:
n!
C (n, r ) =
r!( n − r )!
53

Combinations example

How many different poker hands are there

(5 cards)?
52! 52! 52 * 51* 50 * 49 * 48 * 47!
C (52,5) = = = = 2,598,960
5!(52 − 5)! 5!47! 5 * 4 * 3 * 2 *1* 47!

How many different, initial pontoon

(blackjack) hands are there?
52! 52! 52 * 51
C (52,2) = = = = 1,326
2!(52 − 2)! 2!50! 2 *1
54

27
 Combination formula proof

Let C(52,5) be the number of ways to generate

unordered poker hands
The number of ordered poker hands is P(52,5) =
311,875,200
The number of ways to order a single poker hand
is P(5,5) = 5! = 120
The total number of unordered poker hands is the
total number of ordered hands divided by the
number of ways to order each hand
Thus, C(52,5) = P(52,5)/P(5,5)
55

28