EE800 Stochastic Systems

Welcome and Introduction

Dr. Hassan Aqeel Khan

School of Electrical Engineering & Computer Science (SEECS)
National University of Sciences & Technology (NUST)

Copyright Syed Ali Khayam 2009

What will we cover in this lecture?

This lecture is intended to be an introduction to elementary
probability theory
We will cover:
Mathematical Modeling
Random Experiments and Random Variables
Axioms of Probability
Mutual Exclusivity
Conditional Probability

Independence
Law of Total Probability
Bayes Theorem
2

Mathematical Models & Their Use

What is a model ?
An approximate representation of a physical situation.

What is a Mathematical Model ?

A model that quantifies the measurable properties of the phenomenon,
under observation, using mathematical tools.

Example
OHMs Law (Deterministic Model)
Additive White Gaussian Noise (AWGN) Channel (Probabilistic Model)

Transmitted Signal

Received Signal

Gaussian Noise

Mathematical Modeling (HOW TO?)

Physical
Process/System
Observations

Model:

Mathematical Modeling (HOW TO?)

Physical
Process/System
Observations

Model:

log

Mathematical Modeling (HOW TO?)

Physical
Process/System
Observations

Model:

How you handle noise depends

on your application

Mathematical Modeling (HOW TO?)

Formulate
Hypothesis
Define
experiment to test
hypothesis

Physical
Process/System

Model

Observations

Predictions

Sufficient
Agreement ?

No

Yes
No

All aspect of
interest
investigated ?

Yes

Stop
7

Definition of Probability
Probability:
1 : the quality or state of being possible
2 : something (as an event or circumstance) that is possible
3 : the ratio of the number of outcomes in an exhaustive set of
equally likely outcomes that produce a given event to the
total number of possible outcomes, the chance that a given
event will occur
We will revisit these definitions in a little bit

Definition of a Random Experiment

A random experiment comprises of:
A procedure
An outcome

Procedure
(e.g., flipping a coin)

Outcome
(e.g., the value
observed [head, tail] after
flipping the coin)

Sample Space
(Set of All Possible
Outcomes)

Definition of a Random Experiment:

Outcomes, Events and the Sample Space
An outcome cannot be further decomposed into other outcomes
{s1 = the value 1}, , {s6 = the value 6}

An event is a set of outcomes that are of interest to us

A = {s: such that s is an even number}

The sample space is the set of all possible outcomes, called S

10

Definition of a Random Experiment:

Outcomes, Events and the Sample Space

s1

s5

s4

s2

s6

s3

11

Definition of a Random Experiment:

Outcomes, Events and the Sample Space

Example of a Random Experiment:

Experiment:
Roll a fair dice once and record the number of dots
on the top face

Sample space:
Events:
the outcome is even
the outcome is greater than 4
12

Axioms of Probability
Probability of any event A is non-negative:

The probability that an outcome belongs to the sample

space is 1:

The probability of the union of mutually exclusive events

is equal to the sum of their probabilities:
If,
,
Then,
13

Mutual Exclusivity
Are 1 and 2 mutually exclusive?
For mutually exclusive events 1
A1

A2

, we have:

s5

s1

s2

s4

s6

s3

S
Find
example

and

in the fair dice

14

Mutual Exclusivity
Discarding the condition of exclusivity, in general, we have:

s5

s1

s4

s2

s6

s3

S
15

Mutual Exclusivity
Discarding the condition of exclusivity, in general, we have:

s5

s1

s4

s2

s6

s3

S
16

Conditional Probability
Given that event B has already occurred, what is the probability
that event A will occur?
Given that event B has already occurred, reduces the sample
space of A
s5

s1

s2

Event B has
already occurred

s4
s3

s6

=> s2, s4, s3

cannot occur

s5

s1

s2

s4

s6

s3

17

Conditional Probability
Given that event has already occurred, we define a new
conditional sample space that only contains s outcomes
The new event space for is the intersection of and :
Event space

s5

s1

s2

s4

s5

s1
s6

Event B has
already
occurred

s3

Whats missing here?

s4

s2

s6

s3

|
18

Conditional Probability
The probability of an event

in the conditional sample space is:

=
s5

s1

s2

s4
s3

s5

s1
s6

Event has
already
occurred

s6

s4

s2

s3
1

6
19

Independence
Two events are independent if they do not provide any
information about each other:

In other words, the fact that B has already happened does not
affect the probability of As outcomes
Implications:

20