Cunoastere si rationament cu

incertitudini

Capitolul 4

2023-2024
 Cunoastere incerta

• Agentii umani si artificiali nu sunt omniscienti

sau atotstiutori.
Perceptia lor asupra lumii este limitata, impiedicandu-i sa stie
exact starea curenta a lumii.
• Este imposibil sa specificam toate conditiile cand se poate aplica
o actiune / plan, astfel incat sa garantam intotdeauna executia sa
cu succes.
• Mai mult, efectele actiunilor agentilor asupra starii lumii nu sunt
intotdeauna cunoscute exact, adica sunt incerte.
• Incertitudine = masura ignorantei unui agent asupra starii
lumii.
2023-2024
 Teoria Probabilitatilor
• Interpretarea statistica, obiectiva sau frecventionista :
incertitudinea este egala cu masura unei proportii (frecventa
relativa) intr-o multime de indivizi. Exemplu:
proportia pasarilor zburatoare din multimea tuturor pasarilor
frecventa de aparitia a 6 la aruncarea unui zar
• Interpretarea personala, subiectiva sau Bayesiana :
incertitudinea este masura convingerii unui agent in adevarul
unui enunt pe baza cunoasterii agentului. Exemplu:
Convingerea unui agent in abilitatea unui individ de a zbura pe
baza cunoasterii ca acel individ este o pasare
• Ambele interpretari se bazeaza pe calculul probabilitatilor !
2023-2024
 Teoria Probabilitatilor in IA

• Agenti diferiti pot asigna valori diferite valorii

probabilitatii ca o “anumita pasare poate zbura” pentru
ca:
– Fie au avut experiente diferite privind pasarile
– Sau au cunostinte diferite privitor la acea pasare particulara.

• IA este interesata cum agentii iau decizii in diverse

situatii pe baza experientelor si cunoasterii lor, si/sau pe
baza intereselor lor particulare ⇒
Vom adopta interpretarea Bayesiana (subiectiva).
2023-2024
 Masuri numerice ale incertitudinii
• Teoria Probabilitatilor (abordarea subiectiva) = studiaza modul in
care cunoasterea influenteaza convingerile agentilor in adevarul unui
enunt (formula logica).
• Convingerea unui agent in adevarul formulei 𝑓 poate fi masurat
printr-un numar 𝑃 𝑓 ∈ ,0,1-.
– Daca 𝑃(𝑓) = 0 atunci agentul “are convingerea ca” f este precis
(intotdeauna) falsa.
– Daca 𝑃(𝑓) = 1 atunci agentul “are convingerea ca” f este precis
adevarata.
• Presupunem ca natura incertitudinii este epistemologica – adica
incertitudinea reflecta cunoasterea agentului asupra lumii, si nu
ontologica – adica incertitudinea reflecta lumea asa cum este ea in
realitate, independent de agent.
• 𝑃(𝑓) masoara ignoranta agentului asupra adevarului formulei f.
2023-2024
 Variabile aleatoare
• O variabila aleatoare 𝑋 poate lua valori intr-un
domeniu notat cu 𝑑𝑜𝑚(𝑋).
• Aceste valori sunt exhaustive si exclusive, adica daca
𝑋 = 𝑣 atunci:
– 𝑣 ∈ 𝑑𝑜𝑚(𝑋)
– 𝑋 ≠ 𝑤 pentru toti 𝑤 ∈ 𝑑𝑜𝑚 𝑋 ∖ *𝑣+.
• O expresie 𝑋 = 𝑣 se numetse asignare.
• Un tuplu de variabile aleatoare este o variabila
aleatoare compusa 𝑋1 , 𝑋2 , … , 𝑋𝑛 astfel incat 𝑋𝑖 sunt
variabile aleatoare, 1 ≤ 𝑖 ≤ 𝑛. Domeniul sau este:
𝑑𝑜𝑚 𝑋 = 𝑑𝑜𝑚 𝑋1 × ⋯ × 𝑑𝑜𝑚(𝑋𝑛 )
2023-2024
 Tipuri de variabile aleatoare
• A discrete random variable is a random variable with domain:
– finite set, for example *1,2,3,4,5,6+ or
– infinite countable set, for example set of natural numbers ℕ.
• Otherwise the random variable is called continuous random
variable – its domain of values is an infinite uncountable set,
for example interval 0,1 or set of real numbers ℝ.

• A Boolean random variable is a random variable with domain

of values *𝑓𝑎𝑙𝑠𝑒, 𝑡𝑟𝑢𝑒+.
– Assignment 𝑋 = 𝑡𝑟𝑢𝑒 is denoted by 𝑋
– Assignment 𝑋 = 𝑓𝑎𝑙𝑠𝑒 is denoted by ≦𝑋.

2023-2024
 Example
• Consider two boxes:
– 1 blue box: 3 apples and 1 orange
– 1 red box: 2 apples and 3 oranges
• Choose randomly one box and extract randomly one fruit from
that box. The fruit is then returned to its original box.
• We can define two random variables :
– Color of chosen box 𝐵, 𝑑𝑜𝑚 𝐵 = *𝑟𝑒𝑑, 𝑏𝑙𝑢𝑒+
– Chosen fruit 𝐹, 𝑑𝑜𝑚 𝐹 = 𝑜𝑟𝑎𝑛𝑔𝑒, 𝑎𝑝𝑝𝑙𝑒

2023-2024
 Propositions and Formulae
• An atomic or primitive proposition is:
– An expression 𝑋 𝑜𝑝 𝑣 s.t. 𝑣 ∈ dom(𝑋) and 𝑜𝑝 is a binary relational
operator defined by 𝑑𝑜𝑚 𝑋 × 𝑑𝑜𝑚(𝑋), For example 𝑜𝑝 ∈
=, ≠, <, ≤, >, ≥ for the case when domain of 𝑋 is a totally ordered
set (for example a set of numbers).
– An expression 𝑋 𝑜𝑝 𝑌 s.t. 𝑜𝑝 is a binary relational operator defined by
𝑑𝑜𝑚 𝑋 × 𝑑𝑜𝑚(𝑌).

• Using atomic propositions and usual logical connectives ≦, ∧

and ∨ we can form composite propositions, for example:
𝑋 = 2 ∨ (𝑌 = 3)
𝑋 ≥ 3 ∧ (𝑌 ≠ 𝑋)
• Propositions (atomic and composite) are also called formulae
or events.
2023-2024
 Possible Worlds
• An agent knowledge about the problem domain is uncertain. It
perceives the world state with a level of certainty, so a state
stare is also called possible world.
• The set of all world states is called possible worlds or sample
space and it is denoted by Ω.
• The domain is perceived and captured (modeled) by the agent
using random variables. There is a duality between variables
and possible worlds, i.e.:
– Possible worlds define variables values, or equivalently
– Values of all variables define a specific possible world (state)

2023-2024
 Random Variables vs Possible Worlds
• A possible world is an assignment of a value to each
random variable.
• So, if 𝒱 is the set of variables and 𝒟 is the union of all
their domains then a possible world is a function
𝜔 ∶ 𝒱 → 𝒟 s.t. 𝜔 𝑋 ∈ 𝑑𝑜𝑚(𝑋)
• A random variable can be also defined as a function
defined on set of possible worlds with values in the
variable’s domain.
• Let Ω be the set of all possible worlds. A random
variable is a function 𝑋 ∶ Ω → 𝑑𝑜𝑚(𝑋). Valoarea
variabilei este 𝑋 𝜔 ∈ 𝑑𝑜𝑚(𝑋) pentru o stare 𝜔 ∈ Ω.
2023-2024
 Formulae Semantics
• Let ℱ be the set of formulae. Relation ⊨ Ω × ℱ defines the
truth value of formula 𝑓 ∈ ℱ in a world 𝜔 ∈ Ω. It is defined
inductively as follows:
– 𝜔 ⊨ 𝑋 𝑜𝑝 𝑣 if value assigned by 𝜔 to 𝑋 is in relation 𝑜𝑝 with 𝑣 i.e.
𝑋 𝜔 𝑜𝑝 𝑣.
– 𝜔 ⊨ 𝑋 𝑜𝑝 Y if value assigned by 𝜔 to 𝑋 is in relation 𝑜𝑝 with value
assigned by 𝜔 to 𝑌 i.e. 𝑋 𝜔 𝑜𝑝 𝑌(𝜔).
– 𝜔 ⊨ 𝑓 ∧ 𝑔 if 𝜔 ⊨ 𝑓 and 𝜔 ⊨ 𝑔
– 𝜔 ⊨ 𝑓 ∨ 𝑔 if 𝜔 ⊨ 𝑓 or 𝜔 ⊨ 𝑔 or both
– 𝜔 ⊨ ≦𝑓 if ≦(𝜔 ⊨ 𝑓)
• A tautology is a true formula in all worlds.
• A contradiction or impossible event is a false formula in all
worlds.
• 𝑓 and 𝑔 are mutually exclusive if 𝑓 ∧ 𝑔 is a contradiction.
2023-2024
 Model
• Every formula 𝑓 ∈ ℱ defines a subset Ω𝑓 ⊆ Ω called
subset of models of 𝑓:
Ω𝑓 = 𝜔 ∈ Ω 𝜔 ⊨ 𝑓+ ⊆ Ω
• If 𝑓 is tautology (universally true) then Ω𝑓 = Ω.
• Example: ∨𝑣∈𝑑𝑜𝑚 𝑋 𝑋 = 𝑣 is tautology.
• If 𝑓 is contradiction (universally false, impossible)
then Ω𝑓 = ∅.
• Example: if 𝑣 ≠ 𝑤 ∈ 𝑑𝑜𝑚(𝑋) then 𝑋 = 𝑣 ∧ (𝑋 =
𝑤) is a contradiction.
• Observation: 𝑓 is tautology iff ≦𝑓 is contradiction.
2023-2024
 Set of models: 𝝈-algebras
• Let Ω be the set of possible worlds. We cannot use any
collection of subsets of Ω (subset of 2Ω ) to define probabilities.
• These subsets must be measurable. Mathematically they must
be 𝜍-algebras. So, in general case, defining semantics of
probability is based on measure theory.

• Definition: A 𝛔-algebra is a set 𝔉 ⊆ 2Ω such that:

– ∅, Ω ∈ 𝔉
– If 𝐴 ∈ 𝔉 then Ω ∖ 𝐴 ∈ 𝔉
– If 𝐴1 , 𝐴2 , … , 𝐴𝑘 , … ∈ 𝔉 then ∪𝑖≥1 𝐴𝑖 ,∩𝑖≥1 𝐴𝑖 ∈ 𝔉

2023-2024
 Examples of 𝝈-algebras
• *∅, Ω+ is called trivial 𝝈-algebra. It is useful for defining the
algebra of Boolean functions (logical formulae with two
possible interpretations: true and false).
• 2Ω se called power 𝝈-algebra. It is useful to define discrete
probability distributions.
• If Ω = ℝ we consider the collection of finite or infinite, closed
or open real intervals, together with their countable unions and
intersections. This represents all measurable sets of real
numbers (i.e. their “length” can be intuitively defined) and it is
called Borel algebra. It is useful to define probability
distributions of continuous real random variables. The
elements of this set are called Borel sets.

2023-2024
 Masura de probabilitate
• Se considera intai cazul finit (discret). Cu alte cuvinte
presupunem ca:
– avem un numar finit de variabile aleatoare discrete
– Domeniul fiecarei variabile aleatoare este finit
• Fiecarei lumi posibile 𝜔 ∈ Ω i se asociaza o masura numerica
𝑚(𝜔) numita masura de probabilitate cu proprietatile
urmatoare:

0 ≤ 𝑚 𝜔 pentru orice 𝜔 ∈ Ω
𝜔∈Ω 𝑚 𝜔 = 1

• Valoarea 𝑚(𝜔) semnifica increderea unui agent in faptul ca

lumea 𝜔 este lumea reala.
2023-2024
 Semantica probabilitatilor – cazul discret
• Probabilitatea 𝑃(𝑓) unei formule 𝑓 se defineste astfel:
𝑃 𝑓 = 𝜔⊨𝑓 𝑚 𝜔 = 𝜔∈Ω𝑓 𝑚(𝜔)

• Cu alte cuvinte 𝑃(𝑓) reprezinta proportia ponderata a multimii

lumilor Ω𝑓 in care formula 𝑓 este adevarata, din multimea
tuturor lumilor posibile Ω.
• Observatie:
– 𝑚 satisface niste axiome de baza:
𝑚 𝐴 ∪ 𝐵 = 𝑚 𝐴 ∪ 𝑚(𝐵) daca 𝐴 ∩ 𝐵 = ∅
𝑚 Ω =1
– Este suficient ca 𝑚 sa fie definita pe acele submultimi ale lui
Ω care sunt definite de semantica formulelor logice. O astfel
de multime de submultimi se numeste sigma algebra pe 𝜴.
2023-2024
 Lumi posibile egal probabile
• Se considera cazul in care exista un numar finit de lumi
posibile egal probabile 𝜔1 , 𝜔2 , … , 𝜔𝑛 . Deci:
𝑚 𝜔1 = 𝑚 𝜔2 = ⋯ = 𝑚 𝜔𝑛 = 𝑝
• Rezulta ca:
𝑛 1
1=𝑃 Ω = 𝑖=1 𝑚 𝜔𝑖 = 𝑛𝑝 deci 𝑝 =
𝑛
• Presupunand ca un eveniment (formula) 𝑓 este adevarata pentru
𝑡 lumi posibile va rezulta ca:
𝑡 𝑁𝑓
𝑃 𝑓 = 𝜔⊨𝑓 𝑝 = = unde 𝑛 = 𝑁𝑇
𝑛 𝑁𝑇
unde:
– 𝑁𝑓 este numarul de lumi pentru care 𝑓 este adevarata (lumi favorabile)
– 𝑁𝑇 reprezinta numarul total de lumi posibile
2023-2024
 Distributie de probabilitate
• Fie 𝑋 o variabila aleatoare discreta. Functia 𝑃𝑋 : 𝑑𝑜𝑚(𝑋) →
,0,1- definita prin:
𝑃𝑋 𝑣 = 𝑃(𝑋 = 𝑣)
se numeste distributia discreta de probabilitate sau masa de
probabilitate (engl. probability mass function) sau pmf a
variabilei aleatoare 𝑋. Prin abuz de limbaj ea se mai noteaza
prin 𝑃 𝑋 .
• Exemplu: Se considera o variabila aleatoare 𝑋 care reprezinta
numarul de fete intr-o familie cu 3 copii.
– 𝑑𝑜𝑚 𝑋 = *0,1,2,3+
1
–𝑃 𝑋=0 =𝑃 𝑋=3 = WHY?
8
3
–𝑃 𝑋=1 =𝑃 𝑋=2 =
8
2023-2024
 Distributie completa de probabilitate
• Fie 𝑋1 , 𝑋2 , … , 𝑋𝑛 variabile aleatoare discrete. Functia
𝑃𝑋1 ,𝑋2 ,…,𝑋𝑛 : 𝑑𝑜𝑚 𝑋1 × 𝑑𝑜𝑚 𝑋2 × ⋯ × 𝑑𝑜𝑚 𝑋𝑛 → ,0,1- :
𝑃𝑋1 ,𝑋2 ,…,𝑋𝑛 𝑣1 , 𝑣2 , … , 𝑣𝑛 =
𝑃( 𝑋1 = 𝑣1 ∧ 𝑋2 = 𝑣2 ∧ ⋯ ∧ (𝑋𝑛 = 𝑣𝑛 ))
se numeste distributie completa de probabilitate (engl. joint
probability distribution). Prin abuz de limbaj ea se mai noteaza prin
𝑃 𝑋1 , 𝑋2 , … , 𝑋𝑛 .
• Exemplu: Se considera o familie cu 3 copii. Fie 𝑋𝑖 , 𝑖 = 1,2,3
variabilele aleatoare Booleene care reprezinta situatiile cand al 𝑖-lea
copil este fata, pentru 𝑖 = 1,2,3.
1
– 𝑃 𝑋1 ∧ ≦𝑋2 ∧ 𝑋3 = 𝑃 1,0,1 =
8
1 1 1 3
– 𝑃 𝑋 = 2 = 𝑃 0,1,1 + 𝑃 1,0,1 +𝑃 1,1,0 = + + =
8 8 8 8

2023-2024
 Specificarea distributiei complete de
probabilitate pentru domenii finite
• Fie 𝒳𝑖 = 𝑑𝑜𝑚(𝑋𝑖) finite. Probabilitatile 𝑃(𝑥1 , 𝑥2 , … , 𝑥𝑛 ) pentru toate
valorile posibile 𝑥𝑖 ∈ 𝒳𝑖 se definesc printr-un tabel cu 𝑛 coloane.
• Suma acestor valori este 1. Definirea DCP necesita 𝑛𝑖=1 𝒳𝑖 − 1 valori.
Pentru 𝑛 variabile aleatoare Booleene sunt necesare 2𝑛 − 1 valori.

𝐹𝑜𝑟𝑚𝑎 𝑃𝑙𝑖𝑛 𝑃(𝐹𝑜𝑟𝑚𝑎, 𝑃𝑙𝑖𝑛)

𝑐𝑒𝑟𝑐 0 1/8
𝑐𝑒𝑟𝑐 1 1/4
𝑡𝑟𝑖𝑢𝑛𝑔𝑕𝑖 0 1/8
𝑡𝑟𝑖𝑢𝑛𝑔𝑕𝑖 1 1/8
𝑟𝑜𝑚𝑏 0 1/8
𝑟𝑜𝑚𝑏 1 1/4
2023-2024
 Distributie marginala de probabilitate
• Fie 𝑋1 , 𝑋2 , … , 𝑋𝑛 variabile aleatoare discrete si fie 𝑌1 , 𝑌2 , … , 𝑌𝑚 ⊂
*𝑋1 , 𝑋2 , … , 𝑋𝑛 + o submultime stricta a lor, 𝑚 < 𝑛. Functia
𝑃𝑌1,𝑌2 ,…,𝑌𝑚 : 𝑑𝑜𝑚 𝑌1 × 𝑑𝑜𝑚 𝑌2 × ⋯ × 𝑑𝑜𝑚 𝑌𝑚 → ,0,1- :
𝑃𝑌1,𝑌2 ,…,𝑌𝑚 𝑤1 , 𝑤2 , … , 𝑤𝑚 =
𝑃( 𝑌1 = 𝑤1 ∧ 𝑌2 = 𝑤2 ∧ ⋯ ∧ (𝑌𝑚 = 𝑤𝑚 ))
se numeste distributie marginala de probabilitate (engl. marginal
probability distribution). Prin abuz de limbaj ea se mai noteaza prin
𝑃 𝑌1 , 𝑌2 , … , 𝑌𝑚 .
• Exemplu: Se considera o familie cu 3 copii. Fie 𝑋𝑖 , 𝑖 = 1,2,3
variabilele aleatoare Booleene care reprezinta situatiile cand al 𝑖-lea
copil este fata, pentru 𝑖 = 1,2,3. Probabilitatea ca primii doi copii sa
fie fete este:
1 1 1
– 𝑃 𝑋1 ∧ 𝑋2 = 𝑃 1,1 = 𝑃 1,1,0 + 𝑃 1,1,1 = + =
8 8 4
2023-2024
 Probleme
1. Dupa 10 ani de functionare 40% din calculatoare au probleme cu placa
mama, 30% au probleme cu hard disk-ul si 15% au atat probleme cu
placa mama, cat si cu hard disk-ul. Care este probabilitatea ca dupa 10
ani un calculator sa aiba complet functionale placa mama si hard diskul?
2. Intre angajatii unei firme 60% stiu C/C++, 70% stiu Java si 50% stiu
ambele limbaje. Se cere: a. Cati nu stiu Java?; b. Cati nu stiu nici C/C++
nici Java?; c. Cati stiu Java dar nu stiu C/C++?
3. O familie are 2 copii. Va intalniti cu unul dintre ei – Leo, si el este baiat.
Care este probabilitatea ca celalalt copil sa fie tot baiat?
4. Pentru construirea parolelor se folosesc 8 caractere ce se aleg dintre: 10
cifre, 26 litere mari si 26 litere mici. Un program spargator de parole
poate testa 1 milion de parole pe secunda. Care este probabilitatea ca
acest program sa detecteze o parola intr-o saptamana de rulare?
5. Un magazin de calculatoare are 20 de calculatoare in stoc. Dintre ele, 15
sunt noi si 5 sunt refolosite. Se cumpara 6 calculatoare pentru un
laborator. Care este probabilitatea ca 2 sa fie refolosite? 2023-2024
 Definirea axiomatica a probabilitatilor
• Fie Ω multimea lumilor posibile. Probabilitatea se
defineste pentru acele multimi de formule ℱ care au
proprietatea ca 𝔉 = *Ω𝑓 |𝑓 ∈ ℱ+ este o 𝜍-algebra pe Ω.
• Se defineste o functie de probabilitate 𝑃: ℱ → ℝ a. i.:
1. 0 ≤ 𝑃(𝑓) pentru orice formula 𝑓 ∈ ℱ
2. Daca formula 𝜏 este o tautologie atunci 𝑃(𝜏) = 1.
3. Daca 𝑓1 , 𝑓2 , … , 𝑓𝑖 , … este o colectie finita sau infinit
numarabila de formule mutual exclusive atunci:
𝑃 𝑓1 ∨ 𝑓2 ∨ ⋯ ∨ 𝑓𝑖 ∨ ⋯ = 𝑃 𝑓1 + 𝑃 𝑓2 + ⋯ + 𝑃 𝑓𝑖 + ⋯
• Observatie. Se poate demonstra usor ca 𝑃 ia valori cel
mult egale cu 1, adica 𝑃: ℱ → ,0,1-.
2023-2024
 Echivalenta cu definitia semantica
• Calculul probabilitatilor poate fi definit axiomatic prin cele 3
axiome mentionate anterior.

• Propozitie. Daca in limbajul de reprezentare exista un numar

finit de variabile aleatoare discrete atunci aceste 3 axiome
formeaza o axiomatizare corecta si completa a calculului
probabilitatilor.
• Corolar. Daca masura increderii unui agent verifica aceste
axiome atunci ea este guvernata de teoria probabilitatilor.

• Observatie. Daca 𝑓 ↔ 𝑔 este o tautologie atunci 𝑃(𝑓) = 𝑃(𝑔).

De ce?
2023-2024
 Proprietati ale probabilitatilor
• Negatia unei formule:
𝑃(≦𝑓) = 1– 𝑃(𝑓)
• Rationament pe cazuri:
𝑃(𝑓) = 𝑃(𝑓 ∧ 𝑔) + 𝑃(𝑓 ∧ ≦𝑔)
• Daca 𝑉 este o variabila aleatoare cu domeniul 𝐷 atunci pentru orice
formula f avem:
𝑃(𝑓) = 𝑑∈𝐷 𝑃(𝑓 ∧ 𝑉 = 𝑑)
• Disjunctia unor formule neexclusive / Principiul includerii si
excluderii:
𝑃 𝑓 ∨ 𝑔 = 𝑃 𝑓 + 𝑃 𝑔 − 𝑃(𝑓 ∧ 𝑔)

• Tema: Demonstrarea acestor relatii este lasata ca exercitiu.

2023-2024
 Conditionare
• Definitie. Conditionarea specifica modul in care un agent isi revizuie
increderea in prezenta unor informatii noi.
• Definitie. Fie 𝑕 o formula numita ipoteza (engl.hypothesis). Valoarea
𝑃(𝑕) a probabilitatii ipotezei 𝑕 conform unui model de probabilitate care
ia in considerare toate informatia primara (engl. background) referitoare
la 𝑕 se numeste probabilitate a priori a lui 𝑕 (engl.prior probability).
• Definitie. Se numeste dovada, constatare sau proba (engl.evidence) o
formula e care reprezinta toate observatiile agentului referitoare la lumea
inconjuratoare. De obicei e are forma unei conjunctii.
• Definitie. Revizuirea increderii agentului in ipoteza 𝑕 pe baza
observatiilor din constatarea e se numeste conditionare si ea presupune
determinarea probabilitatii conditionate 𝑃(𝑕|𝑒). 𝑃(𝑕|𝑒) se numeste si
probabilitate a posteriori (engl.posterior probability). 𝑃𝑒 𝑕 = 𝑃(𝑕|𝑒).
• Observatie: Probabilitatea a priori 𝑃(𝑕) este egala cu 𝑃(𝑕|𝑎𝑑𝑒𝑣𝑎𝑟𝑎𝑡).

2023-2024
 Semantica probabilitatii conditionate
• Fie e o constatare cu 𝑃(𝑒) > 0 (daca 𝑃(𝑒) = 0 atunci 𝑒 este
imposibila si deci nu poate fi observata). Constatarea 𝑒 va conduce
la eliminarea tuturor lumilor incompatibile cu 𝑒. Ea va induce astfel
o noua masura a increderii 𝑚𝑒 definita astfel:
1
×𝑚 𝜔 daca ω ⊨ 𝑒
𝑃 𝑒
𝑚𝑒 𝜔 =
0 altfel
• Probabilitatea conditionata a ipotezei 𝑕 data fiind constatarea 𝑒 se
defineste dupa formula generala de definire probabilitatii, folosind
insa noua masura 𝑚𝑒 .
1 𝑃 ℎ∧𝑒
𝑃 𝑕𝑒 = 𝜔⊨ℎ 𝑚𝑒 𝜔 = × 𝜔⊨ℎ∧𝑒 𝑚 𝜔 =
𝑃 𝑒 𝑃 𝑒
𝑃 ℎ∧𝑒
• Ecuatia: 𝑃 𝑕 𝑒 = pentru 𝑃 𝑒 > 0 este deseori considerata
𝑃 𝑒
definitia probabilitatii conditionate.
2023-2024
 Exemplu
• Se stie ca:
– 90% din trenuri pleaca la timp: 𝑃. 𝑃 𝑃 = 0.9
– 80% din trenuri ajung la timp: 𝐴. 𝑃 𝐴 = 0.8
– 75% din trenuri pleaca la timp si ajung la timp: 𝑃 ∧ 𝐴. 𝑃 𝑃 ∧ 𝐴 = 0.75
• Se cere:
1. Ati prins un tren care a plecat la timp. Care este probabilitatea sa
ajunga la timp? 𝑃 𝐴 𝑃 =?
𝑃 𝐴∧𝑃 0.75
𝑃 𝐴𝑃 = = = 0.8333
𝑃 𝑃 0.9
2. Ati coborat dintr-un tren care a ajuns la timp. Care este probabilitatea
sa fi plecat la timp? 𝑃 𝑃 𝐴 =?
𝑃 𝑃∧𝐴 0.75
𝑃 𝑃𝐴 = = = 0.9375
𝑃 𝐴 0.8
3. Sunt evenimentele 𝑃 si 𝐴 independente? Tema ! (vezi slide-urile
urmatoare pentru definitia independentei)
2023-2024
Regula inlantuirii probabilitatilor conditionate

• Demonstratia se poate face prin inductie dupa n. Se foloseste

definitia probabilitatii conditionate.
• Spre exemplu, pentru 𝑛 = 3 formula devine:

𝑃 𝑓1 ∧ 𝑓2 ∧ 𝑓3 = 𝑃 𝑓1 × 𝑃 𝑓2 𝑓1 × 𝑃(𝑓3 |𝑓1 ∧ 𝑓2 )

2023-2024
 Formula de inversiune a lui Bayes
𝑃 𝑒 ℎ ×𝑃 ℎ
• Propozitie. Daca 𝑃 𝑒 ≠ 0 atunci: 𝑃 𝑕 𝑒 =
𝑃 𝑒
• Demonstratie.
𝑃 𝑕∧𝑒 =𝑃 𝑕 𝑒 ×𝑃 𝑒
𝑃 𝑒∧𝑕 =𝑃 𝑒 𝑕 ×𝑃 𝑕
Egaland cei doi membrii, formula lui Bayes rezulta imediat.
• Interpretare. Formula lui Bayes arata cum se calculeaza
probabilitatea a posteriori 𝑃(𝑕|𝑒) pe baza probabilitatii a priori
𝑃(𝑕) si a verosimilitatii (engl.likelihood) 𝑃(𝑒|𝑕) a ipotezei 𝑕.
Deci probabilitatea a posteriori este proportionala cu produsul
dintre probabilitatea a priori si verosimilitate.
• Verosimilitate. Verosimilitatea 𝑃(𝑒|𝑕) a ipotezei 𝑕 reprezinta
probabilitatea conditionata ca dovada 𝑒 sa fi fost intr-adevar
cauzata de ipoteza 𝑕.
2023-2024
 Aplicarea formulei de inversiune a lui Bayes
• Pe cazul general sa presupunem ca dispunem de o multime
completa de ipoteze mutual exclusive *𝑕1, … , 𝑕𝑛+. Acest lucru
inseamna ca pentru orice 1 ≤ 𝑖 < 𝑗 ≤ 𝑛 formula ≦(𝑕𝑖 ∧ 𝑕𝑗)
este o tautologie si ca 𝑕1 ∨ ⋯ ∨ 𝑕𝑛 este o tautologie. Aplicand
formula de rationament pe cazuri:
𝑛 𝑛
𝑃 𝑒 = Σ𝑖=1 𝑃(𝑒 ∧ 𝑕𝑖 ) = Σ𝑖=1 𝑃 𝑒 𝑕𝑖 𝑃(𝑕𝑖 )

• Din formula lui Bayes va rezulta:

𝑃 𝑒 𝑕𝑗 𝑃 𝑕𝑗
𝑃 𝑕𝑗 𝑒 = 𝑛
Σ𝑖=1 𝑃 𝑒 𝑕𝑖 𝑃 𝑕𝑖

2023-2024
 Exemplu
• Un test de identificare a consumului unui medicament este 98 %
senzitiv si 98 % specific. Se stie ca 0.4% din populatie sunt utilizatori
ai medicamentului. Care este probabilitatea ca o persoana testata
pozitiv sa fie utilizator al medicamentului ?
• Senzitivitatea: probabilitatea ca utilizatorii medicamentului sa obtina
rezultate adevarat-pozitive (engl. true-positive).
• Specificitatea: probabilitatea ca neutilizatorii medicamentului sa
obtina rezultate adevarat-negative (engl. true-negative).
𝑃 𝑇𝑃 𝑈 = 0.98
𝑃 ≦𝑇𝑃 ≦𝑈 = 0.98
𝑃 𝑈 = 0.004
𝑃 𝑈 𝑇𝑃 = ?
𝑃 𝑇𝑃 𝑈 𝑃 𝑈
𝑃 𝑈 𝑇𝑃 =
𝑃 𝑇𝑃
𝑃 𝑇𝑃 = 𝑃 𝑇𝑃 ∧ 𝑈 + 𝑃 𝑇𝑃 ∧ ≦𝑈 = 𝑃 𝑇𝑃 𝑈 𝑃 𝑈 + 𝑃 𝑇𝑃 ≦𝑈 𝑃 ≦𝑈 =
𝑃 𝑇𝑃 𝑈 𝑃 𝑈 + 1 − 𝑃 ≦𝑇𝑃 ≦𝑈 1 − 𝑃 𝑈 = 0.98 × 0.004 + 1 − 0.98 ×
1 − 0.004 = 0.02384 Probabilitatea ceruta este de aproximativ 16.44 % De ce ?
2023-2024
Formula lui Bayes cu probabilitati conditionate

• Uneori probabilitatea a priori in ipoteza 𝑕 se bazeaza

pe o dovada 𝑘, uneori numita si cunoastere data sau
preliminara (engl. background knowledge).
• Astfel cunoastem 𝑃(𝑕|𝑘), observam pe 𝑒 si suntem
interesati in determinarea valorii probabilitatii
𝑃 𝑕 𝑒 ∧ 𝑘).

𝑃 𝑒 𝑕∧𝑘 ×𝑃 𝑕 𝑘
𝑃 𝑕 𝑒∧𝑘 =
𝑃 𝑒𝑘

2023-2024
 Independenta conditionata
• Definitie. 𝑕 este independenta conditionat de 𝑓 data fiind
𝑒, fapt notat prin 𝑕 ⊥ 𝑓|𝑒, dnd 𝑃(𝑕|𝑓 ∧ 𝑒) = 𝑃(𝑕|𝑒). Cu
alte cuvinte cunoasterea lui 𝑓 nu afecteaza increderea in 𝑕
data fiind 𝑒.

• Daca 𝑒 = 𝑡𝑟𝑢𝑒 se obtine independenta simpla

(neconditionata). Astfel 𝑕 este independenta de
𝑓, fapt notat prin 𝑕 ⊥ 𝑓, dnd 𝑃 𝑕|𝑓 = 𝑃 𝑕

• Variabila aleatoare 𝑋 este independenta conditionat de

variabila aleatoare 𝑌 data fiind dovada 𝑒, fapt notat prin
𝑋 ⊥ 𝑌|𝑒, dnd pentru orice 𝑥 ∈ 𝑑𝑜𝑚(𝑋) si 𝑦 ∈ 𝑑𝑜𝑚(𝑌)
avem 𝑋 = 𝑥 ⊥ 𝑌 = 𝑦|𝑒.
2023-2024
 Simetria independentei conditionate

• Propozitie. Relatia de independenta conditionata este

simetrica, adica 𝑕 ⊥ 𝑓|𝑒 dnd 𝑓 ⊥ 𝑕|𝑒.

• Demonstratie.
𝑃 𝑕 𝑒 = 𝑃(𝑕|𝑓 ∧ 𝑒) dnd
𝑃 ℎ∧𝑒 𝑃 ℎ∧𝑓∧𝑒
= dnd
𝑃 𝑒 𝑃 𝑓∧𝑒
𝑃 𝑓∧𝑒 𝑃 ℎ∧𝑓∧𝑒
= dnd
𝑃 𝑒 𝑃 ℎ∧𝑒
𝑃 𝑓 𝑒 = 𝑃(𝑓|𝑕 ∧ 𝑒)

2023-2024
 Independenta conditionata – Proprietati
• Propozitie. 𝑕 ⊥ 𝑓|𝑒 dnd 𝑃 𝑕 𝑒 𝑃 𝑓 𝑒 = 𝑃(𝑕 ∧ 𝑓|𝑒)
• Demonstratie.
𝑃 𝑕 𝑒 = 𝑃(𝑕|𝑓 ∧ 𝑒) dnd
𝑃 ℎ∧𝑒 𝑃 ℎ∧𝑓∧𝑒
= dnd
𝑃 𝑒 𝑃 𝑓∧𝑒
𝑃 ℎ∧𝑒 𝑃 𝑓∧𝑒
= 𝑃(𝑕 ∧ 𝑓 ∧ 𝑒) dnd
𝑃 𝑒
𝑃 ℎ∧𝑒 𝑃 𝑓∧𝑒 𝑃 ℎ∧𝑓∧𝑒
× = dnd
𝑃 𝑒 𝑃 𝑒 𝑃 𝑒
𝑃 𝑕|𝑒 𝑃 𝑓|𝑒 = 𝑃 𝑕 ∧ 𝑓 𝑒
• Observatie: 𝑕 ⊥ 𝑓 dnd 𝑃 𝑕 ∧ 𝑓 = 𝑃 𝑕 𝑃(𝑓)
2023-2024
 Problema zilei de nastere
• Se considera un grup de 𝑘 persoane. Sa se determine probabilitatea
ca doua persoane din grup sa aiba aceeasi zi de nastere (se vor
ignora anii bisecti).
• Fie 𝑛 = 365 numarul de zile din an. Fie 𝑋1 , 𝑋2 , … , 𝑋𝑘 variabile
aleatoare astfel incat 𝑋𝑖 ∈ *1,2, … , 𝑛+ reprezinta ziua de nastere a
persoanei 𝑖 ∈ *1,2, … , 𝑘+. Trebuie sa determinam:
𝑃 ∨1≤𝑖<𝑗≤𝑘 𝑋𝑖 = 𝑋𝑗 = 1 − 𝑃(∧1≤𝑖<𝑗≤𝑘 𝑋𝑖 ≠ 𝑋𝑗 )
• Exista 𝑛𝑘 valori posibile ale tuplului (𝑋1 , 𝑋2 , … , 𝑋𝑘 ). Dintre acestea
se pot alege 𝐴𝑘𝑛 = 𝑛 𝑛 − 1 … (𝑛 − 𝑘 + 1) tupluri (aranjamente de
𝑛 luate cate 𝑘) in care toate valorile sunt diferite. Rezulta:
𝑛−1 𝑛−2 … 𝑛−𝑘+1 𝑘−1 𝑛−𝑖
𝑃 ∧1≤𝑖<𝑗≤𝑘 𝑋𝑖 ≠ 𝑋𝑗 = = 𝑖=1 𝑛
𝑛𝑘−1
• Cat este 𝑘 minim astfel incat probabilitatea ca doua persoane sa fie
nascute in aceeasi zi sa fie cel putin 0.5?
2023-2024
 Solutie in Python
import numpy as np
import matplotlib.pyplot as plt
n = 365
p_min = 0.5
gasit = 0
prob = np.zeros(n+1)
for k in range(2,n+1):
prob_k = 1.0
for i in range(n-k+1,n):
prob_k *= i/n
prob[k] = prob_k
if prob_k <= 1-p_min and not gasit:
print("k =",k,"Prob =",1-prob_k)
gasit = 1
print(prob[2:])
plt.plot(range(2,n+1),1.0-prob[2:])
plt.show()
2023-2024
 Problems
1. 3 students X, Y and Z forgot to sign their papers in a written exam. Based
on students’ previous results, the teacher knows that those 3 students
wrote a good paper with probabilities 0.8, 0.7 and 0.6. After grading all
the papers, the teacher observed that among those 3 unsigned papers, 2
have good results and one has weak results. Assuming that students
worked independently, what is the probability that the weak paper was
done by student Z?
2. A program contains two modules. First module contains an error with
probability 0.2. Second module contains an error with probability 0.4. An
error in the first module will determine the program to freeze with
probability 0.5, while an error in the second module will determine the
program to freeze with probability 0.8. If there are errors in both
modules, the program will freeze with probability 0.9. Assuming that the
program froze, what is the probability that errors exist in both modules?

2023-2024
 Probability Distributions
• We are going to introduce few basic issues related to
probability distributions for both discrete and
continuous cases.
𝑃𝑋 𝑥 = 𝑃(𝑋 = 𝑥)
• We present basic definitions of:
– Expected value
– Variance, covariance and standard deviation
– Correlation

• We also introduce few examples of discrete and

continuous probability definitions
2023-2024
 Expected Value
• Let 𝑋 be a random variable and 𝜔 a possible world. Value 𝑥 of
𝑋 in 𝜔 is 𝑋(𝜔). We obtain:
𝜔 ⊨ 𝑋 = 𝑥 or equivalently 𝜔 ⊨ 𝑋 = 𝑋(𝜔)
• Definition. Let 𝑚 the measure function defined on the set of
possible world. Expected value of 𝑋 is denoted by 𝔼,𝑋- or 𝜇𝑋
and it is defined by the following weighted average:
𝜇𝑋 = 𝔼 𝑋 = Σ𝜔∈Ω 𝑚 𝜔 𝑋 𝜔 = Σ𝑥∈𝐷𝑋 𝑥Σ𝜔⊨X=𝑥 𝑚 𝜔 =
Σ𝑥∈𝐷𝑋 𝑥𝑃 𝑥
• Example. Let us consider a dice with 6 faces and random
variable 𝑁 representing the number of dots obtained by dice
throwing. Then dom 𝑁 = *1,2,3,4,5,6+. If the dice is fair:
1 1 1 1 1 1
𝔼 𝑁 = × 1 + × 2 + × 3 + × 4 + × 5 + × 6 = 3.5
6 6 6 6 6 6
2023-2024
 Properties of Expected Value

• Definition: Expected value of 𝑌 = 𝑔(𝑋) is defined by:

𝔼 𝑌 = 𝔼 𝑔 𝑋 = 𝑥 𝑔 𝑥 𝑃(𝑥)

• Properties:
1. 𝔼 𝑎𝑋 + 𝑏𝑌 + 𝑐 = 𝑎𝔼 𝑋 + 𝑏𝔼 𝑌 + 𝑐
2. If 𝑋 ⊥ 𝑌 then 𝔼 𝑋𝑌 = 𝔼 𝑋 𝔼,𝑌-

• HW: Prove these properties

2023-2024
 Example
• Determine the average number of dice rolls until we obtain number 6.
• Let 𝑋 be a random variable representing the number of dice rolls until we
obtain 6.
𝑑𝑜𝑚 𝑋 = 1,2, … = ℕ∗
• We must find 𝔼,𝑋-. Following the definition:
𝔼𝑋 = ∞ 𝑖=1 𝑖 ⋅ 𝑃(𝑋 = 𝑖)
1 5 1 5 2 1
𝑃 𝑋=1 = , 𝑃 𝑋=2 = ⋅ , 𝑃 𝑋=3 = ⋅ ,…
6 6 6 6 6
1 ∞ 5 𝑖−1 1
𝔼𝑋 = ⋅ 𝑖=1 𝑖 ⋅ = ⋅𝑆
6 6 6
• We observe that:
5 5 2 5 1 5 5
𝑆 = 1+ + +⋯ + ⋅𝑆 = 5+ ⋅𝑆 = 6+ ⋅𝑆
6 6 6 1−6 6 6

It follows that 𝑆 = 62 so 𝔼 𝑋 = 6.
2023-2024
 Homework

• Let us consider an urn with 𝑟 red balls and 𝑏 blue balls.

We extract at random 2 balls. Determine (using an
algorithm + Python program) the smallest number of
balls in the urn such that the probability to extract two
𝑚
red balls is 𝑝 = , where 𝑚 < 𝑛 ∈ ℕ∗ are two given
𝑛
natural numbers (inputs).

2023-2024
 Variance and Standard Deviation
• Definition.
– Variance of random variable 𝑋, denoted by 𝑉𝑎𝑟(𝑋), is
defined by:
2
𝑉𝑎𝑟 𝑋 = 𝔼, 𝑋 − 𝔼 𝑋 -
– Standard deviation of random variable 𝑋, denoted by
𝑆𝑡𝑑(𝑋), is defined by:
𝜍𝑋 = 𝑆𝑡𝑑 𝑋 = 𝑉𝑎𝑟 𝑋
• Property.
𝑉𝑎𝑟 𝑋 = 𝔼 𝑋 2 − 𝜇𝑋2
• HW: Prove this property
2023-2024
 Covariance
• Definition. Covariance of two random variables 𝑋 and
𝑌, denoted by 𝐶𝑜𝑣(𝑋, 𝑌) or 𝜍𝑋𝑌 , defined by:
𝐶𝑜𝑣 𝑋, 𝑌 = 𝔼, 𝑋 − 𝔼 𝑋 𝑌 − 𝔼 𝑌 -

• Properties:
1. 𝐶𝑜𝑣 𝑋, 𝑌 = 𝔼 𝑋𝑌 − 𝔼 𝑋 𝔼 𝑌
2. If 𝑋 ⊥ 𝑌 then 𝐶𝑜𝑣 𝑋, 𝑌 = 0
3. 𝑉𝑎𝑟 𝑋 = 𝐶𝑜𝑣(𝑋, 𝑋)

• HW: Prove these properties

2023-2024
 Correlation
• Definition. Correlation of two random variables 𝑋 and 𝑌,
denoted by 𝜌𝑋𝑌 , is defined by:
𝐶𝑜𝑣 𝑋, 𝑌
𝜌𝑋𝑌 =
𝑆𝑡𝑑 𝑋 𝑆𝑡𝑑 𝑌
• Property:
𝜌𝑋𝑌 ≤ 1
• Observations:
1. If 𝜌𝑋𝑌 = 0 then it means that 𝑋 and 𝑌 are uncorrelated
2. If 𝜌𝑋𝑌 ≈ 0 then it means that 𝑋 and 𝑌 are weakly correlated.
3. If 𝜌𝑋𝑌 > 0 then it means that 𝑋 and 𝑌 are positively correlated.
4. If 𝜌𝑋𝑌 < 0 then it means that 𝑋 and 𝑌 are negatively correlated.
5. If |𝜌𝑋𝑌 | ≈ 1 then it means that 𝑋 and 𝑌 are strongly correlated
(positively or negatively depending on sign of 𝜌𝑋𝑌 )
2023-2024
 Bernoulli Distribution

• Definition. Let 𝑝 ∈ ,0,1-. A random variable 𝑋 with 𝑑𝑜𝑚 𝑋 =

*0,1+ such that:
𝑞 =1−𝑝 if 𝑥 = 0
𝑃 𝑥 =
𝑝 if 𝑥 = 1
is called Bernoulli variable with Bernoulli distribution.

• Interpretation. Value of a Bernoulli variable models the

outcome of a Boolean experiment with two possible outputs
success (value 0) with probability 𝑝 or failure (value 1) with
probability q = 1 − 𝑝. This experiment is called Bernoulli
experiment .
2023-2024
 Properties of Bernoulli Distribution

PMF is defined by 𝑃 𝑥 = 𝑝 𝑥 𝑞1−𝑥 for 𝑥 ∈ *0,1+

𝜇𝑋 = 𝔼 𝑋 = 0 ⋅ 𝑃 0 + 1 ⋅ 𝑃 1 = 𝑝
𝔼 𝑋 2 = 02 ⋅ 𝑃 0 + 12 ⋅ 𝑃 1 = 𝑝

𝑉𝑎𝑟 𝑋 = 𝔼 𝑋 2 − 𝜇𝑋2 = 𝑝 − 𝑝2 = 𝑝𝑞

𝜍𝑋 = 𝑉𝑎𝑟 𝑋 = 𝑝𝑞

2023-2024
 Binomial Distribution
• Definition. A random variable 𝑋 represents the
total number of successes from a sequence of 𝑛
independent Bernoulli experiments cu success
probability 𝑝 has binomial distribution.

• For a binomial random variable 𝑋 we have:

𝑑𝑜𝑚 𝑋 = *0,1,2, … , 𝑛+
𝑛 𝑥 𝑛−𝑥
𝑃 𝑥 = 𝑝 𝑞 pentru 𝑥 = 0,1,2, … , 𝑛
𝑥
𝑛
𝑝+ 1−𝑝 =⋯ 2023-2024
 Properties of Binomial Distribution

• 𝑋 can be written as sum of 𝑛 Bernoulli variables 𝑋𝑖

with probability of de success 𝑝, i.e.:
𝑋 = 𝑋1 + 𝑋2 + ⋯ 𝑋𝑛
• We obtain:
𝔼 𝑋 = 𝔼 𝑋1 + 𝔼 𝑋2 + ⋯ + 𝔼 𝑋𝑛 = 𝑛𝑝
𝑉𝑎𝑟(𝑋) = 𝑉𝑎𝑟(𝑋1 ) + 𝑉𝑎𝑟(𝑋2 ) + ⋯ + 𝑉𝑎𝑟(𝑋𝑛 ) = 𝑛𝑝𝑞

2023-2024
 Discrete Uniform Distribution
• Definition. A random variable 𝑋 that can take 𝑛 ≥ 2 equally
probable values has a discrete uniform distribution.
𝑑𝑜𝑚 𝑋 = 𝐴 = *𝑎1 , 𝑎2 , … , 𝑎𝑛 + and 𝐴 = 𝑛 ≥ 2
1
𝑃 𝑥 = for 𝑥 ∈ 𝐴
𝑛
• If 𝐴 ⊂ ℝ then:
𝑎1 + 𝑎2 + ⋯ + 𝑎𝑛
𝔼𝑋 = 𝑥𝑃 𝑥 =
𝑛
𝑥∈≦
2 2 2
𝑎1 + 𝑎 2 + ⋯ + 𝑎 𝑛
𝔼 𝑋2 = 𝑥 2𝑃 𝑥 =
𝑛
𝑥∈≦
𝑉𝑎𝑟 𝑋 = 𝔼 𝑋 2 − 𝔼 𝑋 2 = ⋯ HW !
𝜍 𝑋 = 𝑉𝑎𝑟 𝑋 = ⋯ HW !
2023-2024
 Continuous Uniform Distribution
• Definition. Let 𝑋 be a real random variable with domain ,𝑎, 𝑏-.
If probability density of 𝑋 is constant then 𝑋 has a continuous
uniform distribution with pdf and cdf defined by:
1
𝑓 𝑥 = for 𝑥 ∈ ,𝑎, 𝑏-
𝑏−𝑎
𝑥−𝑎
𝐹 𝑥 = for 𝑥 ∈ ,𝑎, 𝑏-
𝑏−𝑎
• If 𝑎 = 0 and 𝑏 = 1 then 𝑋 has a standard continuous uniform
distribution.
• Properties.
+∞
𝑏+𝑎
𝔼𝑋 = = 𝑥 ⋅ 𝑓 𝑥 𝑑𝑥
2 −∞
𝑏−𝑎 2
𝑉𝑎𝑟 𝑋 = 2023-2024
 Normal Distribution
• Definition: O random variable normally (or Gaussian)
distributed has probability density given by:
𝑥−𝜇 2
1 −
𝑓 𝑥 = 𝑒 2𝜎2 pentru 𝑥 ∈ ℝ
𝜍 2𝜋
• Sums and averages have a normal distribution.
Moreover, measurement errors and fluctuations
resulting by cumulating small deviations have normal
distribution. Consequently, this distribution models
well values obtained by measuring physics variables:
weight, length, temperature, voltage and current,
pollution level, salinity, etc.
2023-2024
 Parameters of Normal Distribution
• Parameter 𝜇 is called expected value, as:
𝔼𝑋 =𝜇
• Parameter 𝜍 is called standard deviation, as:
𝑆𝑡𝑑 𝑋 = 𝜍
• Sometimes:
– 𝜇 is called localization parameter, as it specifies the
reference value around which the values of the
variable are grouped
– 𝜍 is called scaling parameter , as it specifies how the
variable values of the variable are “spread” around
its reference value.
2023-2024
 Standard Normal Distribution
• Definition: If 𝜇 = 0 and 𝜍 = 1 then we have a standard
normal distribution.
• Pdf and cdf of a standard normal random variable are:
1 𝑥2
𝜙 𝑥 =𝑓 𝑥 = 𝑒− 2 for 𝑥 ∈ ℝ
2𝜋
𝑥 𝑧2
1
Φ 𝑥 =𝐹 𝑥 = 𝑒 − 2 𝑑𝑧 for 𝑥 ∈ ℝ
−∞ 2𝜋
• Let 𝑋 be a random variable normally distributed. Then:
𝑋−𝜇
𝑍=
𝜍
is standard normally distributed.
• Let 𝑍 be a standard normal random variable. Then:
𝑋 = 𝜍𝑍 + 𝜇
Is normally distributed with parameters 𝜇 and 𝜍. 2023-2024
 Central Limit Theorem – Intuition
• The mean of a sufficiently large number of independent and
identically distributed random variables is approximately
normally distributed.
• This can be interpreted as if you have a population with mean 𝜇
and standard deviation 𝜍 and take sufficiently large random
samples from the population with replacement, then the
sample mean will be approximately normally distributed.
• Let 𝑛 be the sample size and let 𝑋 be the sample mean. Then:
𝜇𝑋 = 𝜇
𝜍
𝜍𝑋 =
𝑛
• This situation can be experimentally observated for sufficiently
large samples, i.e. 𝑛 ≥ 30.
2023-2024
Centrol Limit Theorem – Mathematical Formulation
• Theorem: Let be 𝑛 independent random variables such that
𝔼 𝑋𝑖 = 𝜇 and 𝑆𝑡𝑑 𝑋𝑖 = 𝜍 for 1 ≤ 𝑖 ≤ 𝑛. Let 𝑆𝑛 = 𝑛𝑖=1 𝑋𝑖 .
Then:
𝑆𝑛 𝑆
−𝔼 𝑛
𝑆𝑛 − 𝔼 𝑆𝑛 𝑆𝑛 − 𝑛𝜇 𝑛 𝑛
𝑍𝑛 = = =
𝑆𝑡𝑑 𝑆𝑛 𝜍 𝑛 𝑆𝑛
𝑆𝑡𝑑
𝑛
converges in cdf to a standard normal random variable:
lim 𝐹𝑍𝑛 𝑧 = Φ 𝑧 for all 𝑧
n→∞
• Observations:
– 𝑆𝑛 diverges
𝑆𝑛 𝑆𝑛
– converges and 𝑉𝑎𝑟( ) converges to zero.
𝑛 𝑛
𝑆𝑛
– behaves like a normal random variable.
𝑛 2023-2024
 Beta Distribution
• Definition: The value of a random variable 𝑋 represents the
success probability of a Bernoulli event, i.e. 𝑑𝑜𝑚(𝑋) = ,0,1-.
Variable 𝑋 has a Beta distribution with parameters 𝛼 and 𝛽 that
represent the number of observed successes and respectively
failures in a sequence of 𝛼 + 𝛽 successive event repetitions.
• Pdf of a random variable with Beta distribution is defined by:
1 Functia Beta
𝑓 𝑥; 𝛼, 𝛽 = 𝑥 𝛼−1 1 − 𝑥 𝛽−1
𝑩 𝜶, 𝜷

• This distribution is defined for any 𝛼, 𝛽 > 0 with domain:

0,1 daca 𝛼, 𝛽 ≥ 1
𝑑𝑜𝑚 𝑋 =
0,1 daca 0 < 𝛼, 𝛽 < 1

2023-2024
 Beta Function
• Beta function with parameters 𝛼, 𝛽 > 0 is defined as:
Γ 𝛼 Γ 𝛽
𝐵 𝛼, 𝛽 =
Γ 𝛼+𝛽
where Γ Gamma function defined by:
∞
Γ 𝑧 = 𝑥 𝑧−1 𝑒 −𝑥 𝑑𝑥
0
for each complex number 𝑧 ∈ ℂ ∖ *−1, −2, … , −∞+.
• Γ function represents the generalization of factorials. If 𝑛 ≥ 1 is
a natural number then:
Γ 𝑛 = 𝑛−1 !
• If 𝛼, 𝛽 ≥ 1 are natural numbers then:
𝛼 𝛼𝛽 𝛼−1
𝑓 𝑥; 𝛼, 𝛽 = 𝐶𝛼+𝛽 𝑥 1 − 𝑥 𝛽−1
𝛼+𝛽
2023-2024
 Properties of Beta Distribution
• If 𝑋 has a Beta distribution with parameters 𝛼, 𝛽 then:
𝛼
𝔼 𝑋; 𝛼, 𝛽 =
𝛼+𝛽
𝛼𝛽
𝑉𝑎𝑟 𝑋; 𝛼, 𝛽 =
𝛼+𝛽+1 𝛼+𝛽 2

• Observations:
– Expected value represents the ratio of successes to the total
number of trials.
– If 𝛼, 𝛽 → ∞ then 𝑉𝑎𝑟 𝑋; 𝛼, 𝛽 → 0, i.e. values of 𝑋 tend to
concentrate around the average value if the number of trials
grows to infinity.
2023-2024
 Histograms
• A histogram illustrates the of pmf or pdf of a random variable
(discrete or continuous).
• We divide the range of values of the variable in equal intervals
(engl. bins) and we count the values from each bin.
• A histogram can illustrate the frequency of values or the
relative frequency of values of the variable for each bin.
– Frequency histogram is represented by a set of bars (one bar per each
interval) such that the height of each bar represents the number of
values from each bin.
– Relative frequency histogram represented by a set of bars (one bar per
each interval) such that the height of each bar represents the
proportion of values from each bin.

2023-2024
 Simulating Binomial Distribution
• We are using Python’s numpy.random package for random
sampling.
• We set the distribution parameters:
– probability of success of an event 𝑝 ∈ (0,1)
– number 𝑛 of trials
• Experiment: choose 𝑥 ∈ (0,1) by uniform random sampling 𝑛
times and count how many times 𝑥 < 𝑝 (event succeeded).
• Repeat experiment of a large number 𝑁 of times recording how
many times the event succeeded for each repetition.
• Determine all integers 𝑘 from 0,1, … , 𝑛 for which the event
succeeded and for how many times.
2023-2024
 Python Solution
import numpy as np
import matplotlib.pyplot as plt
N = 50000 # no. of experiment repetitions
n = 30 # no. of trials / experiment
p = 0.6 # probability of success
q = 1-p # probability of failure
u = np.random.rand(n,N)
y = u<p
x = sum(y)
unique, counts = np.unique(x,
return_counts=True)
unique_all = [i for i in range(0,n+1)]
counts_all = [0 for i in range(0,n+1)]
for i in range(unique.size):
counts_all[unique[i]] = counts[i]
hist = counts_all/sum(counts_all)
plt.figure(1)
plt.plot(unique_all,hist,'.')
plt.figure(2)
plt.hist(x,[i-0.5 for i in range(n+2)],
rwidth=1.0)
2023-2024
 Verification Using binomial
import numpy as np
import matplotlib.pyplot as plt
N = 50000 # no. repetitions of the experim.
n = 30 # number of trials / experiment
p = 0.6 # probability of success
q = 1-p # probability of failure
out = np.random.binomial(n,p,N)
unique, counts = np.unique(out,
return_counts=True)
unique_all = [i for i in range(0,n+1)]
counts_all = [0 for i in range(0,n+1)]
for i in range(unique.size):
counts_all[unique[i]] = counts[i]
hist = counts_all/sum(counts_all)
plt.figure(1)
plt.plot(unique_all,hist,'.')
plt.figure(2)
plt.hist(out,out.max()-out.min())

2023-2024