Probability & Statistics HANDOUT 2. Axiomatic denition of probability. Conditional probability.

AXIOMATIC DEFINITION OF PROBABILITY We call probability on the sample space S a function P which associates to every event A P(S) a number P (A), called probability of A, such that the following conditions (axioms) are fullled: i) P (A) 0, A P(S); ii) P (S) = 1; iii) A B = P (A B) = P (A) + P (B), A, B P(S). The function P : P(S) R1 is called probability measure. + The pair (S, P ) is called probability space. Properties of the probability measure: if A = then P (A) = 0.
k

if A = {e1 , e2 , ..., ek }, where ei are elementary events, then P (A) =

i=1

P ({ei }).

P (A) = 1 P (A). if A, B P(S) and A B then P (A) P (B).

n n

if A1 , A2 , ..., An P(S) and Ai Aj = , i = j, then P

i=1

Ai

=
i=1

P (Ai ).

P (A B) = P (A) + P (B) P (A B) for any A, B P(S).

n n

for any A1 , A2 , .., An P(S) we have: P

i=1 n

Ai Ai
i=1

i=1

P (Ai ) , n N.
n

for any A1 , A2 , .., An P(S) we have: P

1
i=1

P (Ai ) , n N.

INDEPENDENT AND DEPENDENT EVENTS The events A and B from P(S) are called independent if P (A B) = P (A) P (B). if A, B P(S) are independent events having non-zero probabilities, then A B is a set which contains at least one point ei of the sample space S. The events A1 , A2 , ..., An are (totally) independent if for any 1 i1 < i2 < ... < is n, we have: P (Ai1 Ai2 ... Ais ) = P (Ai1 ) P (Ai2 ) ... P (Ais ) The events A1 , A2 , ..., An P(S) are k-independent, k n, if the events of any family of k events (out of these n) are independent. 2 3 n the independence of the events A1 , A2 , ..., An means that Cn + Cn + ... + Cn = 2n n 1 relations have to be satised. if A and B are independent events, then the events A and B; A and B; A and B are also independent. The events B1 , B2 , ..., Bk P(S) form a partition of the sample space S if the following conditions are fullled:
k

i) Bi Bj = ,

i = j

ii)
i=1

Bi = S

iii) P (Bi ) > 0,

i = 1, 2, ..., k

The events of a partition of the sample space are called hypotheses. Two partitions A1 , A2 , ..., An and B1 , B2 , ..., Bk of the sample space S are independent if P (Ai Bj ) = P (Ai ) P (Bj ) for any i = 1, 2, ..., n, j = 1, 2, ..., k.

CONDITIONAL PROBABILITY The probability of the event A conditioned by the occurrence of the event B (probability of A, given B) is denoted by P (A|B) or PB (A) and is dened by P (A|B) = P (A B) P (B) if P (B) = 0

Properties of the conditional probability: 0 P (A|B) 1 P (S|B) = 1 if A1 , A2 are incompatible then P (A1 A2 |B) = P (A1 |B) + P (A2 |B) if A and B are independent events then P (A|B) = P (A) and P (B|A) = P (B) Probability of the intersection of a series of events. If A1 , A2 , ..., An are events such that P (A1 A2 ... An ) = 0 (i.e. they are compatible), then P (A1 A2 ... An ) = P (A1 ) P (A2 |A1 ) P (A3 |(A1 A2 )) ... P (An |(A1 ... An1 )) consequence: if A1 , A2 , ..., An are independent events then P (A1 A2 ... An ) = P (A1 ) P (A2 ) ... P (An ) Probability Trees are very useful for calculating probabilities of intersections of events. Law of Total Probability. If the events A1 , A2 , ..., An form a partition of the sample space S and X P(S), then:
n

P (X) =
i=1

P (Ai ) P (X|Ai )

Bayes Theorem. If the events A1 , A2 , ..., An form a partition of the sample space S and are the cause of the occurrence of an event X, then: P (Ak ) P (X|Ak ) P (Ak |X) = n P (Ai ) P (X|Ai )
i=1

the probabilities P (Ai ), P (X|Ai ), i = 1, n are called prior probabilities the probabilities P (Ai |X) are called posterior probabilities the event X is called evidence Before we receive the evidence, we have a set of prior probabilities P (Ai ), i = 1, n for the hypotheses. If we know the correct hypothesis, we know the probability for the evidence. That is, we know P (X|Ai ), i = 1, n. If we want to nd the probabilities for the hypothesis, given the evidence, that is, we want to nd P (Ai |X), we can use Bayes theorem.

EXERCISES
Exercise 1. A coin is tossed three times. What is the probability that exactly two heads occur, given that: a) the rst outcome was a head? b) the rst outcome was a tail? c) the rst two outcomes were heads? d) the rst two outcomes were tails? e) the rst outcome was a head and the third outcome was a head? Exercise 2. A die is rolled twice. What is the probability that the sum of the faces is greater than 7, given that: a) the rst outcome was a 4? b) the rst outcome was greater than 3? c) the rst outcome was a 1? d) the rst outcome was less than 5? Exercise 3. 50 lottery tickets are sold, only one of them being the winner. Each buyer can choose from those tickets which have not been previously sold. Which player do you prefer to be? The rst? The second? The third? Exercise 4. Suppose that 9 out of 500 chips in a particular box are defective, and suppose that 3 chips are sampled at random from the box, without replacement. If each of the 3 chips sampled is tested to determine whether it is defective (1) or satisfactory (0), then the sample space has 8 outcomes. Compute the probability of each of these 8 outcomes. Exercise 5. A system has 4 computers. Computer 1 works with a probability of 0.88, computer 2 works with a probability of 0.78, computer 3 works with a probability of 0.92 and computer 4 works with a probability of 0.85. Suppose that the operations of the computers are independent of each other. What is the probability that the system works in each of the following cases? (a) the system works only if all four computers are working; (b) the system works if at least one of the four computers works; (c) the system works if at least three of the four computers work. Exercise 6. Two players A and B play a game of 32 cards. When a card is drawn from the deck, it is not re-introduced. Player A draws two cards and wins if he gets two gures (J, Q, K or A). If A does not win, player B draws a card, and wins if he gets a gure. Which player has bigger chances to win? What is the probability that the game does never stop? Exercise 7. A box contains 10 white balls and 20 red balls. n balls are extracted successively, putting back the ball if it is white and not putting it back if it is red. What is the probability that exactly one red ball is extracted? Exercise 8. Two players toss successively a coin, with the probability p that heads occur. Player A starts the game. Player A wins if he obtains heads, and in this case, the game ends. Otherwise, player B tosses the coin, and wins if he obtains tails, in which case, the game ends. 1. What is the probability that player A wins exactly at the n-th toss? 2. What is the probability that A wins? 3. What is the probability that the game does not end? 4. What should be the value of p such that the two player have equal chances of winning the game? Exercise 9. Bag A contains 3 red balls and 7 blue balls. Bag B contains 8 red balls and 4 blue balls. Bag C contains 5 red balls and 11 blue balls. A bag is chosen at random, with each bag being equally likely to be chosen, and then a ball is chosen at random from that bag. Calculate the probabilities that: (a) a red ball is chosen; (b) a blue ball is chosen; (c) a red ball from bag B is chosen. If it is known that a red ball is chosen, what is the probability that it comes from bag A? If it is known that a blue ball is chosen, what is the probability that it comes from bag B? Exercise 10. A doctor assumes that a patient has one of three diseases d1, d2, or d3. Before any test, he assumes an equal probability for each disease. He carries out a test that will be positive with probability 0.8 if the patient has d1, 0.6 if he has disease d2, and 0.4 if he has disease d3. Given that the outcome of the test was positive, what probabilities should the doctor now assign to the three possible diseases? Exercise 11. A doctor gives a patient a test for a particular cancer. Before the results of the test, the only evidence the doctor has to go on, is that 1 man in 1000 has cancer. Experience has shown that, in 99 percent of the cases in which cancer is present, the test is positive; and in 95 percent of the cases in which it is not present, it is negative. If the test turns out to be positive, what probability should the doctor assign to the event that cancer is present? Exercise 12. The weather on a particular day is classied as either cold, warm or hot. There is a probability of 0.15 that it is cold and a probability of 0.25 that it is warm. In addition, on each day it may either rain or not rain. On cold days, there is a probability of 0.30 that it will rain, on warm days, there is a probability of 0.40 that it will rain and on hot days, there is a probability of 0.50 that it will rain. If it is not raining on a particular day, what is the probability that it is cold? Exercise 13. In London, half of the days have some rain. The weather forecaster is correct 2/3 of the time, i.e., the probability that it rains, given that she has predicted rain, and the probability that it does not rain, given that she has predicted that it wont rain, are both equal to 2/3. When rain is forecast, Mr. Pickwick takes his umbrella. When rain is not forecast, he takes it with probability 1/3. Find: a) the probability that Pickwick has no umbrella, given that it rains; b) the probability that it doesnt rain, given that he brings his umbrella.