COSC1127/1125 Artificial Intelligence
School of Computing Technologies Semester 2, 2021
Prof. ̃a
Tutorial No. 7 Probability Reasoning
We use P (·) (sometimes also written Pr(·)) to refer to a probability function or probability distribution. When using an upper letter (e.g., X or Cavity), we refer to the random variable; when using lowercases we refer to a specific value of the corresponding random variable. So, P(Cavity) is a probability distribution over variable Cavity; whereas P (cavity) is a shorthand for P (Cavity = true) and P (¬cavity) is a shorthand for P (Cavity = f alse).
PART1: Probabilities……………………………………………………………………..
(a) Prove that if X is a random variable over a finite range R, then P (x∈R X = x) = 1
(b) Would it be rational for an agent to hold the three beliefs P(A) = 0.4, P(B) = 0.3, and P(A∨B) = 0.5? If so, what range of probabilities would be rational for the agent to hold for A ∧ B? Make up a table like Figure 13.2 or this one, and show how it supports your argument about rationality. Then draw another version of the table where P(A∨B) = 0.7. Explain why it is rational to have this probability, even though the table shows one case that is a loss and three that just break even. (Hint: what is Agent 1 committed to about the probability of each of the four cases, especially the case that is a loss?)
Solution: (Note: this is a partial solution)
ThisfollowsfromtheaxiomP(a∨b)=P(a)+P(b)−P(a∧b)andthatP(X =x1 ∧X =x2)=0 as a random variable can only take one value in an event. With that key insight, now build the full proof for the statement…
Solution: (Note: this is a partial solution)
Probably the easiest way to keep track of what’s going on is to look at the probabilities of the atomic events. A probability assignment to a set of propositions is consistent with the axioms of probability if the probabilities are consistent with an assignment to the atomic events that sums to 1 and has all probabilities between 0 and 1 inclusive. We call the probabilities of the atomic events a, b, c, and d, as follows:
We then have the following equations:
B ¬B Aab ¬A c d
P(A) = …. = 0.4
P(B) = …. = 0.3 P (A ∨ B) = …. = 0.5
P (T rue) = a + b + c + d = 1
1
COSC1127/1125 – AI Tutorial 7: Probability Reasoning Page 2 of 6
From these, it is straightforward to infer that a = 0.2, b = 0.2, c = 0.1, and d = 0.5. Therefore, P (A ∧ B) = a = 0.2. Thus the probabilities given are consistent with a rational assignment, and the probability P (A ∧ B) is exactly determined.
If P(A ∨ B) = 0.7, then P(A ∧ B) = a = 0. Thus, even though the bet outlined in Figure 13.3 loses if A and B are both true, the agent believes this to be impossible so the bet is still rational.
(c) Considerthedomainofdealing5-cardpokerhandsfromastandarddeckof52cards,undertheassumption that the dealer is fair.
i. How many atomic events are there in the joint probability distribution (i.e., how many 5-card hands are there)?
Solution: (Note: this is a partial solution)
This is a classic combinatorics question. The point here is to refer to the relevant axioms of proba- bility, principally, the following axioms:
(1) All probabilities are between 0 and 1. 0 ≤ P (A) ≤ 1
(2) P(True) = 1 and P(False) = 0
(3) The probability of a disjunction is given by P(A ∨ B) = P(A) + P(B) − P(A ∧ B)
The question also helps students to grasp the concept of joint probability distribution over all possi- ble states of the world.
It is important to note here that the hand {♣2, ♦3, ♥4, ♦5, ♠6}, is identical to the hand {♠6, ♦5, ♥4, ♦3, ♣2}. If the order of the cards dealt mattered, then the number of hands would simply be 52! , or
47!
52 × 51 × 50 × 49 × 48, because there are 52 choices for the first card, 51 for the second, 50 for the third, and so on.
Generally though, when dealing a hand of cards, it doesn’t matter the order in which they are dealt. So, you need to divide this number by the number of possible permutations for a hand of 5 cards, which is 5 × 4 × 3 × 2 × 1 = 5!. This means that the total number of 5 card hands that are possible in a 52 card deck is …(your turn)
In combinatorics this is written as the binomial coefficient 52, and means: “out of 52 cards, choose
5”. In general, n = n! . k k!(n−k)!
5
ii. What is the probability of each atomic event?
iii. What is the probability of being dealt a royal straight flush (the ace, king, queen, jack and ten of the same suit)?
Solution: (Note: this is a partial solution)
By the fair-dealing assumption, each of these is equally likely. Each hand therefore occurs with probability . . . (your turn)
Solution: (Note: this is a partial solution)
There are four hands that are royal straight flushes (one in each suit). By axiom 3, since the events
RMITAI2021(Semester2)-SebastianSardin ̃a Exercise1continuesonthenextpage…
COSC1127/1125 – AI Tutorial 7: Probability Reasoning Page 3 of 6
are mutually exclusive, the probability of a royal straight flush is just the sum of the probabilities of the atomic events, i.e., . . . (your turn)
iv. What is the probability of being dealt four-of-a-kind (i.e., four cards of different suit but same face value)?
PART 2: Conditional probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . (a) Prove,formally,thatP(A|B∧A)=1.
Solution: (Note: this is a partial solution)
Again, we examine the atomic events that are four-of-a-kind events. There are 13 possible kinds and for each, the fifth card can be one of 48 possible other cards. The total probability is therefore . . . (your turn)
Solution: (Note: this is a partial solution)
You need to use the definition of conditional probability, P (X | Y ) = P (X ∧ Y ) | P (Y ), and the definitions of the logical connectives. It is not enough to say that if B ∧ A is “given”, then A must be true.FromthedefinitionofconditionalprobabilityandthefactthatA∧A ⇐⇒ Aandthatconjunction is commutative and associative we have,
(You fill in the dots)
P (A | B ∧ A) = · · · = 1
(b) Consider again the domain of dealing 5-card poker hands from a standard deck of 52 cards, under the
assumption that the dealer is fair. You are told that the probability drawing two cards from a deck of 52
and them both being an ace is 1 . You take the deck and draw the first card — it is the ace of spades! 221
What is the probability that the second card you draw will also be an ace? (even though this is easy to work out using binomials, use conditional probability for this question)
Solution: (Note: this is a partial solution)
Even though we are given the suit of the first ace, that information is immaterial as the probability you
are given does not specify suit, so we can ignore it. If we let P (A) be the probability of drawing the
first ace, its value is simply 4 . Let P (A ∧ B) be the probability of drawing two aces in a row, which 52
is 1 . We can work out the probability of drawing a second ace, given we have already drawn an ace, 221
P (B | A), using the definition of conditional probability.
P (B | A) = . . .
(your turn)
(c) Given the full joint distribution shown in the table below, calculate the following:
toothache ¬toothache catch ¬catch catch ¬catch
cavity .108 .012 .072 .008 ¬cavity .016 .064 .144 .576
Note that P(Cavity) denotes a vector of values for the probabilities of each individual state of Cavity. Also note here we use the uppercase (e.g., Cavity) to denote a variable, whereas lowercase (e.g., cavity) to denote a constant.
RMITAI2021(Semester2)-SebastianSardin ̃a Exercise2continuesonthenextpage…
COSC1127/1125 – AI Tutorial 7: Probability Reasoning Page 4 of 6 i. P(toothache)
ii. P(Cavity)
iii. P(Toothache|cavity)
iv. P(Cavity|toothache∨catch)
(d) After your yearly checkup, the doctor has bad news and good news. The bad news is that you tested positive for serious disease and that the test is 99% accurate (i.e., the probability of testing positive when you do have the disease is 0.99, as is the probability of testing negative when you don’t have the disease). The good news is that this is a rare disease, striking only 1 in 10,000 people of your age. Why is it good news that the disease is rare? What are the chances that you actually have the disease?
Solution: (Note: this is a partial solution)
We need to sum up over all entries in the table that relate to toothache. P (toothache) = . . . (your turn)
Solution: (Note: this is a partial solution)
This time we are dealing with a variable, so we need to give a vector of values, i.e., P(A) = ⟨P (a), P (¬a)⟩.
P (Cavity) = . . . (your turn)
Solution: (Note: this is a partial solution)
Here we have a conditional probability of a variable, given a constant. So the solution will be a vector again of the form P(A | b) = P(a∧b), P(¬a∧b)
P (b) P (b) P (T oothache | cavity) = . . . (your turn)
Solution: (Note: this is a partial solution)
P (Cavity | toothache ∨ catch) = . . . (your turn – remember that the values in your vector should add up to 1)
Solution: (Note: this is a partial solution)
We are given the following information:
P(test | disease) = ………… P(¬test | ¬disease) = ………… P(disease) = …………
test
where test means that the test is positive. What the patient is concerned about is P (disease | test). Roughly speaking, the reason it is a good thing that the disease is rare is that P(disease | test) is proportional to P(disease), so a lower prior probability for Disease will mean a lower value for P(disease | test). By and large, if 10,000 people take the test, we expect 1 to actually have the disease, and most likely test positive, while the rest do not have the disease, but 1% of them (about 100 people) will test positive anyway, so P (disease | test) will be about 1 in 100. More precisely, using the following:
RMITAI2021(Semester2)-SebastianSardin ̃a Exercise2continuesonthenextpage…
COSC1127/1125 – AI Tutorial 7: Probability Reasoning Page 5 of 6
P (disease | test) = P (test | disease)P (disease) P (test)
=………… =………… = 0.009804
Note that in the above, to calculate P (test), we need to sum over all other hidden variables, but in this case there is only one, that is Disease :
P (test) = P (test ∧ disease) + P (test ∧ ¬disease) by the product rule, we have:
P (test ∧ disease) = P (test | disease)P (disease)
P (test ∧ ¬disease) = P (test | ¬disease)P (¬disease)
The moral is that when the disease is much rarer than the test accuracy, a positive result does not mean the disease is likely. A false positive reading remains much more likely.
See a video explaining the solution HERE.
(e) Prove,formally,thatP(A∧B∧C)=P(A|B∧C)×P(B|C)×P(C).
Solution: (Note: this is a partial solution)
Starting with the definition of conditional probability,
P (X | Y ) = P (X ∧ Y ) P(Y)
we can rearrange to give the product rule:
P(X∧Y)=… (yourturn)
Using the product rule and substituting X/A and Y/[B ∧ C] into P(A ∧ B ∧ C), we have, P(A∧B∧C)=… (yourturn)
Finally, applying the product rule again to P (B ∧ C ) and substituting X/B and Y /C gives: P(A∧B∧C)=… (yourturn)
(f) Prove Bayes’ Theorem: P(A | B) = P(B | A) × P(A). P(B)
Solution: (Note: this is a partial solution)
The probability of two events A and B happening, P (A ∧ B), is the probablity of A, P (A), times the probability of B given that B has occurred, P (B | A).
P(A∧B)=… (yourturn)
RMITAI2021(Semester2)-SebastianSardin ̃a Exercise2continuesonthenextpage…
COSC1127/1125 – AI Tutorial 7: Probability Reasoning Page 6 of 6
On the other hand, the probability of A and B is also equal to the probability of B times the probability of A given B.
Equating the two yields: and thus,
P(A∧B)=… (yourturn) P(B)×P(A|B)=… (yourturn)
P (A | B) = P (A) × P (B | A) P(B)
(g) For each of the following statements, either prove it is true or give a counterexample: i. IfP(a|b,c)=P(b|a,c),thenP(a|c)=P(b|c)
ii. IfP(a|b,c)=P(a),thenP(b|c)=P(b)
iii. IfP(a|b)=P(a),thenP(a|b,c)=P(a|c)
End of tutorial worksheet
Solution: True. By the product rule we know P(b,c)P(a | b,c) = P(a,c)P(b | a,c), which by assumption reduces to P (b, c) = P (a, c). Dividing through by P (c) gives the result.
Solution: (Note: this is a partial solution)
False. The statement P (a | b, c) = P (a) merely states that a is independent of b and c, it makes no claim regarding the dependence of b and c.
A counter-example: …………
Solution: (Note: this is a partial solution)
False. While the statement P (a | b) = P (a) implies that a is independent of b, it does not imply that a is conditionally independent of b given c.
A counter-example: …………
RMIT AI 2021 (Semester 2) – ̃a