Bayesian Decision Theory
Two (complementary) hypotheses: H1 and H2. An event A.
Prior Odds: P(H1 ) Data:A Likelihood ratio: P(A| H1 )
Example
There are two types of coin: Type 1 coming up Heads with probability p1; Type 2 coming up Heads with probability p2. A coin is randomly selected; the probability it is a Type 1 coin is p. (The probability it’s a Type 2 coin is 1 – p. Be careful: p is a fundamentally different thing from p1 and p2.) The coin is then tossed n times, independently, resulting in the sequence R1 R2 R3 … Rn. For example if n = 6 one possible sequence is: H T H T H H. (The number of Heads in this sequence is X = 4.) The probability of this particular sequence is pi(1 – pi)pi(1 – pi)pipi =
p4 (1− p )2 where i = 1 or 2. All other sequences with 4 Hs and 2Ts have this probability. In ii
general then, if there are x Heads (and n – x Tails) in some particular sequence, the probability of that sequence is pix (1− pi )n−x . We now have:
P(H2 ) P(A| H2 )
Posterior Odds: P(H1 | A) = P(H1 ) P(A| H1 ) = Prior Odds × Likelihood Ratio
P(H |A) P(H )P(A|H )
2 22
PriorOdds Data Likelihood ratio
P(H1)= p P(H2 ) 1− p
A=RR R wherex oftheR areHeads 12ni
P(A|H) px(1−p)n−x 1 = 1 1
P(A|H2) px(1−p)n−x 22
P(H|A) P(H)P(A|H) p px(1−p)n−x PosteriorOdds 1=1 1=11
P(H2|A) P(H2)P(A|H2) 1−ppx(1−p)n−x 22
A reasonable strategy for deciding which coin has been selected is to predict it is a Type 1 coin if the posterior odds are greater than 1 (then P(H1|A) is above 1⁄2) and if not, predict it’s a Type 2 coin. (This is somewhat arbitrary should the posterior odds be exactly 1: this generally can’t happen. We will ignore the situation – there are ways to work around it – such as simply calling it a tie and refusing to make a decision.1)
p px(1−p)n−x Prediction: Type 1 coin if and only if 1 1 >1.
1− p px (1− p )n−x 22
1 When do we see posterior odds of 1? Try p = 0.5 and, take p1 = 0.3, p2 = 0.7 and suppose the tossing results in 10 Heads in 20 tosses. (You can sort of see why neither coin is preferred here.)