CSE 515T (Fall 2019) Assignment 1
Due Monday, 23 September 2019
1. Your doctor supposes that you may have a rare disease, Bayesian syndrome (also known as bs), which occurs in every one out of 100 members of the population. The doctor proposes you take a super-accurate (and expensive!) test to be sure. The scanner is fairly reliable; 95% of all Bayesians are identified as Bayesians, and 95% of non-Bayesians are identified as such.
The test comes back positive and the doctor tells you it’s nearly certain you’re a Bayesian given that result, due to the accuracy of the test. How do you respond?
2. Suppose k has a Poisson distribution with unknown rate parameter θ θke−θ
Pr(k | θ) = k! , k = 0, 1, 2, . . . Let the prior for θ be a gamma distribution:
βα
p(θ | α,β) = Γ(α)θα−1e−βθ, θ > 0
where Γ is the gamma function. Show that, given an observation k, the posterior p(θ | k, α, β) is a gamma distribution with updated parameters (α′, β′).
Hints:
• For nonnegative integer k, k! = Γ(k + 1).
• Γ(x+1)=xΓ(x)forallx>0.
• For common distributions (such as the gamma distribution), Wikipedia has useful properties in an infobox.
• Beware there are two common parameterizations of the gamma distribution.
3. Supposethatinthelastquestion,wereceivedasampleofnobservations{k1,k2,…,kn}. What is the posterior p(θ | k1, k2, . . . , kn, α, β)? What is the posterior mean? The posterior mode?
In light of this and the previous question, can you give an interpretation of the prior parameters α and β? What happens in the limit as n → ∞?
4. (Scenario quoted from Morey, et al.) A 10-meter-long research submersible with several people on board has lost contact with its surface support vessel. The submersible has a rescue hatch exactly halfway along its length, to which the support vessel will drop a rescue line. Because the rescuers only get one rescue attempt, it is crucial that when the line is dropped to the craft in the deep water that the line be as close as possible to this hatch. The researchers on the support vessel do not know where the submersible is, but they do know that it forms distinctive bubbles. These bubbles could form anywhere along the craft’s length, independently, with equal probability, and float to the surface where they can be seen by the support vessel.
We wish to perform inference about the location of the rescue hatch given observed bubbles; call this location θ.
1
A common “trick” when wishing to express absolute prior ignorance of a parameter is to use a so-called uninformative prior. In this case, we will consider the uninformative “prior” p(θ) = 1. This prior does not normalize, but we will see that it does not lead to major problems.
(a) Suppose the researchers observe the locations of exactly two bubbles, x1 and x2. Write down an appropriate likelihood for these data given θ and derive the posterior distribution for the location of the hatch, p(θ | x1, x2), using the uninformative prior described above.
(b) Now find a 50% Bayesian credible interval for θ given (x1, x2). Plot the width of this interval as a function of |x1 − x2|. Is this the relationship you would expect?
5. We are going to consider a Bayesian hypothesis test that a coin is exactly fair. Let θ be the “heads probability” of a coin
Pr(H) = θ.
Let us place a somewhat atypical prior on θ that includes nontrivial prior probability that the
coin is exactly fair (that is, θ = 1/2):
Pr(θ = 1/2) = 0.9 p(θ | θ ̸= 1/2) = U[0,1].
This can be interpreted as a prior that is a mixture between a uniform prior on the interval [0, 1] with weight 0.1 (this is also a special of the beta distribution with α = β = 1) and a
“point mass” (Dirac delta function) at θ = 1/2 with weight 0.9.
Suppose we flip a coin n = 10 times and observe x = 8 “heads.” What is the posterior
distribution p(θ | x, n)? What is the posterior probability that the coin is exactly fair?
Note there is no probability density function corresponding to the prior. It will help to work
in cases.
6. (Effect of weird priors.) Let us consider the following set of observations. We flip a coin independently n = 1 000 times and observe x = 900 successes. Call the unknown bias of the coin θ ∈ (0, 1).
For each of the prior distributions p(θ) below, please:
• plot the prior distribution p(θ) over the range 0 < θ < 1
• plot the posterior distribution given the above data, p(θ | D), over the range 0 < θ < 1 • reporttheposteriormean,E[θ|D]=θp(θ|D)dθ.
(a) A uniform prior on θ, which can be realized by selecting the beta distribution with α = β = 1:
p(θ) = B(θ; α = 1, β = 1). (b) A prior with extreme bias toward small values of θ:
p(θ) = B(θ;α = 1,β = 100). (c) A prior that has no support on values greater than θ = 1/2:
2 θ<1/2; 0 θ≥1/2.
2
p(θ) =
7. (OptimalPriceisRightbidding.)Supposeyouhaveastandardnormalbeliefaboutanunknown parameter θ, p(θ) = N (θ; 0, 12). You are asked to give a point estimate θˆ of θ, but are told that there is a heavy penalty for guessing too high. The loss function is
( θ − θˆ ) 2 θˆ < θ ; l(θˆ,θ;c)= c θˆ≥θ,
where c > 0 is a constant cost for overestimating. What is the Bayesian estimator in this case? How does it change as a function of c? Plot the optimal action as a function of c for 0 < c < 10. Hint: you may need to minimize certain expressions you encounter numerically as an analytic solution may not be available.
3