STAT314. Poisson-Gamma Model. Derivation.
STAT314. Poisson-Gamma Model. Derivation.
2021S2
Let xi|µ ∼ Pois(µ) for i = 1, …, n. Then the joint likelihood (i.e., the product of individual likelihoods):
f(x|µ) = f(x1, …, xn|µ) =
∏
i
f(xi|µ) =
∏
i
µxie−µ
xi!
=
µ
∑
i
xie−nµ∏
i xi!
.
Now, one thing we definitely know about the Poisson intensity parameter µ is that it should be positive. One
of the distributions which can be used for modeling positive quantities is a Gamma distribution. (Remember
to look it up on Wikipedia if you have never ran into it before). Let’s try using it as a prior for µ:
µ|a, b ∼ Gamma(a, b),
i.e.
f(µ|a, b) =
ba
Γ(a)
µa−1e−bµ for µ > 0,
where Γ(a) is a gamma function. (Gamma function is similar to any other mathematical function, such as
sine, cosine, exponent, square root etc. in that it takes input and produces output. It is implemented in R as
gamma(). You may want to look it up, but the details of this function are not too important at the moment. )
Now, that we’ve got the likelihood and the prior, let’s apply Bayes’ Theorem:
f(µ|x, a, b) =
f(x|µ)f(µ|a, b)∫∞
0 f(x|µ)f(µ|a, b)dµ
=
µxie−µ∏
i
xi!
ba
Γ(a)µ
a−1e−bµ∫∞
0
µxie−µ∏
i
xi!
ba
Γ(a)µ
a−1e−bµdµ
=
ba
Γ(a)
1∏
i
xi!
µ
a+
∑
i
xi−1e−(b+n)µ
ba
Γ(a)
1∏
i
xi!
∫∞
0 µ
a+
∑
i
xi−1e−(b+n)µdµ
=
µ
a+
∑
i
xi−1e−(b+n)µ∫∞
0 µ
a+
∑
i
xi−1e−(b+n)µdµ
This may look like a bit of a nightmare. But note that the integral in the denominator will not depend on µ,
i.e., it will be a constant with respect to µ, and we can write:
f(µ|x, a, b) ∝ µa+
∑
i
xi−1e−(b+n)µ,
1
where ∝ denotes propotional up to a constant.
How does that help? Well, remember that any p.d.f. has to integrate to 1, and thus the constant can
always be recovered later. The formula above looks very similar to the p.d.f. of a Gamma distribution
Gamma(a+
∑
i xi, b+ n) up to a constant, which does not include µ. Thus, we can conclude that
µ|x, a, b ∼ Gamma(a+
∑
i
xi, b+ n).
An easier way. . .
is to use proportionality right away:
f(µ|x, a, b) ∝ f(x|µ)f(µ|a, b)
=
µ
∑
i
xie−nµ∏
i xi!
ba
Γ(a)
µa−1e−bµ
∝ µ
∑
i
xie−nµµa−1e−bµ
= µa+
∑
i
xi−1e−(b+n)µ
2
An easier way…