Statistics 100B
Definition:
MX (t) = EetX Therefore,
If X is discrete
MX (t) = etX p(x)
x
If X is continuous
MX(t) = etXf(x)dx
x
Aside:
x xx2x3
e =1+1!+2!+3!+···
University of California, Los Angeles Department of Statistics
Instructor: Nicolas Christou Moment generating functions
Similarly,
tx tx (tx)2 (tx)3
e =1+1!+ 2! + 3! +··· Let X be a discrete random variable.
tx (tx)2 MX(t)=etXp(x)= 1+ 1! + 2! + 3! +··· p(x)
xx
t t2 t3
x 1! x 2! x 3! x
(tx)3
or
MX(t)=p(x)+ xp(x)+ x2p(x)+ x3P(x)+···
To find the kth moment simply evaluate the kth derivative of the MX (t) at t = 0. EXk = [MX(t)]kthderivative
t=0
For example: First moment:
MX(t)′ =xp(x)+2tx2p(x)+··· x 2! x
We see that MX(0)′ = x xp(x) = E(X).
1
Similarly, Second moment
MX(t)′′ =x2p(x)+6tx3p(x)+··· x 3! x
We see that MX(0)′′ = x x2p(x) = E(X2).
Examples:
1. Find the moment generating function of X ∼ b(n, p).
MX (t) = MX(t) =
MX(t) =
MX (t) = Using the binomial theorem (a + b)n = MX(t) =
E etx
etxp(x) x
n n
(pet +1−p)n 2. Find the moment generating function of X ∼ P oisson(λ).
E etx
etxp(x) x
MX (t) = MX(t) =
MX(t) = MX(t) =
−x
distribution with parameters α,β if its pdf is given by f(x) = xα−1e β , x > 0,α >
0, β > 0, where Γ(α) is the gamma function defined as Γ(α) = ∞ xα−1e−xdx. 0
∞
etx x=0
λxe−λ x!
etx
x=0 x n n
px(1−p)n−x
(pet)x(1 − p)n−x x=0 x
n n
axbn−x we get x=0 x
∞ e−λ
x=0 e−λ eλet
(λet )x x!
MX (t) = MX (t) =
eλ(et −1)
3. Find the moment generating function of X ∼ Γ(α, β). We say that X follows a gamma
MX (t) = Eetx
MX(t) = etxf(x) x
2
Γ(α)βα
. MX (t) =
α−1 −x ∞ txx eβ
MX(t) = MX(t) =
aspecialcaseofΓ(α,β)withα=1andβ= 1 (why?),therefore,MX(t)=(1− t)−1. λλ
5. Find the moment generating function of Z ∼ N (0, 1). Aside note: The normal distri- bution X ∼ N(μ,σ) has pdf
1 − 1 (x−μ)2 f(x)= √ e 2σ2 ,
σ 2π
and the standard normal Z ∼ N (0, 1) has pdf
x=0
e α dx Γ(α)β
α−1 −x(1−t)
∞xeβ 1
− t) to get
x=0 Γ(α)β
α dx
Use the transformation y = x(
β
(1 − βt)−α
4. Find the moment generating function of X ∼ exp(λ). The exponential distribution is
1 −1z2 f(z)=√ e2
2π Therefore,
MZ(t)= Add/subtract to the exponent 1t2
MZ(t) = MZ(t) =
Eetz
etzf(z)
z
+∞ tz 1 −1z2 e√e2dz
−∞ 2π
2 MZ(t) =
1 −1(z−t)2
√ e 2 dz =
1t2 +∞ e2
1 −1(z−t)2
√ e 2 dz
2π
+∞ Explain why
−∞
2π
1
−∞
Therefore,
Z2 M (t) = e1t2
3
Theorem:
Let X, Y be independent random variables with moment generating functions MX(t),MY (t) respectively. Then, the moment generating function of the sum of these two random variables is equal to the product of the individual moment generating functions:
MX+Y (t) = MX(t)MY (t) Proof:
Examples: Let X, Y independent random variables. Use this theorem to find the distribution of X + Y :
a. X ∼ b(n1,p),Y ∼ b(n2,p).
b. X ∼ Poisson(λ1),Y ∼ Poisson(λ2).
4
Properties of moment generating functions:
Let X be a random variable with moment generating function MX (t) = EetX , and a, b are constants
1. MX+a(t) = eatMX (t)
2. MbX(t)=MX(bt)
b
Use these properties and the moment generating function of Z ∼ N (0, 1) to find the moment generating function of X ∼ N (μ, σ)
FindthedistributionofX+Y,whereX∼N(μ1,σ1),Y ∼N(μ2,σ2).
3. MX+a =eatM (t)
bXb
5
Let X1, X2, . . . , Xn be i.i.d. random variables from N(μ, σ). Use moment generating func- tions to find the distribution of
a. T =X1 +X2 +…+Xn.
b. X ̄ = ni=1 Xi . n
6
Distribution of the sample mean – Sampling from normal distribution
If we sample from normal distribution N(μ,σ) then X ̄ follows exactly the normal distribution with
mean μ and standard deviation σ regardless of the sample size n. In the next figure we see the n
effect of the sample size on the shape of the distribution of X ̄. The first figure is the N(5,2) distri- bution. The second figure represents the distribution of X ̄ when n = 4. The third figure represents the distribution of X ̄ when n = 16.
N(5, 2)
−3 −1 1 3 5 7 9 11 13
x
2
4
123456789
x
2
16
34567
x
7
√
N(5,
)
N(5,
)
f(x) f(x) f(x)
0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8