Review of Chapter 3: Some Special Distributions
1/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Bernoulli Distribution
2/59
Bernoulli experiment and Bernoulli distribution
Definition: Bernoulli distribution, X ∼ Bern(p):
P(X =1)=p, P(X =0)=1−p, 0≤p≤1.
Properties:
1 Thepmfisp(x)=px(1−p)1−x forx=0,1.
2 ThemeanisEX =μ=1·p+0·(1−p)=p.
3 SinceE(X2)=12 ·p+02(1−p)=p,
σ2 =Var(X)=E(X2)−μ2 =p−p2 =p(1−p).
3/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Binomial Distribution
4/59
Definition of Binomial distribution
A binomial distribution is a common probability distribution that occurs in practice. It arises in the following situation:
(1) There are n independent trials.
(2) Each trial results in a “success” or “failure”.
(3) The probability of success in each and every trial is equal to p.
If the random variable X counts the number of successes in the n trials, then X has a binomial distribution with parameters n and p:
X ∼ Bin(n, p).
Remark The Bernoulli distribution is a special case of Binomial distribution with n = 1.
5/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
6/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Properties of Binomial distribution
If X ∼ Bin(n, p), then
1 The probability distribution of X is
n x n−x P(X=x)= xp(1−p)
for x = 0,1,2,…,n 2 E(X)=μ=np.
3 σ2 =np(1−p).
7/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
The mgf of a binomial distribution is
M(t)= =
= M′(t) = M′′(t) =
tx txn x n−x ep(x)= e xp(1−p)
which gives that and
μ = M′(0) = np,
σ2 =M′′(0)−μ2 =np(1−p).
xx n tx n−x
x (pe)(1−p)
x
[(1 − p) + pet]n, ∀t.
n[(1 − p) + pet]n−1(pet),
n[(1 − p) + pet]n−1(pet) + n(n − 1)[(1 − p) + pet]n−2(pet)2,
8/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Theorem 3.1.1
Let X1, X2, . . . , Xm be independent random variables such that Xi has a Bin(ni,p) distribution, for i = 1,2,…,m. Let
m
Y =Xi.
i=1
Then, Y ∼ Bin(mi=1 ni, p). Proof.
ThemgfofXi isMXi(t)=(1−p+pet)ni.Byindependence,we see
m
MY(t)=1−p+petni =1−p+petmi=1ni .
i=1
9/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Negative Binomial Distribution
10/59
Negative Binomial Distribution
Assumption:
(1) The trials are independent of each other.
(2) Each trial results in a “success” or “failure”.
(3) The probability of success in each and every trial is equal to p.
Definition: The negative binomial random variable X is the total number of trials in the sequence to produce exactly r successes (r is a fixed positive integer):
The pmf of X ∼ NB(r, p):
x−1 x−r r
f(x)= r−1 (1−p) p , x=r,r+1,r+2,r+3,…
11/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Geometric Distribution
12/59
Geometric distribution
The geometric distribution is a special case of a negative binomial distribution with r = 1 : X ∼ NB(1, p) = Geo(p).
The pmf for geometric random variable is f(x)=(1−p)x−1p, x=1,2,3,…
The cdf for geometric random variable is F(x)=1−(1−p)x, x=1,2,3,…
Expectation, variance, and mgf:
EX = 1 and Var(X) = 1 − p. p p2
13/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
14/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Memoryless property of geometric distribution
The geometric distribution has a memoryless property. Suppose X ∼ Geo(p). Let x, y be positive integers. Then
P (X > x + y | X > x) = P (X > y).
We see
which gives E(X) = 1/p.
E(X) = p · 1 + (1 − p)E(X + 1),
15/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Poisson Distribution
16/59
From Binomial distribution to Poisson distribution
Binomial pmf of Bin(n, p): P(X=x)=xp(1−p) .
n x n−x
Consider the limiting case when n → ∞ and p → 0 while
np = m:
lim
n→∞, p→m n
n x n−x P (X = x) = p (1 − p)
x e−m mx
= x! ,
which is pmf of Pois(m).
17/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
If X (1)
∼ Pois(m), then
The probability distribution of X is
P(X = x) = e−mmx x!
for x = 0,1,2,…
As usual, adding up all of the probabilities yields
∞∞
(2)
x=0 The Poisson mgf:
e−mmx
P (X = x) =
x! = 1.
(3)
(3) (4)
M′′(t) = em(et−1)(met) + em(et−1)(met)2.
μ=E(X)=m. σ2 = m.
18/59
M(t) = em(et−1) ∀t, M′(t) = em(et−1)(met),
x=0
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
19/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Theorem 3.2.1
Suppose that X1, . . . , Xn are independent random variables such that Xi ∼ Pois(mi) for i = 1,…,n. Let Y = ni=1 Xi. Then
Y ∼ Pois(ni=1 mi).
Proof. We have
n
MY (t) = E(etY ) = exp{mi(et − 1)}
i=1 n
=exp mi(et−1) . i=1
Thus Y has a Poisson distribution with parameter ni=1 mi due to the uniqueness of mgf.
20/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
The Gamma Distribution
21/59
Gamma Function
Gamma function:
∞
yα−1e−ydy,
Properties: Γ(1) =
dy = 1, ∞
Γ(α) =
∞ −y e
0
α > 0.
Γ(α) = (α − 1) Γ(n) = (n − 1)!
yα−2e−ydy = (α − 1)Γ(α − 1),
0
0
22/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Gamma Distribution
Gamma function with y = x/β:
∞xα−1 1
Γ(α)= e−x/β 0ββ
dx,
α>0, β>0.
Equivalent form:
1 =
∞ 1 α−1 −x/β
dx.
Γ(α)βα x e Gamma distribution X ∼ Gamma(α, β):
1
f(x) = Γ(α)βα xα−1e−x/β
0
Parameters: α shape, β scale.
0 < x < ∞ elsewhere
0
23/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
The first row depicts the pdf of exponential distribution and the second row is for Chi-square distribution.
24/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Properties of Gamma Distribution
Moment generating function:
MX(t)= 1 , t<1/β.
Expectation:
Variance:
(1 − βt)α
M′(t) = (−α)(1 − βt)−α−1(−β), μX= αβ.
M′′(t) = (−α)(−α − 1)(1 − βt)−α−2(−β)2, σ X2 = α β 2 .
25/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Suppose that a random variable X has a probability density function given by,
kx3e−x/2 x > 0 f(x) =
0 otherwise.
Find the value of k that makes f(x) a density function. Find the expectation and variance.
26/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
One Special Γ: Exponential Distribution
27/59
Exponential Distribution
X has exponential distribution X ∼ exp(β) if X is Γ(α = 1, β).
pdf and cdf:
1e−x/β x>0 β
f(x) =
0 otherwise.
F(x)=1−e−x/β x>0.
Expectation: β. Variance: β2.
28/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Memorylessness of Exponential Distribution
The exponential distribution is “memoryless”:
P(X > x) = e−x/β. e−(x+y)/β −y/β
Let x > 0, y > 0, then
P(X >x+y|X >x)= e−x/β =e =P(X >y).
The only memoryless continuous probability distributions are the exponential distributions.
The only memoryless discrete probability distributions are the geometric distributions
29/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
From Poisson Distribution to Exp (Gamma)
Suppose the event occur in time according to a Poisson process with parameter λ, i.e., X ∼ Poisson(λ).
Let T be the length of time until the first arrival.
Then we have T ∼ exp(1/λ).
If T1 is the waiting time of the 1st occurrence, T2 is the waiting time of the 2nd occurrence, … then the distribution of
then T ∼ Γ(k, β).
T = T1 + T2 + . . . + Tk .
30/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Theorem 3.3.2
If X1, . . . , Xn are independent random variables and that Xi ∼ Gamma(αi,β) for i = 1,…,n.
Let Y = ni=1 Xi. Then Y ∼ Gamma ni=1 αi, β. Solution Sketch:
n
MY(t)=(1−βt)−αi =(1−βt)−ni=1αi, t<1/β.
i=1
31/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Another Special Γ: χ2 Distribution
32/59
Chi-square Distribution
A Chi-square distribution with parameter r, X ∼ χ2(r), is defined to be Gamma(α = r/2, β = 2).
pdf
f(x) =
Γ(r/2)2r/2x e x>0
0
MX(t) = (1 − 2t)−r/2, t < 1/2.
1 r/2−1 −x/2
mgf:
Expectation: r.
Variance: 2r.
otherwise.
33/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Corollary 3.3.1
If X1, . . . , Xn are independent random variables and that Xi ∼ χ2(ri) for i = 1,...,n.
LetY =n X.ThenY ∼χ2n r. i=1 i i=1 i
34/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Normal Distribution
35/59
Normal Distribution, aka Gaussian Distribution
A random variable Z is said to follow a standard normal distribution if it has pdf
1 z2
f(z) = √2π exp − 2 , −∞ < z < ∞
Write Z ∼ N(0, 1).
36/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Density Function of Standard Normal Distribution
37/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Properties of Standard Normal Distribution
Moment generating function:
MZ(t) = Eexp(tZ)
∞ 112
= exp(tz)√ exp −2z dz −∞ 2π
12∞1 1 2 =exp 2t √ exp −2(z−t) dz
−∞ 2π 12∞1 12
=exp 2t √ exp −2w dw,w=z−t −∞ 2π
1 2
=exp 2t ,−∞
1 (x−μ)2
f(x) = √2πσ2 exp − 2σ2 , −∞ < x < ∞,
where −∞ < μ < ∞ and σ2 > 0 are two parameters. Write X ∼ N(μ,σ2).
Properties:
Expectation: μ. We call μ the location parameter Variance σ2. We call σ the scale parameter.
Moment generating function:
E exp(tX) = E exp(t(σZ + μ)) = exp (μt) E exp (tσZ)
122 122 =exp(μt)exp 2σt =exp μt+2σt .
39/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Denote by Φ(z) the cdf of the standard normal distribution N (0, 1), that is to say,
z 1 t2
Φ(z) = √ exp − 2 dt. −∞ 2π
40/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
From its picture, show that
Φ(−z) = 1 − Φ(z).
Table III in Appendix C offers an abbreviated table of probabilities for a standard normal.
41/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
42/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Example
Let Z denote a normal random variable with mean 0 and standard deviation 1. Find
1 P(Z>2),
2 P(−2≤Z≤2).
43/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Theorem 3.4.1 From Normal to Chi-square
Suppose Z ∼ N(0, 1), then W = Z2 follows χ2(1).
44/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Theorem 3.4.2
Let X1, . . . , Xn be independent random variables such that Xi follows N(μi, σi2). Then, for constants a1, . . . , an,
nnn Y =aiXi follows N aiμi,a2iσi2 .
Proof.
i=1 i=1 i=1
n 1222 MY(t)= exp taiμi+2taiσi
i=1
n1n
=exp taiμi+2t2a2iσi2 . i=1 i=1
Recall that the mgf of N(μ, σ2) is exp(μt + 1 σ2t2) 2
45/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Corollary 3.4.1
Let X1, . . . , Xn be i.i.d. with common distribution N(μ, σ2). Then,
1 n σ 2 X=n XifollowsNμ,n.
i=1
46/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
t-Distribution
47/59
Definition of t- Distribution
Suppose W ∼ N(0,1), V ∼ χ2(r) and that W and V are independent. The pdf of
W
T = V/r.
is
r + 1
Γ −r+1
f(t)=
2 t2 2
r√ 1+ r , −∞
V −k/2 V −k/2
E(Tk) = E Wk = E(Wk)E rr
=E(Wk)2−k/2Γ(r/2−k/2), ifk
Γ(r/2) Therefore E(T ) = E(W ) = 0 if r > 1,
andVar(T)=E(T2)= r ifr>2. r−2
50/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Properties of t-Distribution
The density function of t-distribution is symmetric, bell-shaped, and centered at 0.
The variance of t-distribution is larger than the standard normal distribution.
The tail of t-distribution is heavier (larger kurtosis).
51/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Student’s Theorem
Suppose X1, · · · , Xn are iid N(μ, σ2) random variables. Define the random variables,
1n 1n
X= XiandS2= n i=1
Xi−X2 n − 1 i=1
Then
1 X∼N(μ,σ2);
n
2 X and S 2 are independent;
3 (n − 1)S2/σ2 ∼ χ2(n−1);
4 The random variable
X−μ T = S/√n
has a t-distribution with n − 1 degrees of freedom.
52/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
53/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Examples
1 Assume that T has a student t-distribution with 5 degrees of freedom. Find P (|T | > 2.571).
2 Suppose that the five random variables X1, X2, . . . , X5 are i.i.d. and each has a standard normal distribution. Determine a constant c such that the random variable
c(X1 + X2) . (X32 + X42 + X52)1/2
will have a t-distribution.
54/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
F -Distribution
55/59
Let U and V are two independent χ2 random variables with degrees of freedom r1 and r2, respectively. The pdf of
r1+r2 r1 r1/2 f(w)=Γ( 2 )(r2)
r1 −1
(w)2 ,0
We have E(F) =
r2 r1
r1
2−1Γ(r2 −1) 2
Γr2 2
=
r2 r2 −2
, being large when r2 is large. 58/59
r2k
E(Fk) = r E(Uk)E(V −k).
1
E(Xk) = 2kΓ(r/2 + k). Γ(r/2)
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021
Facts on F -Distributions
1 If X ∼ Fr1,r2, then 1/X ∼ Fr2,r1.
2 IfX ∼tn,thenX2 ∼F1,n.
3 IfX∼Fr1,r2,then
r1X r r r2 ∼Beta 1,2 .
1+r1X 22 r2
59/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021