CS计算机代考程序代写 Review of Chapter 3: Some Special Distributions

Review of Chapter 3: Some Special Distributions
1/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Bernoulli Distribution
2/59

Bernoulli experiment and Bernoulli distribution
􏰉 Definition: Bernoulli distribution, X ∼ Bern(p):
P(X =1)=p, P(X =0)=1−p, 0≤p≤1.
􏰉 Properties:
1 Thepmfisp(x)=px(1−p)1−x forx=0,1.
2 ThemeanisEX =μ=1·p+0·(1−p)=p.
3 SinceE(X2)=12 ·p+02(1−p)=p,
σ2 =Var(X)=E(X2)−μ2 =p−p2 =p(1−p).
3/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Binomial Distribution
4/59

Definition of Binomial distribution
A binomial distribution is a common probability distribution that occurs in practice. It arises in the following situation:
(1) There are n independent trials.
(2) Each trial results in a “success” or “failure”.
(3) The probability of success in each and every trial is equal to p.
If the random variable X counts the number of successes in the n trials, then X has a binomial distribution with parameters n and p:
X ∼ Bin(n, p).
Remark The Bernoulli distribution is a special case of Binomial distribution with n = 1.
5/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

6/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Properties of Binomial distribution
If X ∼ Bin(n, p), then
1 The probability distribution of X is
􏰍n􏰎 x n−x P(X=x)= xp(1−p)
for x = 0,1,2,…,n 2 E(X)=μ=np.
3 σ2 =np(1−p).
7/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

The mgf of a binomial distribution is
M(t)= =
= M′(t) = M′′(t) =
􏰄 tx 􏰄 tx􏰍n􏰎 x n−x ep(x)= e xp(1−p)
which gives that and
μ = M′(0) = np,
σ2 =M′′(0)−μ2 =np(1−p).
xx 􏰄􏰍n􏰎 tx n−x
x (pe)(1−p)
x
[(1 − p) + pet]n, ∀t.
n[(1 − p) + pet]n−1(pet),
n[(1 − p) + pet]n−1(pet) + n(n − 1)[(1 − p) + pet]n−2(pet)2,
8/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Theorem 3.1.1
Let X1, X2, . . . , Xm be independent random variables such that Xi has a Bin(ni,p) distribution, for i = 1,2,…,m. Let
m
Y =􏰄Xi.
i=1
Then, Y ∼ Bin(􏰃mi=1 ni, p). Proof.
ThemgfofXi isMXi(t)=(1−p+pet)ni.Byindependence,we see
m
MY(t)=􏰅􏰀1−p+pet􏰁ni =􏰀1−p+pet􏰁􏰃mi=1ni .
i=1
9/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Negative Binomial Distribution
10/59

Negative Binomial Distribution
Assumption:
(1) The trials are independent of each other.
(2) Each trial results in a “success” or “failure”.
(3) The probability of success in each and every trial is equal to p.
Definition: The negative binomial random variable X is the total number of trials in the sequence to produce exactly r successes (r is a fixed positive integer):
The pmf of X ∼ NB(r, p):
􏰍x−1􏰎 x−r r
f(x)= r−1 (1−p) p , x=r,r+1,r+2,r+3,…
11/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Geometric Distribution
12/59

Geometric distribution
􏰉 The geometric distribution is a special case of a negative binomial distribution with r = 1 : X ∼ NB(1, p) = Geo(p).
􏰉 The pmf for geometric random variable is f(x)=(1−p)x−1p, x=1,2,3,…
􏰉 The cdf for geometric random variable is F(x)=1−(1−p)x, x=1,2,3,…
􏰉 Expectation, variance, and mgf:
EX = 1 and Var(X) = 1 − p. p p2
13/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

14/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Memoryless property of geometric distribution
􏰉 The geometric distribution has a memoryless property. Suppose X ∼ Geo(p). Let x, y be positive integers. Then
P (X > x + y | X > x) = P (X > y).
􏰉 We see
which gives E(X) = 1/p.
E(X) = p · 1 + (1 − p)E(X + 1),
15/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Poisson Distribution
16/59

From Binomial distribution to Poisson distribution
􏰉 Binomial pmf of Bin(n, p): P(X=x)=xp(1−p) .
􏰍n􏰎 x n−x
􏰉 Consider the limiting case when n → ∞ and p → 0 while
np = m:
lim
n→∞, p→m n
􏰍n􏰎 x n−x P (X = x) = p (1 − p)
x e−m mx
= x! ,
which is pmf of Pois(m).
17/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

If X (1)
∼ Pois(m), then
The probability distribution of X is
P(X = x) = e−mmx x!
for x = 0,1,2,…
As usual, adding up all of the probabilities yields
∞∞
(2)
􏰄
x=0 The Poisson mgf:
􏰄 e−mmx
P (X = x) =
x! = 1.
(3)
(3) (4)
M′′(t) = em(et−1)(met) + em(et−1)(met)2.
μ=E(X)=m. σ2 = m.
18/59
M(t) = em(et−1) ∀t, M′(t) = em(et−1)(met),
x=0
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

19/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Theorem 3.2.1
Suppose that X1, . . . , Xn are independent random variables such that Xi ∼ Pois(mi) for i = 1,…,n. Let Y = 􏰃ni=1 Xi. Then
Y ∼ Pois(􏰃ni=1 mi).
Proof. We have
n
MY (t) = E(etY ) = 􏰅 exp{mi(et − 1)}
i=1 􏰈n􏰡
=exp 􏰄mi(et−1) . i=1
Thus Y has a Poisson distribution with parameter 􏰃ni=1 mi due to the uniqueness of mgf.
20/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

The Gamma Distribution
21/59

Gamma Function
􏰉 Gamma function:
􏰒∞
yα−1e−ydy,
􏰉 Properties: Γ(1) =
dy = 1, 􏰒∞
Γ(α) =
􏰒∞ −y e
0
α > 0.
Γ(α) = (α − 1) Γ(n) = (n − 1)!
yα−2e−ydy = (α − 1)Γ(α − 1),
0
0
22/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Gamma Distribution
􏰉 Gamma function with y = x/β:
􏰒 ∞􏰍x􏰎α−1 􏰍1􏰎
Γ(α)= e−x/β 0ββ
dx,
α>0, β>0.
􏰉 Equivalent form:
1 =
􏰒 ∞ 1 α−1 −x/β
dx.
Γ(α)βα x e 􏰉 Gamma distribution X ∼ Gamma(α, β):
1
f(x) = Γ(α)βα xα−1e−x/β
0
􏰉 Parameters: α shape, β scale.
0 < x < ∞ elsewhere 0 23/59 Boxiang Wang Review of Chapter 3 STAT 4101 Spring 2021 The first row depicts the pdf of exponential distribution and the second row is for Chi-square distribution. 24/59 Boxiang Wang Review of Chapter 3 STAT 4101 Spring 2021 Properties of Gamma Distribution 􏰉 Moment generating function: MX(t)= 1 , t<1/β. 􏰉 Expectation: 􏰉 Variance: (1 − βt)α M′(t) = (−α)(1 − βt)−α−1(−β), μX= αβ. M′′(t) = (−α)(−α − 1)(1 − βt)−α−2(−β)2, σ X2 = α β 2 . 25/59 Boxiang Wang Review of Chapter 3 STAT 4101 Spring 2021 Suppose that a random variable X has a probability density function given by,  kx3e−x/2 x > 0 f(x) =
 0 otherwise.
􏰉 Find the value of k that makes f(x) a density function. 􏰉 Find the expectation and variance.
26/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

One Special Γ: Exponential Distribution
27/59

Exponential Distribution
􏰉 X has exponential distribution X ∼ exp(β) if X is Γ(α = 1, β).
􏰉 pdf and cdf:
1e−x/β x>0  β
f(x) =
 0 otherwise.
F(x)=1−e−x/β x>0.
􏰉 Expectation: β. 􏰉 Variance: β2.
28/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Memorylessness of Exponential Distribution
The exponential distribution is “memoryless”:
P(X > x) = e−x/β. e−(x+y)/β −y/β
Let x > 0, y > 0, then
P(X >x+y|X >x)= e−x/β =e =P(X >y).
􏰉 The only memoryless continuous probability distributions are the exponential distributions.
􏰉 The only memoryless discrete probability distributions are the geometric distributions
29/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

From Poisson Distribution to Exp (Gamma)
􏰉 Suppose the event occur in time according to a Poisson process with parameter λ, i.e., X ∼ Poisson(λ).
􏰉 Let T be the length of time until the first arrival.
􏰉 Then we have T ∼ exp(1/λ).
􏰉 If T1 is the waiting time of the 1st occurrence, T2 is the waiting time of the 2nd occurrence, … then the distribution of
then T ∼ Γ(k, β).
T = T1 + T2 + . . . + Tk .
30/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Theorem 3.3.2
If X1, . . . , Xn are independent random variables and that Xi ∼ Gamma(αi,β) for i = 1,…,n.
Let Y = 􏰃ni=1 Xi. Then Y ∼ Gamma􏰋 􏰃ni=1 αi, β􏰌. Solution Sketch:
n
MY(t)=􏰅(1−βt)−αi =(1−βt)−􏰃ni=1αi, t<1/β. i=1 31/59 Boxiang Wang Review of Chapter 3 STAT 4101 Spring 2021 Another Special Γ: χ2 Distribution 32/59 Chi-square Distribution 􏰉 A Chi-square distribution with parameter r, X ∼ χ2(r), is defined to be Gamma(α = r/2, β = 2). 􏰉 pdf f(x) =   Γ(r/2)2r/2x e x>0
 0
MX(t) = (1 − 2t)−r/2, t < 1/2. 1 r/2−1 −x/2 􏰉 mgf: 􏰉 Expectation: r. 􏰉 Variance: 2r. otherwise. 33/59 Boxiang Wang Review of Chapter 3 STAT 4101 Spring 2021 Corollary 3.3.1 If X1, . . . , Xn are independent random variables and that Xi ∼ χ2(ri) for i = 1,...,n. LetY =􏰃n X.ThenY ∼χ2􏰋􏰃n r􏰌. i=1 i i=1 i 34/59 Boxiang Wang Review of Chapter 3 STAT 4101 Spring 2021 Normal Distribution 35/59 Normal Distribution, aka Gaussian Distribution A random variable Z is said to follow a standard normal distribution if it has pdf 1 􏰂z2􏰏 f(z) = √2π exp − 2 , −∞ < z < ∞ Write Z ∼ N(0, 1). 36/59 Boxiang Wang Review of Chapter 3 STAT 4101 Spring 2021 Density Function of Standard Normal Distribution 37/59 Boxiang Wang Review of Chapter 3 STAT 4101 Spring 2021 Properties of Standard Normal Distribution Moment generating function: MZ(t) = Eexp(tZ) 􏰒∞ 1􏰍12􏰎 = exp(tz)√ exp −2z dz −∞ 2π 􏰍12􏰎􏰒∞1 􏰍1 2􏰎 =exp 2t √ exp −2(z−t) dz −∞ 2π 􏰍12􏰎􏰒∞1 􏰍12􏰎 =exp 2t √ exp −2w dw,w=z−t −∞ 2π 􏰍1 2􏰎 =exp 2t ,−∞ 0 are two parameters. We say that X has a normal distribution if it has the pdf
1 􏰂 (x−μ)2􏰏
f(x) = √2πσ2 exp − 2σ2 , −∞ < x < ∞, where −∞ < μ < ∞ and σ2 > 0 are two parameters. Write X ∼ N(μ,σ2).
Properties:
􏰉 Expectation: μ. We call μ the location parameter 􏰉 Variance σ2. We call σ the scale parameter.
􏰉 Moment generating function:
E exp(tX) = E exp(t(σZ + μ)) = exp (μt) E exp (tσZ)
􏰍122􏰎 􏰍 122􏰎 =exp(μt)exp 2σt =exp μt+2σt .
39/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Denote by Φ(z) the cdf of the standard normal distribution N (0, 1), that is to say,
􏰒z 1 􏰂t2􏰏
Φ(z) = √ exp − 2 dt. −∞ 2π
40/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

From its picture, show that
Φ(−z) = 1 − Φ(z).
Table III in Appendix C offers an abbreviated table of probabilities for a standard normal.
41/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

42/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Example
Let Z denote a normal random variable with mean 0 and standard deviation 1. Find
1 P(Z>2),
2 P(−2≤Z≤2).
43/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Theorem 3.4.1 From Normal to Chi-square
Suppose Z ∼ N(0, 1), then W = Z2 follows χ2(1).
44/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Theorem 3.4.2
Let X1, . . . , Xn be independent random variables such that Xi follows N(μi, σi2). Then, for constants a1, . . . , an,
n􏰐nn􏰑 Y =􏰄aiXi follows N 􏰄aiμi,􏰄a2iσi2 .
Proof.
i=1 i=1 i=1
􏰅n 􏰂 1222􏰏 MY(t)= exp taiμi+2taiσi
i=1
􏰈n1n􏰡
=exp t􏰄aiμi+2t2􏰄a2iσi2 . i=1 i=1
Recall that the mgf of N(μ, σ2) is exp(μt + 1 σ2t2) 2
45/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Corollary 3.4.1
Let X1, . . . , Xn be i.i.d. with common distribution N(μ, σ2). Then,
1 􏰄n 􏰍 σ 2 􏰎 X=n XifollowsNμ,n.
i=1
46/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

t-Distribution
47/59

Definition of t- Distribution
Suppose W ∼ N(0,1), V ∼ χ2(r) and that W and V are independent. The pdf of
W
T = 􏰓V/r.
is
􏰍r + 1􏰎
Γ 􏰍 􏰎−r+1
f(t)=
2 t2 2
􏰋r􏰌√ 1+ r , −∞ 0, we have
􏰙 􏰍V 􏰎−k/2􏰚 􏰙􏰍V 􏰎−k/2􏰚
E(Tk) = E Wk = E(Wk)E rr
=E(Wk)2−k/2Γ(r/2−k/2), ifk −r/2, then E(Xk) = 2kΓ(r/2 + k).
Γ(r/2) Therefore E(T ) = E(W ) = 0 if r > 1,
andVar(T)=E(T2)= r ifr>2. r−2
50/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Properties of t-Distribution
􏰉 The density function of t-distribution is symmetric, bell-shaped, and centered at 0.
􏰉 The variance of t-distribution is larger than the standard normal distribution.
􏰉 The tail of t-distribution is heavier (larger kurtosis).
51/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Student’s Theorem
Suppose X1, · · · , Xn are iid N(μ, σ2) random variables. Define the random variables,
1n 1n
X= 􏰄XiandS2= n i=1
􏰄􏰀Xi−X􏰁2 n − 1 i=1
Then
1 X∼N(μ,σ2);
n
2 X and S 2 are independent;
3 (n − 1)S2/σ2 ∼ χ2(n−1);
4 The random variable
X−μ T = S/√n
has a t-distribution with n − 1 degrees of freedom.
52/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

53/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Examples
1 Assume that T has a student t-distribution with 5 degrees of freedom. Find P (|T | > 2.571).
2 Suppose that the five random variables X1, X2, . . . , X5 are i.i.d. and each has a standard normal distribution. Determine a constant c such that the random variable
c(X1 + X2) . (X32 + X42 + X52)1/2
will have a t-distribution.
54/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

F -Distribution
55/59

Let U and V are two independent χ2 random variables with degrees of freedom r1 and r2, respectively. The pdf of
r1+r2 r1 r1/2 f(w)=Γ( 2 )(r2)
r1 −1
(w)2 ,0 −r/2, then
􏰉 We have E(F) =
r2 r1
r1
2−1Γ(r2 −1) 2
Γr2 2
=
r2 r2 −2
, being large when r2 is large. 58/59
􏰍r2􏰎k
E(Fk) = r E(Uk)E(V −k).
1
E(Xk) = 2kΓ(r/2 + k). Γ(r/2)
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021

Facts on F -Distributions
1 If X ∼ Fr1,r2, then 1/X ∼ Fr2,r1.
2 IfX ∼tn,thenX2 ∼F1,n.
3 IfX∼Fr1,r2,then
r1X 􏰋r r􏰌 r2 ∼Beta 1,2 .
1+r1X 22 r2
59/59
Boxiang Wang
Review of Chapter 3 STAT 4101 Spring 2021