Introduction to information system
Popular Distributions (1/2)
Bowei Chen
School of Computer Science
University of Lincoln
CMP3036M/CMP9063M Data Science
• Univariate Distributions
– Discrete Distributions
• Uniform
• Bernoulli
• Binomial
• Poisson
– Continuous Distributions
• Uniform
• Normal/Gaussian
• Exponential
• Multivariate Distributions
– Multivariate Normal Distribution
Objectives
Today’s Objectives
Popular Distributions (1/2)
Warm-up Questions:
1) What is the probability that the
dice is 2?
2) What is the probability that the
dice is less than 3?
3) If you will be given the money as
the number the dice is shown
(e.g., £1 if dice is 1) and you can
play infinite times of this game,
how much money you can expect
to earn per game?
1
6
2
6
£3.5
𝑋(𝜔1) = 1𝜔1 =
𝜔2 =
𝜔3 =
𝜔4 =
𝜔5 =
𝜔6 =
Ω
𝑋(𝜔2) = 2
𝑋(𝜔3) = 3
𝑋(𝜔4) = 4
𝑋(𝜔5) = 5
𝑋(𝜔6) = 6
ℝ
ℙ 𝜔1 = ℙ 𝑋 𝜔1 = 1 =
1
6
ℙ
ℙ 𝜔2 = ℙ 𝑋 𝜔2 = 2 =
1
6
ℙ 𝜔3 = ℙ 𝑋 𝜔3 = 3 =
1
6
ℙ 𝜔4 = ℙ 𝑋 𝜔4 = 4 =
1
6
ℙ 𝜔5 = ℙ 𝑋 𝜔5 = 5 =
1
6
ℙ 𝜔6 = ℙ 𝑋 𝜔6 = 6 =
1
6
Solutions:
1) What is the probability that the dice is 2?
2) What is the probability that the dice is less than 3?
3) If you will be given the money as the number the dice is (e.g., £1 if dice is
1) and you can play infinite times of this game, how much money you can
expect to earn for each game?
ℙ 𝑋(𝜔1) = 2 =
1
6
ℙ 𝑋(𝜔1) = 1 + ℙ 𝑋(𝜔2) = 2 =
2
6
£1 × ℙ 𝑋(𝜔1) = 1 + £2 × ℙ 𝑋(𝜔2) = 2 + ⋯+ £6 × ℙ 𝑋 𝜔6 = 6
= £1 +⋯+ £6 ×
1
6
= £3.5
Discrete Uniform Distribution
• Notation
𝑋~Uniform( 1,⋯ , 𝑁 )
• PMF/PDF
𝑓 𝑥 = ℙ 𝑋 = 𝑥 =
1
𝑁
, if 𝑥 = 1,⋯ ,𝑁,
0, otherwise.
• Expectation and variance
𝔼(𝑋) =
𝑁 + 1
2
,
𝕍(𝑋) =
𝑁2 − 1
12
.
Questions:
There is a flipping (fair) coin game. If head
is shown, you can earn £1; if tail is shown,
you earn nothing. How much do you think
should you be charged to enter the game?
What if head happens with probability 0.4?
£1 × 0.5 + £0 × 0.5 = £0.5
£1 × 0.4 + £0 × 0.6 = £0.4
Bernoulli Distribution
• Notation
𝑋~Bernoulli 𝑝
• PMF/PDF
𝑓 𝑥; 𝑝 = ℙ 𝑋 = 𝑥 =
𝑝𝑥(1 − 𝑝)1−𝑥, if 𝑥 = 0,1,
0, otherwise,
and 0 ≤ 𝑝 ≤ 1.
• Expectation and variance
𝔼(𝑋) = 𝑝,
𝕍(𝑋) = 𝑝(1 − 𝑝).
Questions:
An unfair coin is flipped 10 times. For each
flipping, head happens with probability 0.4.
What is the probability that exactly 6 heads
will occur?
ℙ 𝑋 = 6 =
6
10
× 0.46 × 0.64 = 0.1114767.
6 heads 4 tails
Choose 6 outcomes from
10 flipping outcomes
Binomial Distribution
• Notation
𝑋~Binomial 𝑛, 𝑝 or Bin 𝑛, 𝑝
• PMF/PDF
𝑓(𝑥; 𝑛, 𝑝) = ℙ 𝑋 = 𝑥 =
𝑛
𝑥
𝑝𝑥(1 − 𝑝)𝑛−𝑥, if 𝑥 = 0,⋯ , 𝑛,
0, otherwise,
and 0 ≤ 𝑝 ≤ 1.
• Expectation and variance
𝔼(𝑋) = 𝑛𝑝,
𝕍(𝑋) = 𝑛𝑝(1 − 𝑝).
Questions:
You usually receive 5 text messages per hour.
How likely that you will receive:
1) Exactly 2 messages in the coming hour?
2) 10 or fewer messages in 2 hours?
Poisson Distribution
The Poisson distribution is a discrete probability distribution for the counts of
events that occur randomly in a given interval of time (or space).
If we let 𝑋 be the number of events in a given interval, Then, if the mean
number of events per interval is 𝜆, the probability of observing 𝑥 events in a
given interval is given by
ℙ 𝑋 = 𝑥 =
𝑒−𝜆𝜆𝑥
𝑥!
, 𝑥 = 0,1,2,⋯ .
Solutions:
You usually receive 5 text messages per hour. How likely that you will receive:
1) Exactly 2 messages in the coming hour?
ℙ 𝑋 = 2 =
𝑒−552
2!
= 0.084
2) 10 or fewer messages in 2 hours?
ℙ 𝑋 ≤ 10 =
𝑥=0
10
𝑒−1010𝑥
𝑥!
= 0.5831
Poisson Distribution
• Notation
𝑋~Poisson(𝜆)
• PMF/PDF
𝑓 𝑥; 𝜆 = ℙ 𝑋 = 𝑥 =
𝑒−𝜆𝜆𝑥
𝑥!
, 𝑥 = 0,1,2,⋯ .
• Expectation and variance
𝔼(𝑋) = 𝜆,
𝕍(𝑋) = 𝜆.
Poisson Approximation of Binomial Probabilities
𝑋 = 1 𝑋 = 0
As you usually receive 5
text messages per hour.
That is 𝔼 𝑋 = 𝑛𝑝 = 5 = 𝜆.
We can consider 𝑛 is very
large and 𝑝 is very small.
Recall the PMF of the Binomial distribution:
𝑓 𝑥; 𝑛, 𝑝 =
𝑛
𝑥
𝑝𝑥 1 − 𝑝 𝑛−𝑥 =
𝑛 𝑛 − 1 ⋯(𝑛 − 𝑥 + 1)
𝑥!
𝜆
𝑛
𝑥
1 −
𝜆
𝑛
𝑛−𝑥
The larger the 𝑛 and the smaller the 𝑝, the better is the approximation.
Poisson Approximation of Binomial Probabilities
≈
𝜆𝑥
𝑥!
1 −
𝜆
𝑛
𝑛
=
𝜆𝑥
𝑥!
𝑒−𝜆 = 𝑓(𝑥; 𝜆)
References
• G.Casella, R.Berger (2002) Statistical Inference. Chapter 3
• K.Murphy (2012) Machine Learning: A Probabilistic Perspective. Chapter 2
Thank You!
bchen@Lincoln.ac.uk
mailto:bchen@Lincoln.ac.uk