Chapter 3: Poisson Process
Chapter 3: Poisson Process
1
The Poisson distribution
Reminder: A random variable N taking values in
Z+ = {0, 1, 2, . . . } has a Poisson distribution with a parameter
λ > 0 (and we write N ∼ Pois(λ)), if its probability mass function
is given by
P(N = n) =
e−λλn
n!
, for n ∈ Z+.
The mean and variance of N are both equal to λ.
2
The exponential distribution
Reminder: A random variable T has an exponential distribution
with parameter λ > 0 (called the rate), denoted by T ∼ exp(λ), if
its distribution function is
FT (t) =
{
1− e−λt , for t ≥ 0,
0, for t < 0.
It follows that the probability density function of T is
fT (t) =
{
λe−λt , for t ≥ 0,
0, for t < 0.
The mean of T is 1/λ and the variance of T is 1/λ2.
3
The law of rare events
The Poisson distribution arises as the limit of the binomial
distribution: Fix λ > 0, k ∈ Z+.
I If Xn ∼ Bin(n, λ/n) and N ∼ Pois(λ), then
lim
n→∞
P(Xn = k) =
e−λλk
k!
= P(N = k).
The exponential distribution arises as the limit of the geometric
distribution: Fix λ, t > 0.
I If Yn ∼ Geo(λ/n) (for n > λ) and T ∼ Exp(λ), then
lim
n→∞
P(Yn/n ≤ t) = 1− e−λt = P(T ≤ t).
4
The Poisson Process
Definition:
A nonnegative integer-valued process (Nt)t≥0 is a Poisson
process with a rate λ if
(i) it has independent increments on disjoint intervals: for
k ≥ 2 and 0 ≤ s1 < t1 ≤ s2 < · · · < tk ,
Nt1 − Ns1 , . . . ,Ntk − Nsk
are independent variables, and
(ii) For each t > s ≥ 0, Nt − Ns ∼ Pois(λ(t − s)).
5
Exercise
If (Nt)t≥0 is a Poisson process with rate λ, show that for fixed
r > 0, the process (N∗t )t≥0 defined by N
∗
t = Nt+r − Nr , is also
Poisson process with rate λ.
[This is true even if r is a stopping time for the process.]
6
A trajectory
0
1
2
3
4
5
T1 T2 T3 T4
τ3τ1
7
Poisson Process Empirical Data
I Earthquakes
I Grazing animals head raises
I Horse kick deaths
8
The Poisson Process
Let T0 = 0 and Tj = min{t : Nt = j} (the time of j th jump) and
define τj = Tj − Tj−1 (time between the (j − 1)st and j th jumps).
Theorem: (Nt)t≥0 is a Poisson process with rate λ if and only if
(τj)j∈N are independent Exp(λ) random variables.
Proof: The key to the proof is to observe that the event {Tj ≤ t}
is the same as {Nt ≥ j}. That is the waiting time until the jth
jump is less than or equal to t if and only if there are j or more
jumps up to (and including) time t.
Assume that (Nt)t≥0 is a Poisson process. Then
P(T1 ≤ t) = P(Nt ≥ 1) = 1− P(Nt = 0) = 1− e−λt , so
T1 ∼Exp(λ).
9
The Poisson Process
Furthermore, we have
P(Tj ≤ t) = P(Nt ≥ j)
=
∞∑
k=j
e−λt
(λt)k
k!
= 1−
j−1∑
k=0
e−λt
(λt)k
k!
.
This final expression is the distribution function for gamma
distribution with parameter k and rate λ. You can check this by
differentiating to get the density function
fTj (t) = e
−λtλj t j−1/(j − 1)!.
So the waiting time until the jth event is the sum of j independent
exponentially-distributed inter-event times with parameter λ.
10
The Poisson Process
This argument also holds in reverse.
Assuming that τ1 ∼Exp(λ), we know that P(T1 ≤ t) = 1− e−λt ,
which tells us that P(Nt = 0) = e−λt .
Furthermore, for j > 1, if {τ1, . . . , τj} are i.i.d. Exp(λ), then Tj has
a Gamma distribution with parameters λ and j . So
P(Nt ≥ j) = P(Tj ≤ t) = 1−
j−1∑
k=0
e−λt
(λt)k
k!
,
which tells us that Nt has a Poisson distribution with parameter λt.
11
The Poisson Process
We’d still need to show that Nti − Nsi are independent
∼Pois(λ(ti − si )) over sets [si , ti ) of disjoint intervals.
This follows from the memoryless property of the exponential
distribution (so the remaining time from si until the next jump
doesn’t depend on the time si − TNsi since the previous jump) and
the independence of the τj .
12
Order statistics
For random variables ξ1, ξ2, . . . , ξk , denote by ξ(i) the i
th smallest
of them. Then ξ(1), ξ(2), . . . , ξ(k) are called the order statistics
associated with ξ1, ξ2, . . . , ξk .
For example, if we sample these random variables and find that
ξ1 = 1.3, ξ2 = 0.9, ξ3 = 0.7, ξ4 = 1.1 and ξ5 = 1.5, then
ξ(1) = 0.7, ξ(2) = 0.9, . . . , ξ(5) = 1.5.
Order statistics play a very important role in applications. For
example, the maximum likelihood estimator of θ for a sample
ξ1, ξ2, . . . , ξk from the uniform [0, θ] distribution is ξ(k).
13
Order Statistics: examples
I If X1,X2 and X3 are independent and identically-distributed
random variables taking values 1, 2 and 3 each with
probability 1/3, find the joint probability mass function of
(X(1),X(2),X(3)).
I If Y1,Y2, . . . ,Yk are i.i.d. random variables with distribution
function F , the distribution function of Y(i) is
FY(i)(x) =
k∑
`=i
(
k
`
)
F (x)`(1− F (x))k−`.
I So, for the special case where Y1,Y2, . . . ,Yk are
i.i.d.∼ U[0, t], for i ≤ k and x ≤ t, the distribution function
of the order statistic Y(i) is given by
FY(i)(x) =
k∑
`=i
(
k
`
)
(x/t)`(1− x/t)k−`.
14
Order Statistics
I Above we saw that if Y1,Y2, . . . ,Yk are i.i.d. with distribution
function F the distribution function of Y(i) is
FY(i)(x) =
k∑
`=i
(
k
`
)
F (x)`(1− F (x))k−`.
If they are also absolutely continuous, with density f , then the
density of the order statistic Y(i) is
fY(i)(x) =
(
k
i − 1
)
(k − i + 1)F (x)i−1f (x)(1− F (x))k−i
=
(
k
i
)
iF (x)i−1f (x)(1− F (x))k−i .
15
Order Statistics
Similarly the joint densities for 1 ≤ r ≤ k and x1 < · · · < xr are fY(i1),...,Y(ir ) (x1, . . . , xr ) = ( k i1 − 1, 1, i2 − i1 − 1, 1 · · · , 1, k − ir ) × r∏ j=1 f (xj) r∏ j=0 (F (xj+1)− F (xj))ij+1−ij−1, where ( ` a1,··· ,aj ) is the number of ways to choose subsets of sizes a1, . . . , aj from a set of size ` and for the sake of brevity we set x0 = −∞ and xr+1 =∞ so F (x0) = 0 and F (xr+1) = 1. 16 Order Statistics In particular for r = k , x1 < · · · < xr , fY(1),...,Y(k)(x1, . . . , xk) = k! k∏ j=1 f (xj). 17 The Poisson Process Theorem: The conditional distribution of (T1, · · · ,Tk) given that Nt = k is the same as the distribution of order statistics of a sample of k independent and identically-distributed random variables uniformly distributed on [0, t]. I.e. (T1, · · · ,Tk)|{Nt = k} d = (U(1), · · · ,U(k)) where U1, · · · ,Uk are independent ∼ U(0, t). The same representation holds for the conditional distribution of (T1, · · · ,Tk) given that Tk+1 = t. 18 “Proof” of theorem According to our derivation for order statistics, (U(1), · · · ,U(k)) has density k!t−k for 0 = x0 < x1 < · · · < xk < t. So we show the LHS has the same density: P(T1 ∈ dx1, · · · ,Tk ∈ dxk |Nt = k) = P(τ1 ∈ dx1, τ2 ∈ d(x2 − x1), . . . , τk ∈ d(xk − xk−1), τk+1 > t − xk)
P(Nt = k)
=
(
∏k
i=1
λe−λ(xi−xi−1))e−λ(t−xk )
(λt)ke−λt/k!
dx1 . . . dxk
= k!t
−k
dx1 . . . dxk .
The proof of the second claim is similar.
19
The Poisson Process
The theorem implies that if τ1, . . . , τn are i.i.d. exponential
variables, then (
τ1∑n+1
j=1 τj
,
τ1 + τ2∑n+1
j=1 τj
, . . . ,
∑n
j=1 τj∑n+1
j=1 τj
)
have the same distribution as U(0, 1) order statistics.
20
Superposition of Poisson processes
Theorem: Let (Nt)t≥0 and (Mt)t≥0 be two independent Poisson
processes with rates λ1 and λ2 respectively and Lt = Nt + Mt .
Then (Lt)t≥0 is a Poisson process with rate λ1 + λ2.
Proof:
I By independence, Lt − Ls ∼ Pois(λ1(t − s) + λ2(t − s)).
I For disjoint [s1, t1] and [s2, t2],
Lt1 − Ls1 = (Nt1 − Ns1) + (Mt1 −Ms1)
Lt2 − Ls2 = (Nt2 − Ns2) + (Mt2 −Ms2)
which are independent because of the same property of
(Nt)t≥0 and (Mt)t≥0. This argument extends to all finite
collections of disjoint intervals.
21
Superposition example
A shop has two entrances, one from East St, the other from West
St. Flows of customers through the two entrances are independent
Poisson processes with rates 0.5 and 1.5 per minute, respectively.
I What is the probability that no new customers enter the shop
in a fixed three minute time interval?
I What is the mean time between arrivals of new customers?
I What is the probability that a given customer entered from
West St?
22
Thinning of a Poisson process
Suppose in a Poisson process (Nt)t≥0 each ‘customer’ is ‘marked’
independently with probability p. Let Mt count the number of
‘marked customers’ that arrive on [0, t].
Theorem. The processes (Mt)t≥0 and (Nt −Mt)t≥0 are
independent Poisson processes with rates λp and λ(1− p)
respectively.
23
Thinning – “proof”
“Proof”.
P(Mt = j ,Nt −Mt = k) = P(Mt = j ,Nt = k + j)
= P(Mt = j |Nt = k + j)P(Nt = k + j)
=
(
k + j
j
)
pj(1− p)k
e−λt(λt)k+j
(k + j)!
=
e−pλt(pλt)j
j!
e−(1−p)λt((1− p)λt)k
k!
.
24
Thinning – example
The flow of customers to a shop is a Poisson process with rate 25
customers per hour. Each of the customers independently has a
probability p = 0.8 of making a purchase.
I What is the probability that all customers who enter the shop
during the time interval from 11.00 am to 11.15 am make a
purchase?
I What is the probability that, conditional on there being two
customers that made a purchase during that period, all
customers who enter the shop during the time interval make a
purchase?
25
The Compound Poisson Process
Suppose that (Nt)t≥0 is a Poisson process and (Xi )i∈N are
independent and identically-distributed random variables, which are
also independent of (Nt)t≥0.
For t ≥ 0, define Yt =
∑
j≤Nt Xj . Then (Yt)t≥0 is called a
compound Poisson process.
It can be shown that (Yt)t≥0 has independent increments and it is
possible to compute the distribution of Yt by conditioning on Nt .
26
Compound Poisson example
Suppose claims made to an insurance company arrive according to
a Poisson process with rate λ, and each policy holder carries a
policy for an amount Xk . Assume X1,X2, . . . are independent and
identically-distributed, and the number of claims and the size of
claims are independent.
Calculate the mean and variance of the total amount of claims on
the company up to time t.
27