CS代考 ARMA Models Properties of AR(p)

ARMA Models Properties of AR(p)
7.4 Properties of AR(p)
We will look at some properties of AR(p) processes
Xt =φ0 +φ1Xt−1 +φ2Xt−2 +···+φpXt−p +Wt.
In particular, Expectation
Autocovariance .
52/74

ARMA Models Properties of AR(p)
Expectation of AR(p)
We will sometimes work with the mean zero AR(p) by assuming φ0 = 0
Xt =φ1Xt−1 +φ2Xt−2 +···+φpXt−p +Wt. (1)
In the general case with φ0 not necessarily 0
Xt =φ0 +φ1Xt−1 +φ2Xt−2 +···+φpXt−p +Wt.
If 0 ̸= 1 − φ1 − · · · − φp , we set μX = φ0
1−φ1 −···−φp
and shift the process by Xt − μX . Re-arranging yields
Xt−μX = φ1(Xt−1−μX)+φ2(Xt−2−μX)+···+φp(Xt−p−μX)+Wt. Hence, E(Xt) = μx.
53/74

ARMA Models Properties of AR(p)
Backshift autoregression formulation
It is common to state the mean zero autoregression in (1) using the backshift or lag operator B Xt = Xt−1
Xt −φ1Xt−1 −φ2Xt−2 −···−φpXt−p =Wt (1−φ1B−φ2B2−···−φpBp)Xt =Wt
A particularly concise formulation is then
φ(B)Xt = Wt (2)
where φ(B) is called the autoregressive operator
φ(B) = (1 − φ1B − φ2B2 − · · · − φpBp). (3)
The AR(p) can then be seen as the solution: Xt= 1 Wt
φ(B )
Let us try to understand what 1 is the case of AR(1). φ(B )
54/74

ARMA Models Properties of AR(p)
Inverting AR(1)
ConsiderXt =φ1Xt−1+Wt.
We eliminate Xt−i from the l.h.s. to achieve Xt =
By recursive substitution we obtain:
1 Wt. φ(B )
Xt =
φ1Xt−1+Wt
= φ1(φ1Xt−2 + Wt−1) + Wt
= φ21Xt−2 + φ1Wt−1 + Wt
= .
= φkX +(φk−1W +···+φW +W)
1 t−k 1 t−k−1 1 t−1 t k−1
= φk1Xt−k +􏰋φj1Wt−j j=0
which can be written as:
k−1
φ k1 X t − k = X t − 􏰋 φ j1 W t − j
j=0
55/74

ARMA Models Properties of AR(p)
Inverting AR(1)
Let’s look at the limiting mean square difference:
k−1 2
lim E X −􏰋φjW = lim φ2kE􏰀X2 􏰁=0
k→∞  t 1 t−j k→∞ 1 t−k j=0
provided |φ1| < 1 and σW2 < ∞. Hence, the limiting form of the zero mean AR(1) model with |φ1|<1andσW2 <∞is ∞1 Xt =􏰋φj1Wt−j=φ(B)Wt. j=0 So with B Wt = Wt−1 1∞ φ ( B ) = 􏰋 φ j1 B j j=0 This is the Would decomposition of AR(1). 56/74 ARMA Models Properties of AR(p) Now it’s easy to calculate the properties of the AR(1) Expectation without φ0 μX = E(Xt) = 􏰋φj1E(Wt−j) = 0 j=0 Expectation with φ0 using the formula for μX above E(Xt)=μX= φ0 1−φ1 ∞ IngeneralXt−μX =􏰋φj1Wt−j j=0 SinceXt −μX isAR(1)withφX =0. ∞ 57/74 ARMA Models Properties of AR(p) Now it’s easy to calculate the properties of the AR(1) Variance for σX2 = = 􏰋E[(φj1Wt−j)2] = Var(Xt)=E[(Xt −μX)2] ∞ j=0 ∞ = 􏰋φ2jE[W2 ] 1 t−j j=0 ∞ = σ2 􏰋φ2j W1 j=0 σW2 1 − φ 21 58/74 ARMA Models Properties of AR(p) Now it’s easy to calculate the properties of the AR(1) Autocovariance γX (h) = Cov[(Xt −μX),(Xt+|h| −μX)] ∞∞ 1 = Cov 􏰋φj Wt−j,􏰋φk1Wt+|h|−k j=0 k=0 ∞∞ = 􏰋􏰋Cov􏰄φjW ,φkW 1 t−j 1 j=0 k=0 ∞ 􏰅 t+|h|−k 􏰅 = 􏰋Cov 􏰄φj W j=0 􏰋∞ σ 2 φ | h | = φj φ|h|+jσ2 = W 1 11 W 1−φ21 j=0 = σ2 φ|h| X1 ,φ|h|+jW 1t−j1 t−j 59/74 ARMA Models Properties of AR(p) Now it’s easy to calculate the properties of the AR(1) Autocorrelation ρX(h) = γX(h) = φ|h| σX2 1 Hence, if |φ1| < 1 and σW2 < ∞, AR(1) is stationary. 60/74 ARMA Models Properties of AR(p) Example: AR(1) autocovariance and autocorrelation AR(1): Xt = 0.5Xt−1 + Wt time series 0 100 Estimated autocovariance 200 300 400 500 Estimated autocorrelation Time, t ● ● ● ● ● ●● ●●●●●●●●●●●●●●●●●●●●● ● ● ● ● sample acf ● true acf ● ●● ●●●●●●●●●●●●●●●●●●●●● 0 5 10 15 20 25 0 5 10 15 20 25 lag, h lag, h ● ● 61/74 0.0 0.2 0.4 0.6 0.8 1.0 1.2 0.0 0.2 0.4 0.6 0.8 1.0 Autocovariance, gamma(h) Autocorrelation, rho(h) Xt −2 0 2 4 ARMA Models Properties of AR(p) Example: AR(1) autocovariance and autocorrelation AR(1): Xt = 0.9Xt−1 + Wt time series 0 100 Estimated autocovariance 200 300 400 500 Estimated autocorrelation Time, t ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● sample acf ● true acf ●● ●●● 0 5 10 15 20 25 0 5 10 15 20 25 lag, h lag, h ● ● 62/74 Autocovariance, gamma(h) 123456 Autocorrelation, rho(h) 0.0 0.2 0.4 0.6 0.8 1.0 Xt −4 −2 0 2 4 6 8 ARMA Models Properties of AR(p) Example: AR(1) autocovariance and autocorrelation AR(1): Xt = −0.9Xt−1 + Wt time series 0 100 200 Estimated autocovariance Time, t 300 400 500 Estimated autocorrelation ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ●●●●●● ● ● sample acf ● true acf ●● ● ●●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● 0 5 10 15 20 25 0 5 10 15 20 25 lag, h lag, h 63/74 Autocovariance, gamma(h) −4 −2 0 2 4 6 Autocorrelation, rho(h) −0.5 0.0 0.5 1.0 Xt −6 −4 −2 0 2 4 6 ARMA Models Properties of AR(p) What if |φ1| ≥ 1? We know if |φ1| < 1, AR(1) is stationary. If φ1 = 1, AR(1) is the random walk. Xt =Xt−1+Wt =X0+􏰋Wi σ2 The formula Var (Xt ) = W 2 is not applicable. t i=0 1−φ1 If we assume that X0 is fixed, say X0 = 0, then 􏰜t􏰛 Var(Xt)=Var 􏰋Wi =tσW2 i=0 Hence, the random walk is not stationary. The variance depends on t and increases over time. This is even worse for |φ1| > 1. These AR(1) processes will literally explode.
64/74

ARMA Models Properties of AR(p)
Example: random walk
The random walk Xt = Xt−1 + Wt is not stationary.
65/74

ARMA Models Properties of AR(p)
Example: explosive AR(1)
Xt = 1.02Xt−1 + Wt
0 50 100 150 200
Time, t
The AR(1) process Xt = 1.02Xt−1 + Wt is not stationary and goes rapidly to ∞ or −∞. It is called an explosive process.
66/74
Xt
0 50 100 150 200 250

ARMA Models Properties of AR(p)
The general case AR(p) with p > 1
Iterating backwards to explore stationarity is easy enough for
an AR(1).
But it is cumbersome for AR(p) with p > 1.
A common approach to the general case is to use the backshift representation of AR(p) models that we mentioned before.
67/74

ARMA Models Properties of AR(p)
Back to . . . backshift of AR(p) model
The backshift operator representation of an AR(p) model is
Xt −φ1Xt−1 −φ2Xt−2 −···−φpXt−p =Wt (1−φ1B−φ2B2−···−φpBp)Xt =Wt.
where B Xt = Xt−1. This gives the concise notation φ(B)Xt =Wt.
We call φ(B) the autoregressive operator for this model φ(B) = 1 − φ1B − φ2B2 − · · · − φpBp.
AR(p) can then be seen as the solution to Xt= 1 Wt=ψ(B)Wt
φ(B )
where ψ is the inverse of the polynomial φ.
68/74

ARMA Models Properties of AR(p)
Example: backshift for AR(1) model
For an AR(1) model φ(B) = 1 − φ1B. Let’s take a look at the limiting form again

X t = 􏰋 φ j1 W t − j .
j=0
Now rewrite in the form of the inverse operator

Xt =􏰋ψjWt−j =ψ(B)Wt
j=0
where ψ(B) = 􏰊∞j=0 ψjBj and ψj = φj1.
But how can we find the inverse coefficients ψj without going through the iterative substitution above?
69/74

ARMA Models Properties of AR(p)
Inverse polynomial method 1: coefficient matching
Since ψ(B) is the inverse of φ(B), φ(B)ψ(B) = 1
which gives in the case of AR(1)
(1 − φ1B) (1 + ψ1B + ψ2B2 + . . . ) = 1
Rearranging gives:
1 + (ψ1 − φ1)B + (ψ2 − ψ1φ1)B2 + · · · = 1.
Coefficient matching between the lhs and rhs gives Clearly, ψ1 = φ1, ψ2 = φ21, …
In general, ψj = φj1 as before.
This method works as well for AR(p) with p > 1.
70/74

ARMA Models Properties of AR(p)
Inverse polynomial method 2: polynomial inversion
It does not matter if we think of φ(B) and ψ(B) as backshift operations or as (infinite) polynomials in complex numbers.
Let z ∈ C be a complex variable. Then still φ(z)ψ(z) = 1 and ψ(z) = φ−1(z).
Working with the autoregressiv operator is just like working with polynomials.
φ(z) = 1 − φ1z − φ2z2 − · · · − φpzp
is called the characteristic polynomial of AR(p).
In the AR(1) case φ(z) = 1 − φ1 z. If |φ1| < 1, φ−1(z) = 1 = 1+φ1z +φ21z2 +φ31z3 +... for |z| ≤ 1. 1−φ1z 71/74 ARMA Models Properties of AR(p) Obervation for AR(1) The root of the characteristic polynomial φ(z) = 1 − φ1 z is z=1. φ1 So |z| > 1 if and only if |φ1| < 1. Hence, the process is stationary if and only if |z| > 1.
The root of the backshift polynomial dictates whether the AR(1) is stationary.
This observation is also true for general AR(p) models.
72/74

ARMA Models Properties of AR(p)
Stationarity condition for AR(p) Model Theorem
A (unique) stationary solution to φ(B)Xt = Wt for an AR(p) model exists if and only if
φ ( z ) = 1 − φ 1 z − φ 2 z 2 − · · · − φ p z p = 0 =⇒ | z | > 1 . A complex polynomial of degree p has p roots with
multiplicity.
All roots of the characteristic polynomial must be outside of
the unit circle in C for the AR(p) process to be stationary. We will only use AR(p) with this property in ARMA models.
If a root is exactly on the circle |z| = 1, the process a called a unit root process. It is not stationary.
The random walk a a unit root process.
73/74

ARMA Models Properties of AR(p)
Example: AR(2) with complex roots
AR(2): Xt = Xt−1 − 0.9Xt−2 + Wt .
φ(z) = 1−z +0.9z2
Roots: z1 ≈ 0.556 + i0.896 and z2 ≈ 0.556 − i0.896 |z1| = |z2| ≈ 1.054
The process is stationary.
74/74