CS计算机代考程序代写 (Continuous) Gaussian Processes

(Continuous) Gaussian Processes

Multivariate normal

A random vector ~X = (X1, . . . ,Xd) is (multivariate) normal or
(multivariate) Gaussian if and only if every linear combination ~x · ~X
is univariate normal (with variance in [0,∞)).

The distribution of a (multivariate) normal vector can be specified
by the mean vector and the covariance matrix.

A Gaussian process (Xt)t∈I is a random process for which the
finite-dimensional distributions are all multivariate normal,
i.e. (Xt1 , . . . ,Xtr ) is multivariate normal for every r ∈ N,
t1 < t2 < · · · < tr all in I . Typically I is [0,∞) or [0, 1]. 2 Continuous Gaussian processes We’ll restrict our attention to Gaussian processes (Xt)t∈I that are continuous in t with probability 1. Recall that for continuous processes, the distribution of the process is determined by the finite-dimensional distributions. For a Gaussian process the f.d.d. are multivariate Gaussian, determined by the mean vectors and covariance matrices. It follows that the distribution of a continuous Gaussian process (Xt)t∈I is determined by two functions: the mean function µ(t) = E[Xt ] for t ∈ I and the covariance function Σ(s, t) = Cov(Xs ,Xt) for s ≤ t both in I . 3 Brownian motion simulation 4 Brownian bridge 5 Brownian motion and Brownian bridge (standard) Brownian motion (Wt)t≥0 is a continuous Gaussian process with µ(t) = 0 and Σ(s, t) = s for s ≤ t. (standard) Brownian bridge (Bt)t∈[0,1] is a continuous Gaussian process with µ(t) = 0 and Σ(s, t) = s(1− t) for s ≤ t. How do we know that such processes exist? We can construct them as limits of things that exist. 6 Sketch construction of Brownian motion (on [0, 1]) Let (Zq)q∈Q∩[0,1] be i.i.d. standard normal random variables. Define a sequence of random functions (W (n) t )t∈[0,1] for n ∈ N by: W (1) t = tZ1, i.e. set W (1) 0 = 0 and W (1) 1 = Z1, and then linearly interpolate. Set W (2) to be the same at 0 and 1 but set W (2) 1/2 = W (1) 1/2 + 1 √ 22 Z1/2, and then linearly interpolate in between. More generally define W (n+1) to be equal to W (n) at points 2i/2n, and define W (n+1) at the points q of the form (2i + 1)/2n by adding some extra randomness 1√ 2n Zq to W (n) q and then linearly interpolating. 7 Sketch construction of Brownian motion (on [0, 1]) 0 1 8 Sketch construction of Brownian motion (on [0, 1]) 0 1 9 Sketch construction of Brownian motion (on [0, 1]) 0 1 10 Sketch construction of Brownian motion (on [0, 1]) It’s possible to check that W (n+1) has the claimed mean and covariance functions if we restrict to times of the form i/2n. This sequence of (random) continuous functions converges (as n→∞ (uniformly) to a random continuous function (Wt)t∈[0,1]. This random function has the correct mean and covariance functions since it does at every dyadic rational point. 11 Brownian motion (BM) BM has independent increments: If 0 < s1 < t1 < s2 < t2, . . . , < sn < tn then (Wti −Wsi )i≤n are independent random variables. e.g. (W2 −W1,W4 −W2) is bivariate normal (why?), and E[(W4 −W2)(W2 −W1)] = E[W4W2]− E[W 22 ]− E[W4W1] + E[W2W1] = 2− 2− 1 + 1 = 0. Definition of BM is equivalent to saying that (Wt)t≥0 is a continuous process with: (i) W0 = 0 and, (ii) with independent increments (if they are disjoint) (iii) and Wt −Ws ∼ N (0, t − s) for every t > s ≥ 0.

Exercises

Suppose that (Wt)t≥0 is a BM:

(a) If s > 0 and Xt = Wt+s −Ws then (Xt)t≥0 is a BM (and in
fact it is independent of (Wu)u≤s).

(b) If X0 = 0 and Xt = tW1/t for t > 0 then (Xt)t≥0 is a BM.

(d) If Xt = Wt − tW1 then (Xt)t∈[0,1] is a Brownian Bridge.

Path properties of BM

Brownian motion is recurrent: for every t > 0 there exists T > t
such that WT = 0.

Sketch proof: Note that Wn −Wn−1 for n ∈ N are
i.i.d.∼ N (0, 1). Eventually one of these (say WN −WN−1) has size
greater than 2. Thus either |WN | > 1 or |WN−1| > 1. Since W is
continuous this shows that T1 = inf{t : |Wt | = 1} is finite.

Similarly we can define Tj = inf{t > Tj−1 : |Wt −WTj−1 | = 1}.
One can show that (Si )i∈Z+ defined by S0 = 0 and Sj = WTj for
j ≥ 1 is a simple (unbiased) random walk. This simple random
walk visits 0 infinitely often…. In fact, something stronger is true,
e.g. in any interval of time [0, ε] where ε > 0, BM visits 0 infinitely
often.

Since this simple random walk also visits every integer infinitely
often this shows that BM visits every point in R infinitely often as
well.

Properties of BM

BM is both a Markov process (with state space R), and a
Martingale.

Brownian motion is not differentiable at any point.

E.g. Wh
h

for small h is like tW1/t for large t, which as a function
of t has the same law as (Wt)t>0 so it oscillates (does not
converge) as t →∞.

Functional CLT

Let (Xi )i∈N be i.i.d. random variables with mean 0 and variance 1.
Let

Z
(n)
t =

∑bntc
i=1 Xi√
n

Then (Z
(n)
t )t≥0

D→ (Wt)t≥0.

(If Z (n),Z are random objects taking values in some space E we

write Z (n)
D→ Z if E[f (Z (n))]→ E[f (Z )] as n→∞ for every

bounded continuous function f : E → R)

Brownian bridge

Brownian bridge does not have independent increments.
E.g. B1 − B1/2 = −(B1/2 − B0).

Exercise: Suppose that (Bt)t∈[0,1] is a BB, and Z ∼ N (0, 1) is
independent of (Bt)t∈[0,1]. Then Xt = Bt + tZ is a BM on [0, 1].

FCLT for empirical processes

Let (Xi )i∈N be i.i.d. with cdf F , and let F
(n)(x) = 1

∑n
i=1 1{Xi≤x}.

Then (√
n(F (n)(x)− F (x))

)
x∈R

D→
(
BF (x)

)
x∈R.

Note that F (x) ∈ [0, 1] so the right hand side is well defined.

If Xi ∼ U(0, 1) then BF (t) = Bt .

Exercise: Compute the mean of the left hand side at the point x .
Calculate the covariance of the left hand side evaluated at the
points x and y where x ≤ y .

More amazing facts

Wt√
t
∼ N (0, 1), for every t ≥ 0,

but
Wt√
t

oscillates unboundedly (see the Law of the Iterated

Logarithm).

(There is a similar result for simple random walk: as n→∞ the
distribution of n−1/2Sn converges to N (0, 1) but as a random
sequence n−1/2Sn does not converge.)

Let (W
[i ]
t )t≥0 be independent BM for i ∈ N. Then(

(W
[1]
t ,W

[2]
t )
)
t≥0 is a 2-dimensional BM,

(
(W

[1]
t ,W

[2]
t ,W

[3]
t )
)
t≥0

is a 3-dimensional BM, etc.

2-dimensional BM

More amazing facts

I (1-dimensional) BM visits every point in R infinitely often.
I for 2-dimensional BM, for every k ∈ Z+ there are (random)

points in R2 visited exactly k times. Every neighbourhood of
every point is visited infinitely often.

I for 3-dimensional BM there are (random) points visited
exactly twice, and no point in R3 is visited 3 or more times,
|Bt | → ∞ as t →∞.

I for 4-dimensional BM no point in R4 is hit more than once.

Related Posts