[SOLVED] Amath 562 advanced stochastic processes homework 2

1. Consider a measurable space (Ω, F) with finite elementary event set Ω = {1, . . . , n},
the corresponding F = 2Ω, and the Lebesgue-measure like counting measure νi =
1, 1 ≤ i ≤ n. A stochastic Markov (chain) dynamics, Xk, has one step transitions
in terms of a set of conditional probabilities p
(ν)
(i, j) = Pr{Xk+1 = j|Xk = i}. This
assumption of a ”counting measure” ν is implicit in all Markov chain theory.
(a) If a Markov chain with p
(ν)
(i, j) has a unique invariant probability π =
{π1, . . . , πn} with all positive πk, express the transition probability as the
Radon-Nikodym derivative w.r.t. π, denoted as p
(π)
(i, j).(b) Show that
πP
(π) = 1
and
P
(π)π
T = 1
T
where P
(π)
is the transition probability matrix w.r.t. π, 1 = (1, . . . , 1), and
1
T
is the column vector of 1’s. Please explain these two equations.(c) The reversibility of a Markov chain is introduced in §4.5 of MLN. What is the
P
(π) of a reversible Markov chain?(d) In discrete time, a deterministic first-order ”dynamics” in the Ω is defined by
a one step map S : Ω → Ω. Since a deterministic first-order dynamics is just a
special case of a Markov dynamics, express the transition probability p
(ν)
(i, j)
corresponding to the map S.(e) Show the deterministic dynamics in (d) has an invariant probability π =
(
1
n
, . . . ,
1
n
) if and only if the map S is one to one. Within the context of a
deterministic S, discuss the notion of irreducibility defined in §4.3 of MLN.2. Consider the continuous time Markov chain with generator
G =


−λ λ 0 0 0
. . .
µ −µ − λ λ 0 0
. . .
0 2µ −2µ − λ λ 0
. . .
0 0 3µ −3µ − λ λ
. . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.


in which λ, µ > 0.
(a) Find its invariant probability distribution π.(b) Assume X0 = 0. Using the matrix exponential symbol (e
Gt
)ij , give the joint
probability for the finite trajectory
Pr{Xt1 = i1, Xt2 = i2, . . . , Xtn = in},
where 0 < t1 < t2 < · · · < tn.(c) Introducing the probability generating function (see MLN §5.2)
GX(s, t) = E[s
Xt
].
Show that GX(s, t) satisfies the following partial differential equation
∂
∂tGX(s, t) = u

s, t, GX,
∂GX
∂s ,
∂
2GX
∂s2

.
Give the explicit form for the function on the rhs.(d) Show that the solution to the PDE in (c), with initial data GX(s, 0), is
GX(s, t) = GX1 + (s − 1)e
−µt
, 0


exp 
λ
µ
(s − 1)
1 − e
−µt

)
(e) Verify that the limit of GX(s, t) as t → ∞ agrees with the π obtained in (3. Let the generator G of a three-state continuous time Markov chain Xt be given by


−α − β α β
β −α − β α
α β −α − β

 = α


−1 − b 1 b
b −1 − b 1
1 b −1 − b

 ,
in which α, β > 0, b = β/α. Note the G matrix is circulant, so its eigenvalues
and eigenvectors have special forms which are readily obtained. Assuming that
X0 follows the invariant probability distribution π; therefore X
(st)
t
is a stationary
Markov chain. Let the function y(X) = −1, 0, 1 corresponding to the states X =
1, 2, 3.
(a) Compute µ = E[y(X
(st)
t
)] and σ
2 = V[y(X
(st)
t
)].(b) For two random variables V (ω) and W(ω),
E [(V − E[V ])(W − E[W])]
is called covariance between V and W. Find an analytical expression for the
covariance function
g(τ ) = E
hy(X
(st)
t+τ
) − µ
 y(X
(st)
t
) − µ
i .(c) Show that
lim
T→∞
1
T
Z T
0
y(X
(st)
t
)dt = µ
where the convergence is by L
2
.(d) Use a computer and Monte Carlo simulation to verify that
lim
T→∞
1
T
Z T
0
y(X
(st)
t+τ
)y(X
(st)
t
)dt − µ
2
agrees with g(τ ) you obtained from (b)4. Let W(t) be a standard Brownian motion. Introducing a function of the Brownian
motion
W˜ (s) = (1 − s)W

s
1 − s

0 < s < 1
Compute its expected value, variance, and covariance function
Cov[W˜ (s1), W˜ (s2)] 0 < s1 < s2 < 1
W˜ (s) is known as a Brownian bridge.5. W(t) is a standard Brownian motion. What is the characteristic function of W(Nt)
where Nt
is a Poisson process with intensity λ, and the brownian motion W(t) is
independent of the Poisson process Nt
.