[SOLVED] Amath 562 advanced stochastic processes homework 1

1. Write about the relationship between the mathematical theory of probability and
its applications to real-world data2. Give two examples (Ω1, F1, P1) with X1(ω) and (Ω2, F2, P2) with X2(ω
′
) and show
that in both cases the cumulative probability function is given by
P1 (X1(ω) > x) = P2 (X2(ω
′
) > x) = e
−rx3. Consider a reference measure P1 and a collection of continuously parameterized
measures P2(θ). Assume the RND Z(ω; θ) = dP2
dP1
(ω; θ) is smooth with respect to θ.
Now let
Ik(θ) = −E
P2

∂
k
∂θk
log 
dP2
dP1
(ω; θ)

Assuming expectations and differentiations with respect to θ are interchangeable,
show the following:
(a)
I0(θ) = −
Z
Ω

dP2
dP1
(ω; θ)

log 
dP2
dP1
(ω; θ)

P1(dw)
This is called the Shannon entropy of P2 w.r.t. the measure P1.(b)
I1(θ) = 0(c)
I2(θ) = E
P2
“
∂
∂θ log 
dP2
dP1
(ω; θ)
2
#
≥ 0
This is known as the Fisher information.4. The Legendre-Fenchel transform (MLN §3.5) is given by
Λ
∗
(x) = sup
t∈R
{xt − Λ(t)}
for x ∈ R.
Assuming that the Λ(t) is strictly convex and twice differentiable, then the supremum in the equation is given by
Λ
∗
(x) = 
Λ
∗
(t) = x(t)t − Λ(t)
x(t) = Λ′
(t)
This gives the function Λ∗
(x) in a parametric form in terms of t as a continuous
parameter. Show that this equation implies the following:
(a) Λ∗
(x) is also convex.(b) An inverse, dual relation
Λ(t) = sup
x∈R
{tx − Λ
∗
(x)}(c) With the pair of convex functions Λ(x) and Λ∗
(t) defined above, show that for
any real x and t,
Λ(t) + Λ∗
(x) − tx ≥ 0
What is the condition for equality to hold?5. For Ω = {1, 2, . . . , n} and two probability measures ν = (ν1, ν2, . . . , νn) and p =
(p1, p2, . . . , pn), where νi
, pi > 0, the Shannon relative entropy is given by
H[ν ∥ p] = Xn
i=1
νi
ln 
νi
pi

(a) Show that for any two probability measures ν >> 0 and p >> 0
H[ν ∥ p] ≥ 0
This is known as Jensen’s inequality.(b) Show that the Legendre-Fenchel transform of H[ν||p] is given by
sup
ν>>0
(Xn
i=1
ϵiνi − H[ν||p]
)
= lnXn
i=1
pie
ϵi
in which ϵ = (ϵ1, ϵ2, . . . , ϵn) is the conjugate variable to ν.6. Let Ω be a simply connected compact domain in R
m. Consider the statistical mechanical energy function E(x) for x ∈ Ω, and the sequence of probability measures
whose density functions w.r.t. the Lebesgue measure is
f
(n)
(x) = Ane
−nE(x)
where An is the normalization factor
A
−1
n =
Z
Ω
e
−nE(x)
dx
which is assumed to satisfy
limn→∞
log An
n
= 0
Note that this is the Bolzmann-Gibbs distribution where n takes the role of inverse
temperature β.
We now add some additional structure by letting x = (x1, y), where x1 ∈ R and
y = (x2, . . . , xm) ∈ R
m−1
. Now let f
(n)
1
(x) be the marginal distribution
f
(n)
1
(x) = An
Z
Ω∩Rm−1
e
nE(x,y)
dy.
Where we assume
limn→∞
1
n
log f
(n)
1
(x) = −Λ
∗
(x)
(a) Show that the n-scaled cumulant generating function
log Z
f
(n)
1
(x)e
ntxdx
has the limit
limn→∞
1
n
log Z
f
(n)
1
(x)e
ntxdx = max
x∈R
{tx − Λ
∗
(x)}
Assume all functions are sufficiently smooth to allow one to freely exchange
limits and integration w.r.t x.(b) Denoting
Λ(t) = max
x∈R
{tx − Λ
∗
(x)}
show that one can obtain Λ∗
(x) parametrically as
Λ
∗
(x) = (
Λ
∗
(t) = −
d
d(1/t)

Λ(t)
t

x(t) = Λ′
(t)7. Let G = (qkl)K×K be the infinitesimal generator, or transition probability rate, for
a continuous-time Markov chain Xt ∈ S = {1, 2, . . . , K}:
Pr{Xt+dt = l|Xt = k} =

qkldt l ̸= k
1 −
P
m̸=k
qkmdt l = k
Let us assume that G has rank K − 1 and is diagonalizable with eigenvalues λ1 =
0, λ2, . . . , λK. It can be shown using the Perron-Frobenius theorem that the real
part of λk is negative for all k ≥ 2.
(a) Let τ be a fixed time interval. Show that
P = e
τG := X∞
j=0
1
j!
(τG)
j
is a transition probability matrix for a Markov chain, which has the same
invariant probability as G, π = (π1, . . . , πK).(b) Let ξ(i) be a real-valued function for i ∈ S. Let X0, Xτ , . . . , Xmτ , . . . be the
sample path of the discrete-time Markov chain. Now let
ξ¯
(m)
=
1
m
mX−1
j=0
ξ(Xjτ )
and show that
limm→∞
E
h
ξ¯
(m)
i
=
X
K
k=1
πkξ(k)