[SOLVED] Probability & statistics for eecs homework 13

1. Let X1, X2, . . . be i.i.d. Expo(1).
(a) Let N = min{n : Xn ≥ 1} be the index of the first Xj to exceed 1. Find the
distribution of N − 1 (give the name and parameters), and hence find E(N).(b) Let M = min{n : X1 + X2 + · · · + Xn ≥ 10} be the number of Xj ’s we observe
until their sum exceeds 10 for the first time. Find the distribution of M − 1 (give
the name and parameters), and hence find E(M).(c) Let X¯n = (X1 + · · · + Xn)/n. Find the exact distribution of X¯n (give the name
and parameters), as well as the approximate distribution of X¯n for n large (give
the name and parameters).2. Let the random variables X1, X2, . . . , Xn be independent with E(Xi) = µ, a ≤ Xi ≤ b
for each i = 1, . . . , n, where a, b are constants. Then for any ! ≥ 0, show the Hoeffding
Bound holds:
P
!”
”
”
1
n
#n
i=1
Xi − µ
”
”
” ≥ !
$
≤ 2 exp
%
− 2n!2
(b − a)2
&
.
Hint: Hoeffding Lemma + Chernoff Inequality.3. Given a random variable X with expectation µ and variance σ2. For any a ≥ 0, show
the following inequality holds:
P (X − µ ≥ a) ≤
σ2
σ2 + a2 .4. We observe a collection X = (X1, . . . , Xn) of random variables, with an unknown
common mean whose value we wish to infer. We assume that given the value of the
common mean, the Xi are normal and independent, with known variances σ2
1, . . . , σ2
n.We model the common mean as a random variable Θ, with a given normal prior (known
mean x0 and known variance σ2
0). Find the posterior PDF of Θ.5. (a) We wish to estimate the parameter for an exponential distribution, denoted by θ,
based on the observations of n independent random variables X1, . . . , Xn, where
Xi ∼ Expo(θ). Find the MLE of θ.(b) We wish to estimate the mean µ and variance ν of a normal distribution using n
independent observations X1, . . . , Xn, where Xi ∼ N (µ, ν). Find the MLE of the
parameter vector θ = (µ, ν).