[SOLVED] Probability & statistics for eecs homework 12

1. Laplace’s law of succession says that if X1, X2, . . . , Xn+1 are conditionally independent
Bern(p) r.v.s given p, but p is given a Unif(0, 1) prior to reflect ignorance about its
value, then
P (Xn+1 = 1 | X1 + · · · + Xn = k) = k + 1
n + 2As an example, Laplace discussed the problem of predicting whether the sun will rise
tomorrow, given that the sun did rise every time for all n days of recorded history; the
above formula then gives (n+1)/(n+2) as the probability of the sun rising tomorrow.(a) Find the posterior distribution of p given X1 = x1, X2 = x2, . . . , Xn = xn, and show
that it only depends on the sum of the xj (so we only need the one-dimensional
quantity x1 +x2 +· · ·+xn to obtain the posterior distribution, rather than needing
all n data points).(b) Prove Laplace’s law of succession, using a form of LOTP to find
P (Xn+1 = 1 | X1 + · · · + Xn = k)
by conditioning on p.(c) Reinterpret the Laplace’s law of succession from the perspective of Beta-Binomial
Conjugacy.2. (a) Let p ∼ Beta(a, b), where a and b are positive real numbers. Find E(p2(1 − p)2),
fully simplified (Γ should not appear in your final answer).Two teams, A and B, have an upcoming match. They will play five games and the
winner will be declared to be the team that wins the majority of games. Given p, the
outcomes of games are independent, with probability p of team A winning and (1 − p)
of team B winning. But you don’t know p, so you decide to model it as an r.v., with
p ∼ Unif(0, 1) a priori (before you have observed any data).To learn more about p, you look through the historical records of previous games
between these two teams, and find that the previous outcomes were, in chronological
order, AAABBAABAB. (Assume that the true value of p has not been changing over
time and will be the same for the match, though your beliefs about p may change over
time.)(b) Does your posterior distribution for p, given the historical record of games between
A and B, depend on the specific order of outcomes or only on the fact that A
won exactly 6 of the 10 games on record? Explain.(c) Find the posterior distribution for p, given the historical data.
The posterior distribution for p from (c) becomes your new prior distribution, and the
match is about to begin!(d) Conditional on p, is the indicator of A winning the first game of the match positively correlated with, uncorrelated with, or negatively correlated with the indicator of A winning the second game of the match? What about if we only condition
on the historical data?(e) Given the historical data, what is the expected value for the probability that the
match is not yet decided when going into the fifth game (viewing this probability
as an r.v. rather than a number, to reflect our uncertainty about it)?3. Let U1, . . . , Un be i.i.d. Unif(0, 1). Let U(j) be the corresponding jth order statistic,
where 1 ≤ j ≤ n.
(a) Find the joint PDF of U(1), . . . , U(n).
(b) Find the joint PDF of U(j) and U(k), where 1 ≤ j < k ≤ n.
(c) Let X ∼ Bin(n, p) and B ∼ Beta(j, n − j + 1), where n is a positive integer and
j is a positive integer with j ≤ n. Show using a story about order statistics that
P(X ≥ j) = P(B ≤ p).This shows that the CDF of the continuous r.v. B is closely related to the CDF
of the discrete r.v. X, and is another connection between the Beta and Binomial.(d) Show that
! x
0
n!
(j − 1)!(n − j)!
t
j−1
(1 − t)
n−j
dt = “n
k=j
#n
k
$
xk(1 − x)
n−k,
without using calculus, for all x ∈ [0, 1] and j, n positive integers with j ≤ n.4. If X ∼ Pois(λ), Z ∼ Gamma(k + 1, 1), where k is a nonnegative integer. Show the
Poisson-Gamma Duality holds:
P(X ≤ k) = P(Z > λ).
Hint: Two possible methods, where one is based on the identity in 3(d), the other is
based on the model of Poisson process.5. Programming Assignment:
(a) Use the Acceptance-Rejection Method to obtain the samples from distribution
Beta(2, 4). You need to plot the pictures of both histogram and the theoretical
PDF.(b) Use the Acceptance-Rejection Method to obtain the samples from the standard
Normal distribution N (0, 1). You are required to show the correctness of your
algorithm in theory.(c) Both the Acceptance-Rejection Method and Box-Mulller Method can obtain the
samples from the standard Normal distribution N (0, 1). Discuss the pros and cons
of such two methods.(d) Use the importance sampling method to evaluate the probability of rare event
c = P(Y > 8), where Y ∼ N(0, 1).