[Solved] CS596 Homework # 3

Problem 1: Let ₁,₂ be random variables which 99% of the time are independent and Normally (Gaussian) distributed, both with mean 0 and variance 1 and 1% of the time they are independent and Normally distributed both with mean 0 and variance ² 6= 1. a) Compute the joint pdf of the two random variables.

1. b) Examine if the two random variables are independent. c) Give an example of two random variables that are uncorrelated but not independent.

Problem 2: Let , be random variables that are related through the equality

= | + s|.

1. a) If the pdf of is f(x) compute the pdf of when s is a deterministic quantity. b) Repeat the previous question when s is a random variable independent from and takes only the two values 0 and 1 with probability 0.2 and 0.8 respectively. c) Under the assumptions of question b) compute the posterior probability P(s = 0| = z). Hint: For the computation of the pdf of a random variable the simplest way is to start with the computation of the cdf and then take the derivative. For b) use total probability.

Problem 3: You are given an unfair coin where every time you throw it the probability to observe head is p (0,1) (and tail 1 p) where p is unknown. Assume that you throw the coin N times and you report the sequence of heads (H) and tails (T). If N_h is the number of heads you observed and N N_h the number of tails then: a) Find the probability of the specific sequence you have observed as a function of p,N,N_h. b) Compute the maximum likelihood estimator (MLE) of p if you are given such a sequence. Is the result familiar/expected? c) Compute the average of your estimate and the mean square error from the true unknown value p. What do you conclude about your estimate as N ?

Problem 4: Consider the random data {x₁,,x_N} and assume that they are Markov related as follows

x_n = x_n1 + w_n, n = 2,,N,

where || < 1, {w_n} are independent and identically distributed Gaussian random variables with mean 0 and variance 1 and independent from x₁. Random variable x₁ is also Gaussian with mean 0 and variance . a) Using the fact that linear combinations of Gaussians is also Gaussian show that all x_n are Gaussian.

1. b) Find the joint probability density function of {x₁,,x_N} assuming is given. c) Find the maximum likelihood estimator (MLE) of parameter . Hint: For b) use the fact that {x₁,x₂ x₁,,x_N x_N1} are independent Gaussians.

Problem 5: Assume that you have two hypotheses H₀,H₁. Under Hypothesis H₀ your observation vector X = [x₁,,x_N]^T is Gaussian with mean vector ₀ and covariance matrix ₀, while under H₁ it is again Gaussian with mean ₁ and covariance matrix ₁. Assume that the two hypotheses are equiprobable (P(H₀) = P(H₁) = 0.5). a) Find the Likelihood ratio test (LRT) and its equivalent form by taking the logarithm on both sides of the inequality. b) What is the latter form reduced to when the two covariance matrices are equal (₀ = ₁)? c) What if additionally the two means are equal? Does this make sense?