[Solved] ift6135 Assigment 0

Question 1. Let . Find using the definition of the derivative (i.e. taking the limit of difference quotients).

Question 2. Compute the gradients of , and x^>Ax w.r.t the input vector x and and matrix X.

Question 3. Recall the variance of X is Var(X) = E[(X E[X])²].

1. Let X be a random variable with finite mean. Show Var(X) = E[X²] E[X]².
2. Let X and Z be random variables on the same probability space. Show that Var(X) = EZ[Var(X|Z)]+ Var_Z(E[X|Z]). (Hint : E[X] = EY [E[X|Y ]].)
3. Question 4. Recall the density function of the uniform distribution on [a,b] for a < b is equal to for x [a,b] and 0 elsewhere.
   1. Use the density function to compute the mean and variance of a uniform distribution on [a,b].
   2. For integer n > 0, derive a formula to compute the moment E[Xⁿ] for X uniformly distributed between a and b.
   3. Question 5. Let X X be a random variable with density function f_X, and g : X Y be continuously differentiable, where X and Y are subsets of R. Let Y := g(X), which is continuously distributed with density function f_Y .
      1. Show that if g is monotonic,.
      2. Let f_X(x) = 1_x[0,1](x) and. Find a monotonic mapping g that translates f_X and f_Y .
      3. Question 6. Let Q and P be univariate normal distributions with mean and variance ,² and m,s², respectively. Derive the entropy H(Q), the cross-entropy H(Q,P), and the KL divergence D_KL(Q||P).