[Solved] Computational stats -TP 2

M2 Mathmatiques,

TP 2: Expectation-Maximisation algorithm Importance samplingExercise 1: Discrete distributions Let n E N* and X = {x1,.. . , xn} a set of n distinct real numbers. Let (pi)iQ1,nj a sequence of real numbers such that :Vi E I1, n, pi > 0 andn i=1pi = 1.1. Explain how to generate a random variable X having the discrete distribution on X given by (pi)iE1,n :ViEI[1,n, P(X = xi) = pi. 2. Write (in Python, Julia, Matlab, Octave) the corresponding algorithm. 3. Generate a sequence (Xi)iQ1,N1 of i.i.d. random variables having the same distribution as X for large values of N. Compare the empirical distribution to the theoretical distribution of X. (In Python, you can use the function numpy.histogram).Exercise 2: Gaussian mixture model and the EM algorithm A Gaussian mixture model (GMM) is useful for modelling data that comes from one of several groups: the groups might be different from each other, but data points within the same group can be well-modelled by a Gaussian distribution. The main issue is to estimate the parameters of the mixture, i.e to find the most likely ones. Moreover, we aim to determine if our sample follow a Gaussian mixture distribution or not.Let consider a n-sample. For each individual, we observe a random variable Xi and assume there is an unobserved variable Zi for each person which encode the class of Xi. More formally, we consider a mixture of m Gaussian: let (c1,.. . , cm) E Rm +such that m i=1 i = 1 and the following hierarchical model:ViE1,n, VjE1,m, P9(Zi = j) = cjandViE1,n, VjE1,m Xi | 9, {Zi = j} N (j j) Unless otherwise stated, we suppose that m is fixed.1. Identify the parameters, denoted 8, of the model and write down the likelihood of 9 given the outcomes (xi)iQ1,nj of the i.i.d n-sample (Xi)iE1n, i.e the p.d.fL(x1 xn; ) =i=1f(xi).