[Solved] EECS491 Assignment4-Multivariate Gaussians

Exercise 1. Multivariate Gaussians1.1 (5 pts) Consider the 2D normal distributionDefine three separate 2D covariance matrices for each of the following cases: and are uncorrelated; and arecorrelated; and and are anti-correlated. Plot samples from these distributions to show these properties. Use adifferent mean for each. Make sure your plots show the density.1.2 (5 pts) Compute the principal axes for each of these distributions, i.e. the eigenvectors of the covariance matrices.Use can use a linear algebra package. Plot the samples again, but this time overlay the 1, 2, and 3-sigma contours andwith the scaled eigenvectors.Exercise 2. Linear Gaussian ModelsConsider two independent multi-dimensional Gaussian random vector variablesNow consider a third variable that is the sum of the first two:2.1 What is the expression for the distribution ?2.2 What is the expression for the condidtional distribution ?2.3 Write code to illustrate the result in Q2.1. Show both the components of and that the samplingfrom the analytic result is the same as adding two samples.Exercise 3. Dimensionality Reduction and PCAIn this quesiton you will use principal component analysis to reduce the dimensionality of your data and analyze theresults.3.1 Find a set of high dimensional data. It should be continuous and have at least 6 dimensions, e.g. stats forsports teams, small sound segments or images patches also work. Note that if the dimensionality of the data is toolarge, you might run into computational efficiency problems using standard methods. Describe the data and illustrate it,if appropriate.3.2 Compute the principal components of the data. Plot a few of the largest eigenvectors and interpret them interms of how there are modeling the structure of the data.3.3 Plot, in decreasing order, the cumulative percentage of variance each eigenvector accounts for as a functionof the eigenvector number. These values should be in decreasing order of the eigenvalues. Interpret these results.3.4 Plot the original data projected into the space of the two principal eigenvectors (i.e. the eigenvectors withthe largest two eigenvalues). Be sure to either plot relative to the mean, or subtract the mean when you do this. Interpretyour results. What insights can you draw? Interpret the dimensions of the two largest principal components. Whichdimensions of the data are correlated? Or anti-correlated?Exercise 4. Gaussian Mixture Models4.1 Use the EM equations for multivariate Gaussian mixture model to write a program that implements theGaussian Mixture Model to estimates from an ensemble of data the means, covariance matrices, and class probabilities.Choose reasonable values for your initial values and a reasonable stopping criterion. Explain your code and the steps ofthe algorithm. Do not assume a diagonal or isotropic covariance matrices.4.2 Write code to plot the 3-sigma contours of each Gaussian overlayed on the data (try to find a library functionto plot ellipses). Illustrate with an example.4.3 Define a two-model Gaussian mixture test case, synthesize the data, and verify that your algorithm infers the(approximately) correct values based on training data sampled from the model and plotting the results.4.4 Apply your model to the Old Faithful dataset (supplied with the assignment files). Run the algorithm for thecases , , and . For each case, plot the progression of the solutions at the beginning, middle, andfinal steps in the learning. For each your plots (you should have 9 total), you should also print out the correspondingvalues of the mean, covariance, and class probabilities.ExplorationLike in previous assignments, in this exercise you have more lattiude and are meant to do creative exploration. Theintention is for you to teach yourself about a topic beyond whats been covered above. Please consult the rubric belowfor what is expected.Exploration Grading RubricExploration problems will be graded according the elements in the table below. The scores in the column headersindicate the number of points possible for each rubric element (given in the rows). A score of zero for an element ispossible if it is missing entirely.Substandard (+1) Basic (+2) Good (+3) Excellent (+5)PedagogicalValueNo clear statement ofidea or concept beingexplored or explained;lack of motivatingquestions.Simple problem withadequate motivation; stillcould be a usefuladdition to anassignment.Good choice of problem witheffective illustrations ofconcept(s). Demonstrates adeeper level of understanding.Problem also illustrates orclarifies common conceptualdifficulties or misconceptions.Novelty ofIdeasCopies existing problemor makes only a trivialmodification; lack ofcitation(s) for source ofinspiration.Concepts are similar tothose covered in theassignment but withsome modifications of anexisting exericse.Ideas have clear pedagogicalmotivation; creates different typeof problem or exercise to explorerelated or foundational conceptsmore deeply.Applies a technique orexplores concept not coveredin the assignment or notdiscussed at length in lecture.Clarity ofExplanationLittle or confusingexplanation; figures lacklabels or usefulcaptions; no explanationof motivations.Explanations are present,but unclear, unfocused,wordy or contain toomuch technical detail.Clear and concise explanations ofkey ideas and motivations.Also clear and concise, butincludes illustrative figures;could be read and understoodby students from a variety ofbackgrounds.Depth ofExplorationContent is obvious orclosely imitatesassignment problems.Uses existing problem fordifferent data.Applies a variation of a techniqueto solve a problem with aninteresting motivation; explores aconcept in a series of relatedproblems.Applies several concepts ortechniques; has clear focus ofinquiry that is approachedfrom multiple directions.p(x, y) N (, ) x y x yx yp(x) = N (x|x, x)p(z) = N (z|z, z)y = x + zp(y)p(y|x)y = x + zK = 1 K = 2 K = 3In [ ]