[Solved] Lab 8 Neural Networks and Deep Learning Advanced Machine Learning DATA442/642

Exercise 1

Consider a two-dimensional class problem that involves two classes ₁(+1) and ₂(1). Each one of them is modeled by a mixture of equiprobable Gaussian distributions. Specifically, the means of the Gaussians associated with ₁ are [5,5]^> and [5,5]^>, while the means of the Gaussians associated with ₂ are [5,5]^>,[0,0]^>, and [5,5]^>. The covariances of all Gaussians are ²I, where ² = 1.

• Generate and plot a data set X₁ (training set) containing 100 points from ₁ (50 points from each associated Guassian) and 150 points from ₂ (again 50 points from each associated Gaussian). In the same way, generate an additional set X₂ (test set).
• Based on the training set X₁, train a train a two-layer neural network with two nodes in the hidden layer, each one having the rectified linear activation function or ReLU and a single output node with linear activation function using the standard backpropagation algorithm for 6000 iterations and step size equal to 0.01. Compute the training and test errors, based on X₁ and X₂ Also, plot the test points as well as the decision lines formed by the network.
• Repeat step (b) for step size equal to 0.0001 and comment on the results.
• Repeat step (b) for step size equal to 0.0001 and k = 1,4,20,50 hidden layer nodes and comment on the results.