[Solved] CS534 Assignment 2

In this assignment, we are using 3 variations of perceptron to to classify handwritten digits of numbers 3 or 5.

Preprocessing

1. The output Y from the train and validation datasets are changed to 1 and -1 ( +1 to 3 and -1 to 5) and removed from the dataset for further processing with the train and validation data.
2. Add bias feature ( equals to 1 ) to train, validation and test data to improve the fit.

Part 1 : Online Perceptron

• ​Implement the online perceptron model with algorithm described in Algorithm 1. Set the iters = 15. During the training, at the end of each iteration use the current w to make prediction on the validation samples. Record the accuracies for the train and validation at the end of each iteration. Plot the recorded train and validation accuracies versus the iteration number.
• Does the train accuracy reach to 100%? Why? Solution

Part 2 : Average Perceptron

• Please implement the average perceptron described in Algorithm 2
• Plot the train and validation accuracies versus the iteration number for iters = 1, …, 15.
• How average model has affected the validation accuracy comparing to the online perceptron?

Part 3 : Polynomial Kernel Perceptron

• Implement the polynomial kernel function k_p​ in the Algorithm 3. This function takes_​ two vectors x​1 and x_​ ​2 and an integer p for the polynomial degree, and returns a real value._​
• Define a Gram matrix K with size N × N where N is the number of training samples. Fill matrix K(i, j) = k p (x​i , x_​ ​j ) for all of the pairs in the training set._​
• Implement the rest of the kernel perceptron in Algorithm 3. For each p in [1, 2, 3, 7,15]:
• Run the algorithm to compute α.
• At the end of each iteration use the current α to predict validation set.

• Record the train and validation accuracy for each iteration and plot the train and validation accuracies versus the iteration number.

• Record the best validation accuracy achieved for each p over all iterations.

POWER	ACCURACY (in %)
P = 1	95.0276243094
P = 2	98.3425414365
P = 3	98.4653161449
P = 7	97.7286678944
P = 15	96.5623081645

• Plot the recorded best validation accuracies versus degrees. Please explain how p is affecting the train and validation performance.
• Use your best α (the best you found over all d and iterations above) to predict the test data-set. Please name the predicted file as kplabel.csv.