[SOLVED] COMP3314 Machine Learning Quiz 1 Processing

COMP3314 Machine Learning

Quiz 1

19 Oct. 2020

Q1(5 marks): Consider the perceptron algorithm and the following sequence of samples.

Write down all steps of the perceptron algorithm using the given sequence of training data. Let all weights be initialized with 1 and let the bias be 0.

Q2.1 (1 mark): Consider data that is linearly separable. Explain what that means.

Q2.2 (1 mark): What is the impact of the parameter C in SVM?

Q2.3 (2 mark): Consider soft margin SVM on a binary classification problem. Let the data be linearly separable. If we would use a relatively small value ofC, could it hurt the training accuracy? Explain.

Q2.4 (3 marks): Consider, again, soft margin SVM on a binary classification problem. This time let’s use a relatively large value for C on the data below.

Copy the figure to your handwritten notes and sketch the decision boundary that you think we would get with SVM. Provide a short explanation.

Q2.5 (3 marks): Again, copy the figure above to your handwritten notes and this time indicate which points are special points. A special point is a point that you could remove from the training set and then a retrained SVM would get a different decision boundary compared to the full training set.

Q3 (5 marks): We will use the dataset below to learn a decision tree which predicts if students pass COMP3314 (Yes or No), based on their cumulative GPA (High, Medium, or Low) and if they studied (True, False).

Draw the full decision tree that would be learned for this dataset using the entropy. Show all calculations and write down the entropy values (use log2) for all nodes in the tree.