[Solved] CSE 5243 Homework 3

• Please work on this assignment as an individual, not in a group.
• A 15% penalty all homework handed in up to 24 hours late; homework will not be accepted more than 24 hours late
• Show your work

Question #1: Given the below confusion matrix for classifier C, compute the accuracy rate, error rate, true positive, false positive, precision and F-measure.

Question #2: Consider the training examples shown in the following table for a binary classification problem and answer the following questions:

1. What is the entropy of this collection of training examples with respect to the class attribute?
2. What are the information gains for attributes a₁ and a₂ relative to these training examples?
3. For a₃, which is a continuous attribute, find the binary split which maximizes the information gain.
4. If you were building a decision tree with a single (binary) split, using the information gain measure, which attribute would you split on? Draw the decision tree and show the number of positive and negative examples in each leaf node.
5. Would your choice of optimal split in part d) change if you used the gain ratio instead of the information gain?

Question #3: Using the dataset given in the table below, build a 2-level decision tree [one split from the root node, followed by 1 additional split from each child node] using the Gini index as the splitting criteria. You should use multiway splits for each attribute. Draw the decision tree and show the number of class C0 and C1 records in each leaf node. Show your work by computing the Gini index of the possible splits. You do not need to worry about pruning the tree that youve built.

Question #4: You have built a decision tree model and would like to estimate the generalization performance of the model. Before building the model, you set aside a test dataset with 1200 records. The predicted class label for each of the 1200 records is calculated, and the results are summarized in the table below. The number of actual positive and negative class examples are aggregated for each score range based on the posterior probability P(+ | x). For example, there are a total of 131 test records with P(+ | x) >= 0.9, 118 of which are actually positive and 13 are actually negative.

1. Given a decision threshold of 0.5, write the confusion matrix and calculate the accuracy of the model.
2. Change the decision threshold to 0.7, write the confusion matrix and calculate the accuracy. Compare the TPR and FPR of this choice to those obtained in part a.
3. Using the data in the table, draw the ROC curve. The curve should have 10 points, one for each row in the table.

Table for Exercise #4

Note that some problems are taken from the textbook Introduction to Data Mining.