[Solved] CSE176 HomeWork#1

Exercise 1: Bayes rule (. Suppose that 5% of competitive athletes use performanceenhancing drugs and that a particular drug test has a 2% false positive rate and a 1.5% false negative rate.

1. (3 points) Athlete A tests positive for drug use. What is the probability that Athlete A is using drugs?
2. (3 points) Athlete B tests negative for drug use. What is the probability that Athlete B is not using drugs?

Exercise 2: Bayesian decision theory: losses and risks Consider a classification problem with K classes, using a loss _ik 0 if we choose class i when the input actually belongs to class k, for i,k {1,,K}.

1. (2 points) Write the expression for the expected risk R_i(x) for choosing class i as the class for a pattern x, and the rule for choosing the class for x.

Consider a two-class problem with losses given by the matrix.

2. (3 points) Give the optimal decision rule in the form p(C₁|x) > as a function of ₂₁.
3. (3 points) Imagine we consider both misclassification errors as equally costly. When is class 1 chosen (for what values of p(C₁|x))?
4. (3 points) Imagine we want to be very conservative when choosing class 2 and we seek a rule of the form p(C₂|x) > 0.99 (i.e., choose class 2 when its posterior probability exceeds 99%). What should ₂₁ be?

Exercise 3: association rules . Given the following data of transactions at a supermarket, calculate the support and confidence values of the following association rules: meat avocado, avocado meat, yogurt avocado, avocado yogurt, meat yogurt, yogurt meat. What is the best rule to use in practice?

Exercise 4: true- and false-positive rates Consider the following table, where xn is a pattern, y_n its ground-truth label (1 = positive class, 2 = negative class) and p(C₁|xn) the posterior probability produced by some probabilistic classification algorithm:

We use a classification rule of the form p(C₁|x) > where [0,1] is a threshold.

1. (8 points) Give, for all possible values of [0,1], the predicted labels and the corresponding confusion matrix and classification error.
2. (2 points) Plot the corresponding pairs (fp,tp) as an ROC curve.

Exercise 5: ROC curves . Imagine we have a classifier A that has false-positive and true-positive rates fp_A,tp_A [0,1] such that fp_A > tp_A (that is, this classifier is below the diagonal on the ROC space). Now consider a classifier B that negates the decision of A, that is, whenever A predicts the positive class then B predicts the negative class and vice versa. Compute the false-positive and true-positive rates fp_B,tp_B for classifier B. Where is this point in the ROC space?

Exercise 6: least-squares regression (14 points). Consider the following model, with parameters = {₁,₂,₃} R and an input x R:

h(x;) = ₁ + ₂ sin2x + ₃ sin4x R.

1. (2 points) Write the general expression of the least-squares error function of a model h(x;) with parameters given a sample .
2. (2 points) Apply it to the above model, simplifying it as much as possible.
3. (6 points) Find the least-squares estimate for the parameters.
4. (4 points) Assume the values are uniformly distributed in the interval [0,2]. Can you find a simpler, approximate way to find the least-squares estimate ? Hint: approximate by an integral.

Exercise 7: maximum likelihood estimate . A discrete random variable x {0,1,2} follows a Poisson distribution if it has the following probability mass function:

where the parameter is > 0.

1. (2 points) Verify that
2. (2 points) Write the general expression of the log-likelihood of a probability mass function p(x;) with parameters for an iid sample x₁,,x_N.
3. (5 points) Apply it to the above distribution, simplifying it as much as possible.
4. (6 points) Find the maximum likelihood estimate for the parameter .

Exercise 8: multivariate Bernoulli distribution Consider a multivariate Bernoulli distribution where [0,1]^D are the parameters and x {0,1}^D the binary random vector:

.

1. (5 points) Compute the maximum likelihood estimate for given a sample X = {x1,,xN}.

Let us do document classification using a D-word dictionary (element d in xn is 1 if word d is in document n and 0 otherwise) using a multivariate Bernoulli model for each class. Assume we have K document classes for which we have already obtained the values of the optimal parameters _k = (_k1,,_kD)^T and prior distribution p(C_k) = _k, for k = 1,,K, by maximum likelihood.

2. (2 points) Write the discriminant function g_k(x) for a probabilistic classifier in general (not necessarily Bernoulli), and the rule to make a decision.
3. (5 points) Apply it to the multivariate Bernoulli case with K Show that g_k(x) is linear on x, i.e., it can be written as g_k(x) = w_k^T x + w_k0 and give the expression for wk and w_k0.
4. (3 points) Consider K = 2 classes. Show the decision rule can be written as if w^T x + w₀ > 0 then choose class 1, and give the expression for w and w₀.
5. (5 points) Compute the numerical values of w and w₀ for a two-word dictionary where ₁ = 0.7,

) and ). Plot in 2D all the possible values of x {0,1}^D and the boundary

corresponding to this classifier.

Exercise 9: Gaussian classifiers . Consider a binary classification problem for x R^D where we use Gaussian class-conditional probabilities) and

That is, they have the same mean and the covariance matrices are isotropic but different. Compute the expression for the class boundary. What shape is it?