[Solved] CSE 802 Pattern Recognition and Analysis-Homework2

2. Consider a set of 1-dimensional feature values (i.e., points) pertaining to a class that is available here.
   • [3 points] Plot the histogram of these points using a bin size of 2, i.e., each bin should have a range of 2.
   • [4 points] Compute and report the mean and the biased variance of these points.
   • [3 points] Assuming that the given points are generated by an underlying Gaussian distribution, plot the pdf function on the same graph as (a).
3. Consider the three-dimensional normal distribution p(x) N(,), where = (1,1,1)^t and =

1 0 0

0 5 2.

0 2 5

• [2 points] Compute and report the determinant of the covariance matrix, i.e., | |.
• [2 points] Compute and report the inverse of the covariance matrix, i.e., ¹.
• [3 points] Compute and report the eigen-vectors and eigen-values of the covariance matrix.
• [3 points] Compute and report the probability density at (0,0,0)^t and at (5,5,5)^t.
• [2 points] Compute the Euclidean Distance between and the point (5,5,5)^t.
• [3 points] Compute the Mahalanobis Distance between and the point (5,5,5)^t.

3. [10 points] Suppose we have two normal distributions, corresponding to two classes (₁ and ₂) with the same covariances but different means: N(₁,) and N(₂,). In terms of their prior probabilities P(₁) and P(₂), state the condition that the Bayes decision boundary not pass between the two means.
4. [10 points] Consider a two-class problem with the following class-conditional probability density functions (pdfs): p(x | ₁) N(0,²)

and p(x | ₂) N(1,²).

Show that the threshold, , corresponding to the Bayes decision boundary is:

= 2ln₂₁12_PP(⁽₁2)),

where we have assumed that ₁₁ =₂₂ = 0.

5. [15 points] Consider a two-category classification problem involving one feature, x. Let both classconditional densities conform to a Cauchy distribution as follows:

1 1 p(x | ⁱ⁾⁼ _b.₁+ xbai 2, i = 1,2.

Assume a 0-1 loss function and equal priors.

• Compute the Bayes decision boundary.
• Show that the probability of misclassification according to the Bayes decision rule is

1 1

P(error)= 2 tan1a22ba1.

• Plot P(error) as a function of |a₂ a₁|/b.
• What is the maximum value of P(error). Under what conditions will this occur?

6. [10 points] Consider the following class-conditional densities for a three-class problem involving twodimensional features:

p(x|₁) N (1,1)^t,I; p(x|₂) N (1,1)^t,I;

p.

(Here, class ₃ conforms to a Gaussian Mixture Model (GMM) with two components one component is N (0.5,0.5)^t,I and the other component is N whose weights are equal (i.e., )).

• In a 2D graph, mark the mean of ₁, ₂, and the two components of ₃. In the same graph, mark the point x =(0.1,0.1)^t.
• Assuming a 0-1 loss function and equal priors, determine the class to which you will assign the two-dimensional point x =(0.1,0.1)^t based on the Bayes decision rule.

7. Consider the following bivariate density function for the random variable x:

0 20 10

p^original(x) N 0 , 10 30 .

• [5 points] What is the whitening transform, A_w, of x? (You can use matlab (or any other programming language) to compute the eigenvectors/values).
• [3points]Whenthewhiteningtransformationisappliedto x, whatisthedensityfunction, p_{transf orm} of the resulting random variable?
• [5 points] Generate 10,000 bivariate random patterns from p_original (if you are using matlab, then the mvnrnd function can be used to generate these patterns). Plot these patterns in a graph.
• [5 points] Apply the whitening transform, A_w, to the 10,000 bivariate patterns generated above. Plot the transformed patterns in a separate graph.
• [2 points] Compare the patterns in 7c and 7d. What do you observe?.

8. Consider a two-category (₁ and ₂) classification problem with equal priors. Each feature is a twodimensional vector x = (x₁, x₂)^t. The class-conditional densities are:

p(x|₁) N(₁ =(0,0)^t,₁ = 2I), p(x|₂) N(₂ =(2,2)^t,₂ = I).

• [7 points] Compute the Bayes decision rule and the Bayes decision boundary.
• [3 points] Generate 10,000 bivariate random patterns from each of the two densities (if you are using matlab, then the mvnrnd function can be used to generate these patterns). Plot these patterns in a graph using different markers to distinguish the two classes. On the same graph, plot the Bayes decision boundary.
• [5 points] Compute and report the empirical error rate and the confusion matrix when classifying these 10,000 patterns using the Bayes decision policy.

9. Consider a two-category classification problem involving two-dimensional feature vectors of the form x = (x₁, x₂)^t. The two categories are ₁ and ₂, and

p(x | ₁) N (1,1)^t,I, p(x | ₂) N (1,1)^t,I,

P.

• [10 points] Calculate the Bayes decision boundary and write down the Bayes decision rule assuming a 0-1 loss function. The Bayes decision rule has to be written in terms of the Bayes decision boundary.
• [5 points] What are the Bhattacharyya and Chernoff theoretical bounds on the probability of misclassification, P(error)?
• [5 points] Generate n = 25 test patterns from each of the two class-conditional densities and plot them in a two-dimensional feature space using different markers for the two categories (if you are using matlab, then the mvnrnd function can be used to generate these patterns). Draw the Bayes decision boundary on this plot for visualization purposes.
• [5 points] What is the confusion matrix and empirical error rate when classifying the generated patterns using the Bayes decision rule?
• [5 points] Can the empirical error rate exceed the theoretical bounds on the probability of misclassification? Why or why not? (Hint: Compute the empirical error rate multiple times by repeatedly generating n = 25 test patterns from each of the two class-conditional densities.)

10. [15 points] Consider a 1-dimensional classification problem involving two categories ₁ and ₂ such that P(₁) = 2/3 and P(₂) = 1/ Assume that the classification process can result in one of three actions:

₁ choose ₁; ₂ choose ₂; ₃ do not classify.

Consider the following loss function, :

(₁|₁)=(₂|₂)= 0; (₂|₁)=(₁|₂)= 1; (₃|₁)=(₃|₂)= 1/4.

For a given feature value x, assume that p(x|₁)= ²₂^x and p(x|₂)= 1/2. Here, 0 x 2.

Based on the Bayes minimum risk rule, what action will be undertaken when encountering the value x = 0.5?