[Solved] ML Homework2-Naive Bayes classifier

1. Naive Bayes classifierCreate a Naive Bayes classifier for each handwritten digit that support discrete and continuous features.Input:1. Training image data from MNISTYou Must download the MNIST from this website and parse the data by yourself. (Please do not use the build in dataset or youll not get 100.) Please read the description in the link to understand the format.Basically, each image is represented by bits (Whole binary file is in big endian format; you need to deal with it), you can use a char arrary to store an image.There are some headers you need to deal with as well, please read the link for more details.2. Training lable data from MNIST.3. Testing image from MNIST4. Testing label from MNIST 5. Toggle option0: discrete mode1: continuous modeTRAINING SET IMAGE FILE (train-images-idx3-ubyte)offset type value description0000 32 bit integer 0x00000803(2051) magic number0004 32 bit integer 60000 number of images0008 32 bit integer 28 number of rows0012 32 bit integer 28 number of columns0016 unsigned byte ?? pixel0017 unsigned byte ?? pixel xxxx unsigned byte ?? pixelTRAINING SET LABEL FILE (train-labels-idx1-ubyte)offset type value description0000 32 bit integer 0x00000801(2049) magic number0004 32 bit integer 60000 number of items0008 unsigned byte ?? label0009 unsigned byte ?? label xxxx unsigned byte ?? labelThe labels values are from 0 to 9.Output:Print out the the posterior (in log scale to avoid underflow) of the ten categories (0-9) for each image in INPUT 3. Dont forget to marginalize them so sum it up will equal to 1.For each test image, print out your prediction which is the category having the highest posterior, and tally the prediction by comparing with INPUT 4.Print out the imagination of numbers in your Bayes classifierFor each digit, print a binary image which 0 represents a white pixel, and 1 represents a black pixel.The pixel is 0 when Bayes classifier expect the pixel in this position should less then 128 in original image, otherwise is 1. Calculate and report the error rate in the end.Function:1. In Discrete mode:Tally the frequency of the values of each pixel into 32 bins. For example, The gray level 0 to 7 should be classified to bin 0, gray level 8 to 15 should be bin 1 etc. Then perform Naive Bayes classifier. Note that to avoid empty bin, you can use a peudocount (such as the minimum value in other bins) for instead.2. In Continuous mode:Use MLE to fit a Gaussian distribution for the value of each pixel. Perform Naive Bayes classifier.

40414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 1 1 1 1 0 0 0 0 0 00 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 00 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 00 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 00 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 00 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 00 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 00 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 00 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 00 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 00 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 00 0 0 0 0 0 0 1 1 1 1 1 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 all other imagination of numbers goes here 9:0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 1 1 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 1 1 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

2. Online learningUse online learning to learn the beta distribution of the parameter p (chance to see 1) of the coin tossing trails in batch.Input:1. A file contains many lines of binary outcomes:

2. parameter a for the initial beta prior3. parameter b for the initial beta priorOutput: Print out the Binomial likelihood (based on MLE, of course), Beta prior and posterior probability (parameters only) for each line.Function: Use Beta-Binomial conjugation to perform online learning.

567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253 case 2: 0110101Likelihood: 0.29375515303997485Beta prior: a = 11 b = 11 Beta posterior: a = 15 b = 14case 3: 010110101101 Likelihood: 0.2286054241794335Beta prior: a = 15 b = 14 Beta posterior: a = 22 b = 19case 4: 0101101011101011010 Likelihood: 0.18286870706509092Beta prior: a = 22 b = 19 Beta posterior: a = 33 b = 27case 5: 111101100011110 Likelihood: 0.2143070548857833Beta prior: a = 33 b = 27 Beta posterior: a = 43 b = 32case 6: 101110111000110 Likelihood: 0.20659760529408Beta prior: a = 43 b = 32 Beta posterior: a = 52 b = 38case 7: 1010010111Likelihood: 0.25082265600000003Beta prior: a = 52 b = 38 Beta posterior: a = 58 b = 42case 8: 11101110110 Likelihood: 0.2619678932864457Beta prior: a = 58 b = 42 Beta posterior: a = 66 b = 45case 9: 01000111101 Likelihood: 0.23609128871506807Beta prior: a = 66 b = 45 Beta posterior: a = 72 b = 50case 10: 110100111Likelihood: 0.27312909617436365Beta prior: a = 72 b = 50 Beta posterior: a = 78 b = 53case 11: 01101010111Likelihood: 0.24384881449471862Beta prior: a = 78 b = 53