[Solved] CSC7343 Homework1-CNN-VAE

Your tasks in this homework are to explore CNN classification and autoencoders.

Task 1: Implement a CNN with the following structure.

Train and test the network on fashion mnist dataset (train using train data and test using test data in the dataset). Use cross-entropy loss and an optimizer of your choice. Train the network until the loss converges.

1. Collect train and test accuracy about every 10 epochs during the process. Plot the two accuracy numbers against the epochs.

Task 2: Implement a variational autoencoder (VAE) whose encoding part should have the same structure as the above CNN (i.e. 3 conv layers and an encoding of length 256). The decoding part should have a mirror structure of the encoder (i.e., the input/output shapes of a decoding layer should be the inverse of the shapes at the corresponding encoding layer. Use ConvTranspose2D as reverse of Conv2d). You may use squared error or binary cross-entropy as reconstruction loss. Train the VAE with the fashion mnist training data until convergence.

1. Plot the distribution of the 2-norms of the encoding vectors
2. Plot 3-4 original images and the reconstructions side by side to exam how good the reconstructions are.

Task 3: Investigate whether combining VAE with the CNN can improve the CNNs classification. Modify the VAE such that the encoding (the mean) is passed to a softmax layer for classification. Train the joint model with the combined loss: cross-entropy from the supervised component together with reconstruction and KullbackLeibler divergence loss from the AE component. Train the model with fashion mnist data until convergence.

1. Collect train and test accuracy about every 10 epochs during the process. Plot the two accuracy numbers against the epochs
2. Is the test accuracy of the joint model better than the CNN in task 1? If so, why? If not, why not?
3. Alternatively, one can train a VAE first and then replace the decoder with a softmax layer to form a classifier. Afterwards, train the classifier with the labeled data. Try this approach and compare it with the joint model. Which one gives better performance on testing data?