[SOLVED] Go Assignment 4 AE & RL

Assignment 4 AE & RL
Assignment overview. This assignment is designed to explore more supervised learning in the form of autoencoders, and to practice some of the reinforcement learning principles.
Submission. Submit your answers on Brightspace by Wednesday, Nov 27, 10:00 pm. Questions:
1. [35 marks] Write an autoencoder for the MNIST data set and explore how the reconstruction accuracy is influenced by the compression rate. With a small bottleneck layer of two nodes, plot the representation in this bottleneck layer similar to Fig 4.7 for the classes 0,1,2, and 3.
2. [10 marks] What are the values of the optimal Q(s,a) function for the grid below, assuming that we value diminishing returns with =0.5?
8
6
1
-4

3. [35 marks] A good tutorial on a DQN implementation for the inverted pendulum by Greg Surma is given at https://towardsdatascience.com/cartpole-introduction-to-reinforcement-learning- ed0eb5b58288. Go through this code to understand this implementation.
a) Add a comment line at the appropriate place to explain where the square of the temporal
difference error is calculated.
b) Run the program for a sufficient number of steps (explain how you choose this number) and
determine the average score at this point.
c) Also determine the average score at this point when only random actions are taken.
d) Finally, determine the average score when the exploration is kept at 20%.