Use this template to record your answers for Homework 5. Add your answers using L^ATEXand then save your document as a PDF to upload to Gradescope. You are required to use this template to submit your answers. You should not alter this template in any way other than to insert your solutions. You must submit all 15 pages of this template to Gradescope. Do not remove the instructions page(s). Altering this template or including your solutions outside of the provided boxes can result in your assignment being graded incorrectly. You may lose points if you do not follow these instructions.

You should also save your code as a .py or .zip file and upload it to the separate Gradescope coding assignment. Remember to mark all teammates on both assignment uploads through Gradescope.

Instructions for Specific Problem Types

On this homework, you must fill in (a) blank(s) for each problem; please make sure your final answer is fully included in the given space. Do not change the size of the box provided. For short answer questions you should not include your work in your solution. Only provide an explanation or proof if specifically asked. Otherwise, your assignment may not be graded correctly, and points may be deducted from your assignment.

Fill in the blank: What is the course number?

Problem 0: Collaborators

Enter your teams names and Andrew IDs in the boxes below. If you do not do this, you may lose points on your assignment.

1.1 Planning with Cross Entropy Method (CEM) [20 pts]Problem 1: Model-based Reinforcement Learning with PETS (80 pts)

1.1.3 MPC implementation; Comparison of MPC in environment w/o noise (10 pts)

1.2 Probabilistic Ensemble and Trajectory Sampling (PETS) [60 pts]

1.2.1 Derive the Loss of a Probabilistic Model (5 pts)

1.2.2 Loss and RMSE of a single dynamic model (5 pts)

1.2.3 Planning on single model with random actions + MPC (5 pts)

1.2.4 Planning on single model with CEM+MPC (10 pts)

1.2.5 Discussion on the comparison between CEM and random actions (5 pts)

1.2.6 Description of the implementation details (5 pts)

1.2.7 Loss and RMSE of the probabilistic ensemble model (10 pts)

1.2.8 Success percentage of CEM+MPC and random actions + MPC (10 pts)

1.2.9 Limitation of MBRL (5 pts)

Problem 2: Theoretical Questions (20 pts)

2.1 Deterministic vs Stochastic Model (10 pts)

2.2 Aleatoric vs Epistemic Uncertainty (5 pts)

2.3 Failure Modes without Considering Uncertainty (5 pts)

Feedback (1pts): You can help the course staff improve the course for future semesters by providing feedback. You will receive a point if you provide actionable feedback for each of the following categories.

What advice would you give to future students working on this homework?

Time Spent (1pt): How many hours did you spend working on this assignment? Your answer will not affect your grade.