COMP532 Assignment 2 – Deep Reinforcement Learning
For this assignment you will work in groups of 3 students (maximum 4). The assignment aims to design and implement a deep reinforcement learning agent for a video game from OpenAI Universe. You must also include a brief report describing and discussing your solutions to the problems.
o This assignment is worth 15% of the total mark for COMP532
o 20% of the marks will be awarded for importing an OpenAI universe game
o 20% of the marks will be awarded for creating a network
o 10% of the marks will be awarded for connecting of the game to the network
o 30% of the marks will be awarded for implementing the deep reinforcement learning model o 20% of the marks will be awarded for discussing experimental results in the report
Submission Instructions
o Submit all solutions as a single PDF document containing your results, and discussion of the results. Attach the source code for the programming problems as separate files.
o Submit your solution via Vital.
o The deadline for this assignment Friday 12/04/2019, 5:00pm
o Penalties for late submission apply in accordance with departmental policy as set out in the student handbook, which can be found at http://intranet.csc.liv.ac.uk/student/msc-handbook.pdf
and the University Code of Practice on Assessment, found at
https://www.liverpool.ac.uk/media/livacuk/tqsd/code-of-practice-on- assessment/code_of_practice_on_assessment.pdf
1. Requirement and Description
Language and Platform: Python (version 3.5 or above) and Tensorflow (newest version). You may use any libraries available on Python platform, such as numpy, scikit-learn, panda, etc. You may also need to install OpenAI universe (https://github.com/openai/ universe).
Game environment: You can use any game in OpenAI universe.
Learning Task: You need to train a deep reinforcement learning model to play the game
you selected.
DNN Architecture: There are a few existing deep reinforcement models, including
• Deep Q-Network
• Double Q-Learning
• Actor-Critic algorithm
You can find various tutorials and implementations from Github, for example goo.gl/1q6L31.
Task: To train a deep reinforcement agent for a simple video game. Here, the main task is not to design a new algorithm, but to get yourself familiar with the concept of reinforcement learning and understand how the existing methods work.
Submission files: Your submission needs to contain the following two files:
• a package containing your source codes (with the instruction on how to run them) and
• a document explaining your connection to OpenAI universe, your network architectures, your training parameters, and the results. Detailed requirements on the document is given below.
2. Marking Criteria
The assignment is split in a number of steps. Every step gives you some marks. Step 1: Import an OpenAI universe game 20%
As the starting point, you need to import a game. In your report, you need give a snapshot that you have loaded the game successfully.
Step 2: Creating a network 20%
You need to use a deep neural network. In the report, you need to show the network structure, and show in the code how to set up different elements of the deep reinforcement learning agent, e.g., the input, output, hidden layers and reinforcement agent etc.
Step 3: Connection of the game to the network 10%
You will need to explicitly associate the observations, actions, and rewards of the game to the network’s input and output. Clearly identify this part of the code in your document.
Step 4: Deep reinforcement learning model 30%
In this part of your report, describe your deep reinforcement learning model. You need to clearly state which model you are using, the parameters of the model, and how do you train/update the model.
Step 5: Experimental results 20%
You may record a video demo to show what your agent can do. Alternatively, you can describe it in detail within the text.