Project Overview

This project is an application that plays Sonic the Hedgehog Genesis (Sonic) using artificial intelligence. It is able to generalize to any other game with similar game mechanics and controls. The application executes a decision making algorithm which plays the game by training neural networks to see what the user sees, extract useful information and make decisions to navigate the environment. Our application interfaces with the game using Gym Retro, an open source platform for reinforcement learning training and visualization. The platform solves two very challenging problems: The first is that the game was compiled to run on a processor which is incompatible with modern computers. The second is that its interface was developed to work directly with monitors and game controllers, and not with other programs. This platform allows a program to interface with supported games. Gym Retro runs the game in a builtin emulator as a child process. All this is conveniently encapsulated in an object oriented API made available by the Gym Retro package.

Approach

In order to develop the optimal agent to play Sonic, the team constructed a stable software architecture as a platform for experimentation across implementations of reinforcement learning agents, computer vision image processing techniques, and helper functions to support explainability. This experimentation led us to a solution that can finish the first level of Sonic and generalizes fairly well to unseen environments. This solution involves generating a synthetic dataset of images from the game to train a DeepLab V3 semantic segmentation model. We then apply this trained model to the Sonic emulator as a preprocessing step to feed segmented images into a Deep Q Learning Agent. The Deep Q Learning Agent was then on several levels of Sonic to develop the optimal policy of state-action pairs to support generalization on unseen environments. This final approach is open-source and can be replicated by running the following command line arguments below from the root project directory.

Implementation

The following utilizes the default command line arguments as input, except where specified. Defaults can be found in the driver code files below, or by adding the -h command after the command to view defaults and options for each argument

Install Software Architecture

git clone https://git.cs.vt.edu/dmath010/supersonicai.git

pip install -r requirements.txt

Generate Synthetic Image Dataset

python source/drivers/generate_dataset.py -n 20000

Train Semantic Segmentation Model

python source/drivers/deeplab_train.py -e 45

Train Deep Q Learning Agent

python source/drivers/train.py --agent_type dqn --level 1 --epochs 100  --seg results/deeplab_ckpts/SegmentationModel.pt

Watch your Agent Play

python source/drivers/play.py --seg results/deeplab_ckpts/SegmentationModel.pt --render --level 1