Phase 5: DeepQ with Segmentation

Introduction

This experimentation discussed in the preceding phases led us to a solution that can finish the first level of Sonic and generalizes fairly well to unseen environments. This solution involves generating a synthetic dataset of images from the game to train a DeepLab V3 semantic segmentation model. We then apply this trained model to the Sonic emulator as a preprocessing step to feed segmented images into a Deep Q Learning Agent. The Deep Q Learning Agent was then on several levels of Sonic to develop the optimal policy of state-action pairs to support generalization on unseen environments.

DeepQ with Image Segmentation

Once the Image Segmentation Model is trained, it is added to the beginning of the DeepQ Model. During inference, the game image is passed through the Image Segmentation Model which classifies each pixel and outputs the segmented image. The output image has the same width and height as the input image put only has 1 channel. This channel contains the classifications of each pixel. This image is easier to make decisions on because the image is simplified into what each pixel is. Redundant and noisy pixels are cleaned up so that DeepQ only needs to see what's relevant for making decisions. But this only mitigates the “Curse of Dimensionality” slightly. The segmented image is still large and therefore requires a smaller but still large DeepQ model to extract meaningful information necessary for decision making. The real benefit of image segmentation is that the image can be made smaller without losing significant information. Most adjacent pixels are the same anyway, so shrinking the image will result in an image very similar to the original. This results in a smaller DeepQ model with fewer nodes and fewer layers. This smaller DeepQ model can be trained many times faster without reducing the total model's overall potential.