Replay Agent
Replay Agent is a class that supports development of reinforcement learning agent models by rendering a human-playable action sequence. This agent simply follows a sequence of moves determined by the button presses of a human playing the game at an earlier date. This allows us to start training from various stages in the level to overcome obstacles that are particularly difficult for the agent. In practice, we run the Replay Agent for a determined number of timesteps to encounter the obstacle, then train our Reinforcement Learning agent from that obstacle. This tailors our training for the most difficult cases and leads to optimal model performance.
Replay Agent Code
source/agents/replay_agent.py
1# Various imports as needed with system paths set
2import sys
3import os
4import retro
5
6script_dir = os.path.dirname(os.path.abspath(__file__))
7project_dir = os.path.abspath(script_dir + "/../..")
8
9sys.path.append(os.path.abspath(project_dir + '/source/agents'))
10sys.path.append(os.path.abspath(project_dir + '/source/interface'))
11
12from agent_base import *
13
14class ReplayAgent(AgentBase):
15 def __init__(self, filename='SonicTheHedgehog-Genesis-GreenHillZone.Act1-0000.bk2'):
16 # filename = level.game_name() + '-' + level.to_state() + '-0000.bk2'
17
18 # We need to figure out how to get the proper stage from play.py
19 stage = filename
20 movie_path = f'{project_dir}/source/datasets/contest/' + stage
21 self.movie = retro.Movie(movie_path)
22 self.movie.step()
23 self.moves = []
24 while self.movie.step():
25 keys = []
26 for i in range(12):
27 keys.append(self.movie.get_key(i, 0))
28 self.moves.append(keys)
29 self.i = -1
30
31 def save(self, filename):
32 None # nothing to do here
33
34 def train(self):
35 None # nothing to do here
36
37 def decide(self, obs, info) -> list:
38 self.i += 1
39 return self.moves[self.i]
40
41 # Returns name of agent as a string
42 def name(self) -> str:
43 return "ReplayAgent"
44
45 def to_string(self) -> str:
46 return self.name()