So for anyone struggling to understand the OpenAI's Spinning Up educational resource, I'll provide the answer to my question here.
Firstly, it's important to understand that the algorithms expect a 2-dimensional input shape, in rudimentary terms a shape of
Box(int), which isn't the case with the default Breakout-v0 game environment, which supplies inputs in the shape
Box(210, 160, 3), which is the game screen's height, width, and RGB colour space.
The Breakout-ram-v0 game environment, however, DOES provide an input of the appropriate shape, consequently
Box(128,). So switching to this environment solves the initial problem I was having. To train any of the on-policy algorithms (PPO in my case), the command should be:
python -m spinup.run ppo --env Breakout-ram-v0 --exp_name simpletest
To take this a step further, how would you train an off-policy algorithm (such as DDPQ)? Well, the off-policy algorithms expect a continuous action space, which can be seen in the differences in the source code here between MountainCar-v0, and MountainCarContinuous-v0:
self.action_space = spaces.Discrete(3)
self.action_space = spaces.Box(low=self.min_action, high=self.max_action,
So, to sum up, how would you train something like the Lunar Lander game environment with the DDPQ algorithm? Well, you'd need the continuous action space version, and by providing it with 2 dense layers with 192 nodes each, I was able to achieve the following results with just 200 epochs using the following command:
python -m spinup.run ddpg --env LunarLanderContinuous-v2 --exp_name ddpq-test --hid [192,192] --epochs 200
Pretty cool...anyways, I hope this helps someone sometime, and I would thoroughly recommend looking at the source code for the various OpenAI game environments, as well as their table of game environments for a quick reference to the observation and action spaces. Good luck!