# Why does the Atari Gym Amidar environment only move after a certain number of episodes? [closed]

When I try to run Amidar even without RL code, I cannot get the environment to move immediately. It takes about 100 steps before the game actually starts moving. I use the following simple code to display some images and print some actions (I always try to do the same action, namely going up):

env = gym.make('Amidar-v0')
env.reset()

for i in range(1000):
action = 2
next_state, reward, terminated, info = env.step(action) # take a random action
print(f"Timestep {i}")
print(next_state.shape)
print(reward)
print(action)
print(info)
plt.imshow(next_state)
plt.show()


When running this code, it takes until about step 85 before the environment starts to move. After that, each step, it moves until the agent is hit by the enemy. Then the environment restarts in the start state, and it takes quite some time before it starts to move again. I have tried doing 'FIRE' as my first action; however, this is not working since it also takes a while before the environment starts moving. Because of this, my buffer is almost always filled with the same images and hence my network isn't learning anything. How do I get this environment to move immediately?

• Hi @Lennart and welcome to AI Stack Exchange! It looks like this question was closed because it is off-topic. If you would like to keep this question open, you will need to edit it to meet the on-topic rules discussed here. Just comment here if you would like help in any way with this. I hope to see you around the site! – DeepQZero May 25 at 18:57
• Hi @DeepQZero, thanks for the response. I just read the off-topic rules and it was probably closed because the question is about specific software. No need to keep this question open. – Lennart May 26 at 19:24

I suggest creating a gym wrapper to change the reset function of the environment to produce a different start state. Since the initial 85 frames are not influenced by the agent's actions, these frames are unhelpful and unnecessary for your agent's training. You could consider calling the step function 85 times in your new implementation of the reset function using a random action; then return the resultant state. When calling your newly implemented reset function, the agent's start state will be frame 85 instead of frame 0. As you noted, frame 85 is immediately followed with many frames of meaningful movement, which will provide higher-quality training data. In contrast, the original implementation of the reset function starts at frame 0, which is followed by 85 unhelpful fixed frames.