When I try to run Amidar even without RL code, I cannot get the environment to move immediately. It takes about 100 steps before the game actually starts moving. I use the following simple code to display some images and print some actions (I always try to do the same action, namely going up):
env = gym.make('Amidar-v0')
env.reset()
for i in range(1000):
action = 2
next_state, reward, terminated, info = env.step(action) # take a random action
print(f"Timestep {i}")
print(next_state.shape)
print(reward)
print(action)
print(info)
plt.imshow(next_state)
plt.show()
When running this code, it takes until about step 85 before the environment starts to move. After that, each step, it moves until the agent is hit by the enemy. Then the environment restarts in the start state, and it takes quite some time before it starts to move again. I have tried doing 'FIRE' as my first action; however, this is not working since it also takes a while before the environment starts moving. Because of this, my buffer is almost always filled with the same images and hence my network isn't learning anything. How do I get this environment to move immediately?