I have written code in OpenAI's gym to simulate a random playing in Montezuma's Revenge where the agent randomly samples actions from the action space and tries to play the game. A success for me is defined as the case when the agent is atleast able to successfully retrieve the key (Gets a reward of 100). And such cases I dump in a pickle file. I got 44 successful cases when I kept it to run for a day or so. Here is the code I use to generate the training set :
import numpy import gym import cma import random import time import pickle as pkl env = gym.make('MontezumaRevenge-ram-v0') observation = env.reset() #print(observation) #print(env.action_space.sample()) obs_dict =  action_dict =  success_ctr = 0 for i in range(0, 1000000): print('Reward for episode',i+1) done = False rew = 0 action_list =  obs_list =  while not done: action = env.action_space.sample() observation, reward, done, _ = env.step(action) action_list.append(action) obs_list.append(observation) rew += reward env.render() time.sleep(0.01) if done: env.reset() if rew > 0: success_ctr += 1 print(action_list) action_dict.append(action_list) obs_dict.append(obs_list) pkl.dump(obs_dict, open("obslist.pkl", "wb")) pkl.dump(action_dict, open("action.pkl", "wb")) print(rew) time.sleep(1) try: print(obs_dict.shape) except: pass print("Took key:", success_ctr)
I loaded the successful cases from my generated pickle file, and simulated the agent's playing using those exact same cases. But, the agent never receives a reward of 100. I dont understand why this is happening. A little search online suggested it could be because of noise in the game. So, I gave a sleep time, before running each episode. Still, doesn't work. Can someone please explain why is this happening? And suggest a way I could go about generating the training set?