make_env = lambda: ptan.common.wrappers.wrap_dqn(gym.make("PongNoFrameskip-v4")) envs = [make_env() for _ in range(NUM_ENVS)]
Here is a code you can look at.
The two above lines of code create multiple environments for the game of Atari Pong with the A2C algorithm.
I understand why it is very useful to have multiple agents working on different instances of the same environment as it is presented in A3C (i.e. an asynchronous version of A2C). However, in the above code, it has a single agent working on different instances of the same environment.
What is the advantage of using more than one environment with a single agent?
class GymEnvVec: def __init__(self, name, n_envs, seed): self.envs = [gym.make(name) for i in range(n_envs)] [env.seed(seed + 10 * i) for i, env in enumerate(self.envs)] def reset(self): return [env.reset() for env in self.envs] def step(self, actions): return list(zip(*[env.step(a) for env, a in zip(self.envs, actions)]))