# Tag Info

### What do the different actions of the OpenAI gym's environment of 'Pong-v0' represent?

You can try the actions yourselves, but if you want another reference, check out the documentation for ALE at GitHub. In particular, 0 means no action, 1 means fire, which is why they don't have an ...
Accepted

### Why did the openai's gym website close?

According to Open AI's Greg Brockman, the Gym website never had a big impact and so was never maintained. This is the reason he gives for shutting down the website. A read only export of the site ...
• 1,146
Accepted

### How powerful is OpenAI's Gym and Universe in board games area?

OpenAI's Gym is a standardised API, useful for reinforcement learning, applied to a range of interesting environments many of which you can then access for free with little effort. It is very simple ...
• 23.8k

### Deep Q-Learning "catastrophic drop" reasons?

This is a case of overfitting the Q function leading to compounding errors when selecting actions. You have been training your neural network as function approximator for too long on the same data ...
• 522
Accepted

### Has anyone been able to solve OpenAI's hardcore bipedal walker with their implementation of DDPG?

You may be very interested to know that there was a bug in the v2 Lidar tracing, making the agent think there were phantom objects, and sometimes intersecting with its own legs: https://github.com/...
• 196

### What do the different actions of the OpenAI gym's environment of 'Pong-v0' represent?

You can try to figure out what exactly does an action do using such script: ...
• 141
Accepted

### My Deep Q-Learning Network does not learn for OpenAI gym's cartpole problem

There is a really small mistake in here that causes the problem: ...
• 349

### How does an episode end in OpenAI Gym's "MountainCar-v0" environment?

To answer your question, the specifics of some of the OpenAI Gym environments can be found on their wiki: The episode ends when you reach 0.5 position, or if 200 iterations are reached. There is a ...
• 928
Accepted

### How does an episode end in OpenAI Gym's "MountainCar-v0" environment?

The episode ends when either the car reaches the goal, or a maximum number of timesteps has passed. By default the episode will terminate after 200 steps. You can customize this with the ...
• 961
Accepted

### OpenAI Gym: Multiple actions in one step

What I was looking for is multi-agent RL, where I have multiple RL agents, each controlling actions of one user. All RL agents/user make an action in each environment step and each get their own ...
• 183

### What do the different actions of the OpenAI gym's environment of 'Pong-v0' represent?

There seems to be no difference between 2 & 4 and 3 & 5. The inconsistency mentioned by Icyblade is due to the mechanics of the Pong environment. "Each action is repeatedly performed for a ...
• 31

### What is the difference between A2C and running an agent in an environment vector?

I believe if you run a single agent in multiple parallel environments many times you will get similar actions in similar states, the reason behind multiple agents is that you will have different ...
• 2,226

### Should I always start from the same start state in reinforcement learning?

It is your choice. This can even be different between training and target system. The approach called "exploring starts" chooses a random start state (and action if you are assessing a deterministic ...
• 23.8k
Accepted

### Should I always start from the same start state in reinforcement learning?

It depends on the task the agent is trying to learn and of course on the environment constrains. In an Atari game agents have a pre-fixed starting point because that's part of the games rules, so I ...
• 3,513
Accepted

### How to define an action space when an agent can take multiple sub-actions in a step?

One way to handle an arbitrarily large sequence is by adding a STOP signal as one possible token in the sequence, just like LSTM. So you could divide your game in turns: What you now call a single ...

### Simulating successful trajectories in Montezuma's Revenge turns out to be unsuccessful

Since the environment has some randomness in it, purely memorizing a trajectory to victory will not work. You will have to memorize every single trajectory for that to work, and there are an infinite ...

### Simulating successful trajectories in Montezuma's Revenge turns out to be unsuccessful

What's exactly the point of time.sleep() in this code? I don't really understand it, you're simply stopping the execution of the program for $0.01$ seconds, how ...
• 2,226
Accepted

### What is the mapping between actions and numbers in OpenAI's gym?

There is no way to tell via the Gym API, and to any RL-based learning agent this is entirely unimportant, discrete actions are just arbitrary labels, and their effects are learned by trial and error ...
• 23.8k
Accepted

### What is a RAM state in the gym's breakout-ram environment?

Yes, it is the state of the memory; this would mainly involve variables, since the code would be in ROM. Since it is only 128 bytes in size, the screen memory would also not be included in this. The ...
• 5,062

### How do I get started with multi-agent reinforcement learning?

After checking the Internet, you will probably find several resources such as https://github.com/mohammadasghari/dqn-multi-agent-rl https://rlss.inria.fr/files/2019/07/RLSS_Multiagent.pdf https://...
• 164

### Why is training longer not better in reinforcement learning?

Could it be due to catastrophic forgetting/interference? If once the agent reaches 320K steps it becomes good at the task, it might start to experience only success. This could cause the agent to ...
1 vote

### How do I create a custom gym environment based on an image?

The question is conceptually wrong, because of misunderstanding of area. Explanation: The idea is to replace open ai gym by something different. For example: web-site or computer game. There is no way ...
• 21
1 vote
Accepted

### DDPG doesn't converge for MountainCarContinuous-v0 gym environment

I had to change the actions selection function for this and tune some hyper-parameters. Here's what I did to make it converge: Sampled the noise from a standard normal distribution instead of ...
1 vote

### How can I change observation states' values in OpenAI gym's cartpole environment?

I don't recommend changing the rules of the environment. What you could do: Perform a method called bucketing i.e. take a value from a continuous state space see which discrete bucket it should go ...
• 320
1 vote

### Should I build an environment from scratch myself or it is not always needed?

I guess it would always be better if you can reuse existing environments to make it work for yourself. Since most of the environment codes is anyway opensourced, you can always edit it to your liking. ...
• 313
1 vote
Accepted

### How can I model and solve the Knight Tour problem with reinforcement learning?

Model your problem as an MDP To solve a problem with reinforcement learning, you need to model your problem as a Markov decision process (MDP), so you need to define the state space, the action ...
• 33.8k
1 vote
Accepted

### What should the action space for the card game Crib be?

I've actually implemented this game before using deep reinforcement learning. You are dealing with a dynamic action space here, where the action space may change at each time step of the game (or more ...
• 928
1 vote
Accepted

### How can I implement policy evaluation when reward is tied to an action outcome?

The renowned book Reinforcement Learning: An Introduction (2nd edition), by Sutton and Barto, provides a different update rule than your first update rule for policy evaluation. Their update rule is ...
• 33.8k
1 vote
Accepted

### Why is this deep Q agent constantly learning just one action?

I thought about my input-layer. I had the 500 states one hot encoded. So 499 of every input node would be 0. And 0 is very bad in an neural network. I tried the same code with the "CardPole-v0" and it ...
• 99
1 vote

### Are there OpenAI Gym continuing environments (other than inverted pendulum) and baselines?

I'm not sure why you need a continuing environment, but actually you can make most (if not all) OpenAI Gym environments continuing. When you perform a step, you receive information about the next ...
• 395

Only top scored, non community-wiki answers of a minimum length are eligible