8
votes
What do the different actions of the OpenAI gym's environment of 'Pong-v0' represent?
You can try the actions yourselves, but if you want another reference, check out the documentation for ALE at GitHub.
In particular, 0 means no action, 1 means fire, which is why they don't have an ...
7
votes
Deep Q-Learning "catastrophic drop" reasons?
This is a case of overfitting the Q function leading to compounding errors when selecting actions.
You have been training your policy for too long on the same data distribution.
Overfitting Q ...
- 532
6
votes
Accepted
Why did the openai's gym website close?
According to Open AI's Greg Brockman, the Gym website never had a big impact and so was never maintained. This is the reason he gives for shutting down the website.
A read only export of the site ...
- 1,176
5
votes
Accepted
How powerful is OpenAI's Gym and Universe in board games area?
OpenAI's Gym is a standardised API, useful for reinforcement learning, applied to a range of interesting environments many of which you can then access for free with little effort. It is very simple ...
- 26.2k
5
votes
Accepted
Has anyone been able to solve OpenAI's hardcore bipedal walker with their implementation of DDPG?
You may be very interested to know that there was a bug in the v2 Lidar tracing, making the agent think there were phantom objects, and sometimes intersecting with its own legs:
https://github.com/...
- 206
4
votes
What do the different actions of the OpenAI gym's environment of 'Pong-v0' represent?
You can try to figure out what exactly does an action do using such script:
...
- 141
3
votes
Accepted
My Deep Q-Learning Network does not learn for OpenAI gym's cartpole problem
There is a really small mistake in here that causes the problem:
...
- 359
3
votes
How does an episode end in OpenAI Gym's "MountainCar-v0" environment?
To answer your question, the specifics of some of the OpenAI Gym environments can be found on their wiki:
The episode ends when you reach 0.5 position, or if 200 iterations are reached.
There is a ...
- 1,122
3
votes
Accepted
How does an episode end in OpenAI Gym's "MountainCar-v0" environment?
The episode ends when either the car reaches the goal, or a maximum number of timesteps has passed. By default the episode will terminate after 200 steps. You can customize this with the ...
- 1,071
3
votes
Accepted
OpenAI Gym: Multiple actions in one step
What I was looking for is multi-agent RL, where I have multiple RL agents, each controlling actions of one user. All RL agents/user make an action in each environment step and each get their own ...
- 183
3
votes
What do the different actions of the OpenAI gym's environment of 'Pong-v0' represent?
There seems to be no difference between 2 & 4 and 3 & 5. The inconsistency mentioned by Icyblade is due to the mechanics of the Pong environment.
"Each action is repeatedly performed for a ...
- 31
2
votes
What is the difference between A2C and running an agent in an environment vector?
I believe if you run a single agent in multiple parallel environments many times you will get similar actions in similar states, the reason behind multiple agents is that you will have different ...
- 2,286
2
votes
Should I always start from the same start state in reinforcement learning?
It is your choice.
This can even be different between training and target system. The approach called "exploring starts" chooses a random start state (and action if you are assessing a deterministic ...
- 26.2k
2
votes
Accepted
Should I always start from the same start state in reinforcement learning?
It depends on the task the agent is trying to learn and of course on the environment constrains.
In an Atari game agents have a pre-fixed starting point because that's part of the games rules, so I ...
- 4,753
2
votes
Accepted
How to define an action space when an agent can take multiple sub-actions in a step?
One way to handle an arbitrarily large sequence is by adding a STOP signal as one possible token in the sequence, just like LSTM.
So you could divide your game in turns:
What you now call a single ...
- 824
2
votes
Simulating successful trajectories in Montezuma's Revenge turns out to be unsuccessful
Since the environment has some randomness in it, purely memorizing a trajectory to victory will not work. You will have to memorize every single trajectory for that to work, and there are an infinite ...
2
votes
Simulating successful trajectories in Montezuma's Revenge turns out to be unsuccessful
What's exactly the point of time.sleep() in this code? I don't really understand it, you're simply stopping the execution of the program for $0.01$ seconds, how ...
- 2,286
2
votes
Accepted
What is a RAM state in the gym's breakout-ram environment?
Yes, it is the state of the memory; this would mainly involve variables, since the code would be in ROM. Since it is only 128 bytes in size, the screen memory would also not be included in this.
The ...
- 5,242
2
votes
What are the state-of-the-art results in OpenAI's gym environments?
There is the leaderboard page at the gym GitHub repository that contains links to specific implementations that "solve" the different gym environments, where "to solve" means "...
- 37k
2
votes
Accepted
What is the mapping between actions and numbers in OpenAI's gym?
There is no way to tell via the Gym API, and to any RL-based learning agent this is entirely unimportant, discrete actions are just arbitrary labels, and their effects are learned by trial and error ...
- 26.2k
2
votes
How do I get started with multi-agent reinforcement learning?
After checking the Internet, you will probably find several resources such as
https://github.com/mohammadasghari/dqn-multi-agent-rl
https://rlss.inria.fr/files/2019/07/RLSS_Multiagent.pdf
https://...
- 164
2
votes
Why is training longer not better in reinforcement learning?
Could it be due to catastrophic forgetting/interference? If once the agent reaches 320K steps it becomes good at the task, it might start to experience only success. This could cause the agent to ...
- 21
1
vote
How do I create a custom gym environment based on an image?
The question is conceptually wrong, because of misunderstanding of area. Explanation: The idea is to replace open ai gym by something different. For example: web-site or computer game. There is no way ...
- 21
1
vote
Accepted
DDPG doesn't converge for MountainCarContinuous-v0 gym environment
I had to change the actions selection function for this and tune some hyper-parameters. Here's what I did to make it converge:
Sampled the noise from a standard normal distribution instead of ...
- 105
1
vote
How can I change observation states' values in OpenAI gym's cartpole environment?
I don't recommend changing the rules of the environment.
What you could do:
Perform a method called bucketing i.e. take a value from a continuous state space see which discrete bucket it should go ...
- 320
1
vote
Should I build an environment from scratch myself or it is not always needed?
I guess it would always be better if you can reuse existing environments to make it work for yourself. Since most of the environment codes is anyway opensourced, you can always edit it to your liking.
...
- 313
1
vote
Accepted
How can I model and solve the Knight Tour problem with reinforcement learning?
Model your problem as an MDP
To solve a problem with reinforcement learning, you need to model your problem as a Markov decision process (MDP), so you need to define
the state space,
the action ...
- 37k
1
vote
Accepted
What should the action space for the card game Crib be?
I've actually implemented this game before using deep reinforcement learning. You are dealing with a dynamic action space here, where the action space may change at each time step of the game (or more ...
- 1,122
1
vote
Accepted
How can I implement policy evaluation when reward is tied to an action outcome?
The renowned book Reinforcement Learning: An Introduction (2nd edition), by Sutton and Barto, provides a different update rule than your first update rule for policy evaluation. Their update rule is ...
- 37k
1
vote
Accepted
Why is this deep Q agent constantly learning just one action?
I thought about my input-layer. I had the 500 states one hot encoded. So 499 of every input node would be 0. And 0 is very bad in an neural network. I tried the same code with the "CardPole-v0" and it ...
- 99
Only top scored, non community-wiki answers of a minimum length are eligible