8
votes
What do the different actions of the OpenAI gym's environment of 'Pong-v0' represent?
You can try the actions yourselves, but if you want another reference, check out the documentation for ALE at GitHub.
In particular, 0 means no action, 1 means fire, which is why they don't have an ...
8
votes
Deep Q-Learning "catastrophic drop" reasons?
This is a case of overfitting the Q function leading to compounding errors when selecting actions.
You have been training your policy for too long on the same data distribution.
Overfitting Q ...
- 562
6
votes
Accepted
Why did the openai's gym website close?
According to Open AI's Greg Brockman, the Gym website never had a big impact and so was never maintained. This is the reason he gives for shutting down the website.
A read only export of the site ...
- 1,186
6
votes
Accepted
How do you deal with movement inertia in an environment after a step?
The simplest answer is that the inertia (or velocity, if the mass of the ball is not a variable) should be part of the observable state that the agent has access to. In RL there are usually two parts ...
- 32.9k
5
votes
Accepted
How powerful is OpenAI's Gym and Universe in board games area?
OpenAI's Gym is a standardised API, useful for reinforcement learning, applied to a range of interesting environments many of which you can then access for free with little effort. It is very simple ...
- 32.9k
5
votes
Accepted
Has anyone been able to solve OpenAI's hardcore bipedal walker with their implementation of DDPG?
You may be very interested to know that there was a bug in the v2 Lidar tracing, making the agent think there were phantom objects, and sometimes intersecting with its own legs:
https://github.com/...
- 206
5
votes
Accepted
Finding the true Q-values in gymnaiusm
In simple environments, the gold standard for true values to arbitrary accuracy would be to use dynamic programming, either policy iteration or value iteration (to evaluate a fixed policy, then use a ...
- 32.9k
4
votes
What do the different actions of the OpenAI gym's environment of 'Pong-v0' represent?
You can try to figure out what exactly does an action do using such script:
...
- 141
4
votes
How do you deal with movement inertia in an environment after a step?
In RL usually a "discretized" time MDP is used in case of continuous time problems, with the idea that if we evaluate enough time the policy, we can approximate it the continuous case. This ...
- 2,473
3
votes
How does an episode end in OpenAI Gym's "MountainCar-v0" environment?
To answer your question, the specifics of some of the OpenAI Gym environments can be found on their wiki:
The episode ends when you reach 0.5 position, or if 200 iterations are reached.
There is a ...
- 1,568
3
votes
Accepted
How does an episode end in OpenAI Gym's "MountainCar-v0" environment?
The episode ends when either the car reaches the goal, or a maximum number of timesteps has passed. By default the episode will terminate after 200 steps. You can customize this with the ...
- 1,136
3
votes
Accepted
My Deep Q-Learning Network does not learn for OpenAI gym's cartpole problem
There is a really small mistake in here that causes the problem:
...
- 389
3
votes
Accepted
OpenAI Gym: Multiple actions in one step
What I was looking for is multi-agent RL, where I have multiple RL agents, each controlling actions of one user. All RL agents/user make an action in each environment step and each get their own ...
- 183
3
votes
What do the different actions of the OpenAI gym's environment of 'Pong-v0' represent?
There seems to be no difference between 2 & 4 and 3 & 5. The inconsistency mentioned by Icyblade is due to the mechanics of the Pong environment.
"Each action is repeatedly performed for a ...
- 31
3
votes
Accepted
Should I make my environment with gym or gymnasium?
As you correctly pointed out, OpenAI Gym is less supported these days. It makes sense to go with Gymnasium, which is by the way developed by a non-profit organization.
Regarding backwards ...
- 274
3
votes
Accepted
What kind of observation state would you give for that environment?
Provided the discs do not have any properties that vary over their surfaces, and you are not trying to generalise to different sizes/weights of discs for a single agent, then your state variables seem ...
- 32.9k
2
votes
What is the difference between A2C and running an agent in an environment vector?
I believe if you run a single agent in multiple parallel environments many times you will get similar actions in similar states, the reason behind multiple agents is that you will have different ...
- 2,406
2
votes
Accepted
How to define an action space when an agent can take multiple sub-actions in a step?
One way to handle an arbitrarily large sequence is by adding a STOP signal as one possible token in the sequence, just like LSTM.
So you could divide your game in turns:
What you now call a single ...
- 854
2
votes
Simulating successful trajectories in Montezuma's Revenge turns out to be unsuccessful
Since the environment has some randomness in it, purely memorizing a trajectory to victory will not work. You will have to memorize every single trajectory for that to work, and there are an infinite ...
2
votes
Simulating successful trajectories in Montezuma's Revenge turns out to be unsuccessful
What's exactly the point of time.sleep() in this code? I don't really understand it, you're simply stopping the execution of the program for $0.01$ seconds, how ...
- 2,406
2
votes
Should I always start from the same start state in reinforcement learning?
It is your choice.
This can even be different between training and target system. The approach called "exploring starts" chooses a random start state (and action if you are assessing a deterministic ...
- 32.9k
2
votes
Accepted
Should I always start from the same start state in reinforcement learning?
It depends on the task the agent is trying to learn and of course on the environment constrains.
In an Atari game agents have a pre-fixed starting point because that's part of the games rules, so I ...
- 5,378
2
votes
Accepted
What is the mapping between actions and numbers in OpenAI's gym?
There is no way to tell via the Gym API, and to any RL-based learning agent this is entirely unimportant, discrete actions are just arbitrary labels, and their effects are learned by trial and error ...
- 32.9k
2
votes
Accepted
What is a RAM state in the gym's breakout-ram environment?
Yes, it is the state of the memory; this would mainly involve variables, since the code would be in ROM. Since it is only 128 bytes in size, the screen memory would also not be included in this.
The ...
- 5,397
2
votes
Accepted
DDPG doesn't converge for MountainCarContinuous-v0 gym environment
I had to change the actions selection function for this and tune some hyper-parameters. Here's what I did to make it converge:
Sampled the noise from a standard normal distribution instead of ...
- 125
2
votes
What are the state-of-the-art results in OpenAI's gym environments?
There is the leaderboard page at the gym GitHub repository that contains links to specific implementations that "solve" the different gym environments, where "to solve" means "...
- 41.1k
2
votes
How do I get started with multi-agent reinforcement learning?
After checking the Internet, you will probably find several resources such as
https://github.com/mohammadasghari/dqn-multi-agent-rl
https://rlss.inria.fr/files/2019/07/RLSS_Multiagent.pdf
https://...
- 274
2
votes
Why is training longer not better in reinforcement learning?
Could it be due to catastrophic forgetting/interference? If once the agent reaches 320K steps it becomes good at the task, it might start to experience only success. This could cause the agent to ...
- 21
2
votes
OpenAI Gym implementation of the delayed rewards
I think what you have here (with an important caveat, which I will get to later) is a common misunderstanding about how rewards should be structured for a reinforcement learning (RL) problem. It is ...
- 32.9k
1
vote
How can I change observation states' values in OpenAI gym's cartpole environment?
I don't recommend changing the rules of the environment.
What you could do:
Perform a method called bucketing i.e. take a value from a continuous state space see which discrete bucket it should go ...
- 320
Only top scored, non community-wiki answers of a minimum length are eligible