Questions tagged [gym]

For questions about OpenAI's gym library, which provides a set of APIs to access different types of environments to train reinforcement learning agents.

Filter by
Sorted by
Tagged with
1
vote
0answers
29 views

What method is better to use for a two-player reinforcement learning environment?

I want to create an RL agent for a mancala-type two-player game as my first actual project in the field. I've already completed the game itself and coded a minimax algorithm. The question is: how ...
-1
votes
1answer
43 views
1
vote
1answer
73 views

Delayed state observation or caching action in OpenAI gym. Can it still learn?

I am planning to use OpenAI gym for my experiment in real life. In my experiment design, by the limits of a real-life scenario, I can only receive the state information or the rewards about 2-3 ...
1
vote
1answer
31 views

Where to start with reinforced learning on actions and rewards sampled from slow ongoing real life system

I would like some pointers, possible projects that solve conceptually similar goals, code examples or tutorials. I am trying to achieve a system that is able to start or stop ventilation of a given ...
0
votes
0answers
20 views

Reinforcement learning: better performance with a stochastic environment

I modified the cartpole environment with one line to make it stochastic: ...
1
vote
0answers
61 views

Too slow search using MCTS in OpenAI Atari games

I'm recently using Monte Carlo Tree Search in OpenAi Gym Atari, but the result isn't satisfying. Without render, the game lasts about 180 steps ( env.step() was called this much time ) with random ...
0
votes
0answers
34 views

Bridging the gap between simulation and real-world scenarios!

I've got a DRL model that was trained on a simulation at a frame rate of 100fps, after testing it with 100fps it gives good results however when testing it with another frame rate say 50fps it gives a ...
4
votes
1answer
278 views

Deep Q-Learning "catastrophic drop" reasons?

I am implementing some "classical" papers in Model Free RL like DQN, Double DQN, and Double DQN with Prioritized Replay. Through the various models im running on ...
0
votes
0answers
39 views

CartPoleV0 model is not getting trained in even after 1500+ episodes using deep Q-learning

I am new to deep Q learning and trying to train the open AI cartpole_V0 game using deep Q learning. Here is my code: ...
1
vote
1answer
45 views

Why does the Atari Gym Amidar environment only move after a certain number of episodes? [closed]

When I try to run Amidar even without RL code, I cannot get the environment to move immediately. It takes about 100 steps before the game actually starts moving. I use the following simple code to ...
3
votes
0answers
116 views

How to deal with a moving target in the Lunar Lander environment with DDPG?

I have noticed that DDPG does rather well at solving environments with a static target. For example, the default of Lunar Lander, the flags do not change position. So the DDPG model learns how to get ...
0
votes
0answers
60 views

Open AI Taxi - Agent fails to learn an effective policy

I'm trying to solve the openai gym taxi problem (v3) using deep q learning. I've already had some success with the q-table approach, but for the life of me cannot manage to train a NN to learn a ...
0
votes
1answer
39 views

Is is not possible to achieve average reward of more than 20-40 with simple Q-Learning

I have implemented the simple Q-Learning based solution for AI-gym's Cartpole-v0. However, despite changing hyper-parameters, and rechecking my code, I cannot get an average reward (N-running reward) ...
0
votes
0answers
44 views

Deployment of a DeepRL model trained on a custom OpenAI-GYM environment

I developed a custom OpenAI-GYM environment and trained a CDQN model on it, now I am trying to figure out how can I test it not using my gym environment but in production (using real world ...
0
votes
0answers
98 views

How to scale the action in a custom environment with DDPG?

I am trying to implement DDPG in a custom gym environment. The action is the relative allocation of funds between each asset. The action space is a Box with the ...
3
votes
1answer
1k views

How do I get started with multi-agent reinforcement learning?

Is there any tutorial that walks through a multi-agent reinforcement learning implementation (in Python) using libraries such as OpenAI's Gym (for the environment), TF-agents, and stable-baselines-3? ...
1
vote
0answers
233 views

Why does Q-value become negative during training of DQN, while the agent learns to play?

I have implemented a simple version of the DQN algorithm for CartPole-v0. The algorithm works fine, in the sense that achieves the highest possible scores. The ...
1
vote
0answers
31 views

Difference in average rewards between taking random actions and following random policies

I wrote two programs that simulated 10000 episodes in gym environment CartPole-v0. The first program takes random moves in every steps in each episode. The average reward over 10000 episodes is 22....
1
vote
0answers
117 views

How to deal with KerasRL DDPG algorithm getting stuck in a local optima?

I am using KerasRL DDPG to try to learn a policy on my own custom environment, but the agent is stuck in a local optima although I am adding the OrnsteinUhlenbeck randomization process. I used the ...
1
vote
1answer
314 views

How do I create a custom gym environment based on an image?

I am trying to create my own gym environment for the A3C algorithm (one implementation is here). The custom environment is a simple login form for any site. I want to create an environment from an ...
6
votes
1answer
171 views

What are the state-of-the-art results in OpenAI's gym environments?

What are the state-of-the-art results in OpenAI's gym environments? Is there a link to a paper/article that describes them and how these SOTA results were calculated?
5
votes
2answers
357 views

My Deep Q-Learning Network does not learn for OpenAI gym's cartpole problem

I am implementing OpenAI gym's cartpole problem using Deep Q-Learning (DQN). I followed tutorials (video and otherwise) and learned all about it. I implemented a code for myself and I thought it ...
1
vote
1answer
572 views

DDPG doesn't converge for MountainCarContinuous-v0 gym environment

I am trying to implement Deep Deterministic policy gradient algorithm by referring to the paper Continuous Control using Deep Reinforcement Learning on the MountainCarContinuous-v0 gym environment. I ...
2
votes
0answers
72 views

Most of state-action pairs remain unvisited in the q-table

In building my first Q-learning algorithm for OpenAI gym's CartPole problem, many of my states remain unvisited. I believe it is the reason that my agent does not learn. Can I be told of the reasons I ...
1
vote
0answers
91 views

OpenAI gym's CartPole problem system does not learn

My OpenAI CartPole-v0 problem's implementation using basic Q-learning does not learn at all. I am a beginner and have implemented my first ever Q-learning from scratch after learning from tutorials. ...
0
votes
1answer
386 views

How can I change observation states' values in OpenAI gym's cartpole environment?

I am learning with the OpenAI gym's cart pole environment. I want to make the observation states discrete (with small stepsize) and for that purpose, I need to change two of the observations from [$ -\...
3
votes
2answers
659 views

How does an episode end in OpenAI Gym's "MountainCar-v0" environment? [closed]

I am working on OpenAI's "MountainCar-v0" environment. In this environment, each step that an agent takes returns (among other values) the variable named ...
0
votes
1answer
41 views

Should I build an environment from scratch myself or it is not always needed?

I am inspired by the paper Neural Architecture Search with Reinforcement Learning to use reinforcement learning for optimizing a child network (learner). My meta-learner (controller or parent network) ...
1
vote
1answer
339 views

What is a RAM state in the gym's breakout-ram environment?

I have encountered the gym environment and decided to create AI that plays breakout. Here is the link: https://gym.openai.com/envs/Breakout-ram-v0/. The documentation says that the state is ...
1
vote
1answer
319 views

How can I model and solve the Knight Tour problem with reinforcement learning?

I've read about the Knight Tour problem. And I wanted to try to solve it with a reinforcement learning algorithm with OpenAI's gym. So, I want to make a bot that can move on the chess table like the ...
3
votes
1answer
115 views

What should the action space for the card game Crib be?

I'm working on creating an environment for a card game, which the agent chooses to discard certain cards in the first phase of the game, and uses the remaining cards to play with. (The game is Crib if ...
1
vote
1answer
824 views

OpenAI Gym: Multiple actions in one step

I'm trying to design an OpenAI Gym environment in which multiple users/players perform actions over time. It's round based and each user needs to take an action before the round is evaluated and the ...
2
votes
1answer
55 views

How can I implement policy evaluation when reward is tied to an action outcome?

I'm following Stanford reinforcement learning videos on youtube. One of the assignments asks to write code for policy evaluation for Gym's FrozenLake-v0 environment. In the course (and books I have ...
2
votes
2answers
684 views

Should I always start from the same start state in reinforcement learning?

In an episodic training of an RL agent, should I always start from the same initial state or I can start from several valid initial states? For example, in a gym environment, should my ...
2
votes
1answer
117 views

How can I implement the reward function for an 8-DOF robot arm with TRPO?

I need to get an 8-DOF (degrees of freedom) robot arm to move a specified point. I need to implement the TRPO RL code using OpenAI gym. I already have the gazebo environment. But I am unsure of how to ...
2
votes
1answer
281 views

Why isn't my implementation of DQN using TensorFlow on the FrozenWorld environment working?

I am trying to test DQN on FrozenWorld environment in gym using TensorFlow 2.x. The update rule is (off policy) $$Q(s,a) \leftarrow Q(s,a)+\alpha (r+\gamma~ max_{a'}Q(s',a')-Q(s,a))$$ I am using an ...
5
votes
1answer
548 views

How to define an action space when an agent can take multiple sub-actions in a step?

I'm attempting to design an action space in OpenAI's gym and hitting the following roadblock. I've looked at this post which is closely related but subtly different. The environment I'm writing needs ...
2
votes
2answers
284 views

Simulating successful trajectories in Montezuma's Revenge turns out to be unsuccessful

I have written code in OpenAI's gym to simulate a random playing in Montezuma's Revenge where the agent randomly samples actions from the action space and tries to play the game. A success for me is ...
5
votes
1answer
1k views

How powerful is OpenAI's Gym and Universe in board games area? [closed]

I'm a big fan of computer board games and would like to make Python chess/go/shogi/mancala programs. Having heard of reinforcement learning, I decided to look at OpenAI Gym. But first of all, I would ...
0
votes
1answer
48 views

Why is this deep Q agent constantly learning just one action?

I'm trying to implement deep q learning in the OpenAI's gym "Taxi-v3" environment. But my agent only learns to do one action in every state. What am I doing wrong? Here is the Github repository with ...
1
vote
1answer
415 views

Are there OpenAI Gym continuing environments (other than inverted pendulum) and baselines?

I would like to use OpenAI Gym to solve a continuing environment, that is, a problem with a single, never-ending episode (please note I don't mean a continuous environment with continuous state and ...
4
votes
0answers
73 views

Unable to train Coach for Banana-v0 Gym environment

I have just started playing with Reinforcement learning and starting from the basics I'm trying to figure out how to solve Banana Gym with coach. Essentially ...
2
votes
3answers
2k views

Has anyone been able to solve OpenAI's hardcore bipedal walker with their implementation of DDPG?

As the question suggests, I'm trying to see if I can solve OpenAI's hardcore version of their gym's bipedal walker using OpenAI's DDPG algorithm. Below is a performance graph from my latest attempt, ...
2
votes
0answers
101 views

Is it possible to use Reward Function of type R(s, a, s') if more than one action is applied?

I am applying a reinforcement learning agent (PPO2, stable baselines implementation) to a custom built environment using OpenAI Gym. One reward function (formualted as loss function, that is, all ...
1
vote
1answer
431 views

OpenAI Gym interface when reward calculation is delayed? (continuous control with considerable reaction time)

I'm about to create an OpenAI Gym environment for a flight simulator. I'm wondering, how to cope with the fact, that the result and reward for some action needs a considerable time to advance through ...
3
votes
1answer
363 views

What is the difference between A2C and running an agent in an environment vector?

I've implemented A2C. I'm now wondering why would we have multiple actors walk around the environment and gather rewards, why not just have a single agent run in an environment vector? I personally ...
1
vote
1answer
159 views

Should I ignore the actions RIGHTFIRE and LEFTFIRE in the SpaceInvaders environment? [closed]

I'm trying to replicate the DeepMind DQN paper. I'm using OpenAI's Gym. I'm trying to get a decent score with Space Invaders (using SpaceInvaders-v4 environment). I ...
0
votes
2answers
160 views

Is there a way to run other platforms (other than Atari) in an OpenAI's Gym-like environment? [closed]

Is there a way to run C64, Nintendo Gameboy, or other platforms in an OpenAI's Gym-like environment? It seems that we can run only Atari games.
0
votes
1answer
3k views

What is the use of the seed function in the gym Environment 'Pendulum-v0'? [closed]

Why do we use the seed function in the 'Pendulum-v0' environment? https://github.com/openai/gym/blob/master/gym/envs/classic_control/pendulum.py#L25
4
votes
1answer
845 views

What is the mapping between actions and numbers in OpenAI's gym? [closed]

In a gym environment, the action space is often a discrete space, where each action is labeled by an integer. I cannot find a way to figure out the correspondence ...