Questions tagged [gym]
For questions about OpenAI's gym library, which provides a set of APIs to access different types of environments to train reinforcement learning agents.
70
questions
0
votes
0
answers
23
views
Deep Reinforcement Learning that takes action from two different sets
I am working on a problem where I want to schedule multiple activities (a1, a2, a3, ... aN) requiring different resource types ...
- 1
0
votes
1
answer
26
views
My first experience with gym environment has raised many questions, and I need some guidance
As I'm new to the AI/ML field, I'm still learning from various online materials. In this particular instance, I've been studying the Reinforcement Learning tutorial by deeplizard, specifically ...
- 111
1
vote
1
answer
92
views
What kind of observation state would you give for that environment?
I'm making a new environment where I have two sphere (one above the other) in a 2D plan.
I would like some advice on what observation state I should give to my RL.
Today I have given the following:
...
- 135
0
votes
0
answers
41
views
Using deep reinforcement learning for malware detection; trained agent mostly performs the same action
I'm trying to implement this article:
Ransomware early detection using deep reinforcement learning on portable executable header
The article uses an unpublished dataset of benign and ransomware ...
-1
votes
0
answers
38
views
Why my Policy Gradient algo is minimizing the rewards instead of maximizing it?
I am using tensorflow 2.x to implement the REINFORCE algorithm for the Cartpole but instead of maximizing the rewards, it is minimizing the rewards. This algorithm is going to be implemented in my ...
2
votes
2
answers
533
views
How do you deal with movement inertia in an environment after a step?
I was wondering how can we deal with movement inertia in an environment that is constantly changing?
Imagine that you make a step on an environment that moves a ball. When you make the step, you make ...
- 135
0
votes
0
answers
21
views
Changing Gym Environment Mid Training
I am using a custom gym environment for a research project. As my agent solves the environment, I want the task to get progressively harder. Right now, I am doing that like so:
...
-1
votes
1
answer
40
views
RL framework to optimize my custom multi-agent simulator [closed]
I have built a custom discrete event simulator with multiple agents and want to optimize the system using RL frameworks that support multi-agent configurations. I will use custom policies. Which ...
0
votes
1
answer
23
views
Can I add additional arguments to my custom Gym Environment? [closed]
I have a custom gym (not gymnasium) environment that I am using for research. I am currently using gym version 0.19.0 installed using conda-forge. Glossing over a lot of details, the agent is learning ...
1
vote
1
answer
73
views
Should I make my environment with gym or gymnasium?
For context, I am looking to make my own custom Gym environment because I am more interested in trying a bunch of different architectures on this one problem than I am in seeing how a given model ...
- 145
3
votes
1
answer
89
views
Finding the true Q-values in gymnaiusm
I'm very interested in the true Q-values of state-action pairs in the classic control environments in gymnasium. Contrary to the usual goal, the ordering of the Q-values itself is irrelevant; a very ...
- 33
0
votes
1
answer
49
views
Getting always the same action on an A2C from stable_baselines3
I'm quite new to RL and have been trying to train an A2C model from stable_baselines3 to derive an integer sequence based on 3 other input sequences of floats. I have a custom gym environment that ...
- 121
0
votes
1
answer
67
views
How do you define an action space for a card game with an unlimited and variable hand size?
I'm new to the world of AI and have been primarily reading through the documentation for OpenAI's Gym/Gymnasium in hopes of training an AI to play a board game. One piece of information I haven't been ...
- 1
0
votes
0
answers
59
views
Tips for solving OpenAI/Faramas Gymnasium Car Racing Environment
Im quite new to ML and wanna solve Gyms Car Racing v2 using Q-Learning with a Q-Table.
But I am having problems approaching this. Thats why I am hoping someone more advanced in this field could give ...
0
votes
0
answers
53
views
RLLib seems to stop learning at certain point
I work on an AI system for queue management but the learning curve flattens at a certain point and the AI seems to stop learning at all, even if the learning before that point was very good.
My model:
...
- 1
1
vote
1
answer
218
views
OpeanAI Gym. Train problem: invalid values [closed]
I have a problem with my reinforcement learning model.
I am trying to simulate an electric battery storage. To keep it as simple as possible, the efficiency of charge, storage and discharge are 100%. ...
- 13
1
vote
0
answers
174
views
How to normalize input data to Reinforcement learning platform (Gym and stable-baselines)
I created a custom environment with Gym and trained it with stable baseline 3 algorithms.
The observation and space action are both continues.
The observation space includes 10 values and action space ...
- 11
0
votes
0
answers
23
views
Designing a Target Location Environment for DeepRL
I'm trying to make an environment where my agent needs to navigate through a continuous space (using a continuous action space) to get to a target location. Currently, I spawn the agent and the target ...
- 101
1
vote
2
answers
242
views
RL solutions for OpenAI Gym environments?
Is there any place where people share their agent's settings for solving OpenAI Gym Environments?
For example, I'd like to know what are good parameters for a DDPG agent to learn the task in Reacher-...
- 41
0
votes
1
answer
418
views
Time taken to solve cartpole environment using DQN
I am trying to solve the cartpole environment (GitHub) using DQN agent. I have been building my own DQN agent by following a tutorial by Jon Krohn.
I am able to solve the environment with a maximum ...
1
vote
2
answers
835
views
Should PPO always converge toward the global optimum?
I'm trying to "solve" the OpenAI gym environment "Humanoid-v3" using PPO. I got it to work to some degree (The NN is learning a policy and perfecting it. Average reward of about 5....
- 13
2
votes
1
answer
547
views
Why is training longer not better in reinforcement learning?
I have trained an RL agent (PPO) for 6 million steps to solve the OpenAI gym LunarLander-v2. Surprisingly, the agent performs best already after 320K steps and is getting worse after that.
In the ...
- 213
-1
votes
1
answer
632
views
what does the OpenAI ALE/Breakout-RAM-V5 observation return [closed]
I haven't been able to understand the output that OpenAI gym return for observation from this snippet
...
1
vote
0
answers
337
views
What method is better to use for a two-player reinforcement learning environment?
I want to create an RL agent for a mancala-type two-player game as my first actual project in the field. I've already completed the game itself and coded a minimax algorithm.
The question is: how ...
- 11
-1
votes
1
answer
161
views
What does the line of code "self.buffer[-1] = observation" do in this BufferWrapper class for DQN?
So the code is related to using a buffer
...
- 185
2
votes
1
answer
414
views
Delayed state observation or caching action in OpenAI gym. Can it still learn?
I am planning to use OpenAI gym for my experiment in real life.
In my experiment design, by the limits of a real-life scenario, I can only receive the state information or the rewards about 2-3 ...
- 55
1
vote
1
answer
34
views
Where to start with reinforced learning on actions and rewards sampled from slow ongoing real life system
I would like some pointers, possible projects that solve conceptually similar goals, code examples or tutorials.
I am trying to achieve a system that is able to start or stop ventilation of a given ...
- 111
1
vote
0
answers
328
views
Too slow search using MCTS in OpenAI Atari games
I'm recently using Monte Carlo Tree Search in OpenAi Gym Atari, but the result isn't satisfying.
Without render, the game lasts about 180 steps ( env.step() was called this much time ) with random ...
- 11
7
votes
1
answer
3k
views
Deep Q-Learning "catastrophic drop" reasons?
I am implementing some "classical" papers in Model Free RL like DQN, Double DQN, and Double DQN with Prioritized Replay.
Through the various models im running on ...
- 71
0
votes
0
answers
59
views
CartPoleV0 model is not getting trained in even after 1500+ episodes using deep Q-learning
I am new to deep Q learning and trying to train the open AI cartpole_V0 game using deep Q learning. Here is my code:
...
- 101
1
vote
1
answer
249
views
Why does the Atari Gym Amidar environment only move after a certain number of episodes? [closed]
When I try to run Amidar even without RL code, I cannot get the environment to move immediately. It takes about 100 steps before the game actually starts moving. I use the following simple code to ...
- 113
3
votes
0
answers
327
views
How to deal with a moving target in the Lunar Lander environment with DDPG?
I have noticed that DDPG does rather well at solving environments with a static target.
For example, the default of Lunar Lander, the flags do not change position. So the DDPG model learns how to get ...
- 131
0
votes
0
answers
177
views
Open AI Taxi - Agent fails to learn an effective policy
I'm trying to solve the openai gym taxi problem (v3) using deep q learning. I've already had some success with the q-table approach, but for the life of me cannot manage to train a NN to learn a ...
- 103
0
votes
1
answer
145
views
Is is not possible to achieve average reward of more than 20-40 with simple Q-Learning
I have implemented the simple Q-Learning based solution for AI-gym's Cartpole-v0.
However, despite changing hyper-parameters, and rechecking my code, I cannot get an average reward (N-running reward) ...
- 371
4
votes
2
answers
5k
views
How do I get started with multi-agent reinforcement learning?
Is there any tutorial that walks through a multi-agent reinforcement learning implementation (in Python) using libraries such as OpenAI's Gym (for the environment), TF-agents, and stable-baselines-3?
...
- 221
1
vote
0
answers
1k
views
Why does Q-value become negative during training of DQN, while the agent learns to play?
I have implemented a simple version of the DQN algorithm for CartPole-v0. The algorithm works fine, in the sense that achieves the highest possible scores. The ...
- 11
1
vote
0
answers
37
views
Difference in average rewards between taking random actions and following random policies
I wrote two programs that simulated 10000 episodes in gym environment CartPole-v0.
The first program takes random moves in every steps in each episode. The average reward over 10000 episodes is 22....
- 11
1
vote
0
answers
279
views
How to deal with KerasRL DDPG algorithm getting stuck in a local optima?
I am using KerasRL DDPG to try to learn a policy on my own custom environment, but the agent is stuck in a local optima although I am adding the OrnsteinUhlenbeck randomization process. I used the ...
- 36
1
vote
1
answer
864
views
How do I create a custom gym environment based on an image?
I am trying to create my own gym environment for the A3C algorithm (one implementation is here). The custom environment is a simple login form for any site. I want to create an environment from an ...
- 21
7
votes
1
answer
838
views
What are the state-of-the-art results in OpenAI's gym environments?
What are the state-of-the-art results in OpenAI's gym environments? Is there a link to a paper/article that describes them and how these SOTA results were calculated?
- 71
6
votes
2
answers
1k
views
My Deep Q-Learning Network does not learn for OpenAI gym's cartpole problem
I am implementing OpenAI gym's cartpole problem using Deep Q-Learning (DQN). I followed tutorials (video and otherwise) and learned all about it. I implemented a code for myself and I thought it ...
- 371
1
vote
1
answer
2k
views
DDPG doesn't converge for MountainCarContinuous-v0 gym environment
I am trying to implement Deep Deterministic policy gradient algorithm by referring to the paper Continuous Control using Deep Reinforcement Learning on the MountainCarContinuous-v0 gym environment. I ...
- 105
2
votes
0
answers
201
views
Most of state-action pairs remain unvisited in the q-table
In building my first Q-learning algorithm for OpenAI gym's CartPole problem, many of my states remain unvisited. I believe it is the reason that my agent does not learn.
Can I be told of the reasons I ...
- 371
1
vote
0
answers
203
views
OpenAI gym's CartPole problem system does not learn
My OpenAI CartPole-v0 problem's implementation using basic Q-learning does not learn at all.
I am a beginner and have implemented my first ever Q-learning from scratch after learning from tutorials.
...
- 371
0
votes
1
answer
699
views
How can I change observation states' values in OpenAI gym's cartpole environment?
I am learning with the OpenAI gym's cart pole environment.
I want to make the observation states discrete (with small stepsize) and for that purpose, I need to change two of the observations from [$
-\...
- 371
3
votes
2
answers
2k
views
How does an episode end in OpenAI Gym's "MountainCar-v0" environment? [closed]
I am working on OpenAI's "MountainCar-v0" environment. In this environment, each step that an agent takes returns (among other values) the variable named ...
- 371
0
votes
2
answers
56
views
Should I build an environment from scratch myself or it is not always needed?
I am inspired by the paper Neural Architecture Search with Reinforcement Learning to use reinforcement learning for optimizing a child network (learner). My meta-learner (controller or parent network) ...
1
vote
1
answer
738
views
What is a RAM state in the gym's breakout-ram environment?
I have encountered the gym environment and decided to create AI that plays breakout. Here is the link: https://gym.openai.com/envs/Breakout-ram-v0/.
The documentation says that the state is ...
- 862
1
vote
1
answer
664
views
How can I model and solve the Knight Tour problem with reinforcement learning?
I've read about the Knight Tour problem. And I wanted to try to solve it with a reinforcement learning algorithm with OpenAI's gym.
So, I want to make a bot that can move on the chess table like the ...
- 15
3
votes
1
answer
362
views
What should the action space for the card game Crib be?
I'm working on creating an environment for a card game, which the agent chooses to discard certain cards in the first phase of the game, and uses the remaining cards to play with. (The game is Crib if ...
- 33