Questions tagged [gym]
For questions about OpenAI's gym library, which provides a set of APIs to access different types of environments to train reinforcement learning agents.
81
questions
0
votes
1
answer
16
views
Stable Baselines: 'int' object does not support item assignment
I want to train my Gymnasium model using stable-baselines3 PPO. I have used the structure below before, however for my particular problem, I keep getting the following problem when calling PPO.train():...
- 101
1
vote
1
answer
48
views
What is the right DRL algorithm to use when the goal in an environment is not fixed?
Let's take the LunarLander environment from the package Gym as an example.
In this case, one can run thousand of episodes until the agent learns a good policy. However, there is a condition: the goal ...
- 204
0
votes
0
answers
24
views
How to design a custom openai gym environment to carry out 5G resource slicing?
PROBLEM AT HAND: I have a resource (Bandwidth) of B Hz. I have to distribute the bandwidth B to users as per their requirements. For instance, voice calls would require some amount of bandwidth while ...
1
vote
0
answers
26
views
Why completely two different algorithms are being used in Deep Q Learning?
I'm a new student in reinforcement learning. Recently, I've been studying about different algorithms of RL. But I'm quite surprized that there are some algorithms which are named as "same" ...
0
votes
0
answers
29
views
Parametric noise over Input noise
I came across this research from 2017 that talked about using "Parametric noise" instead of input based noise. I have tried to have it in my PPO based Boid flocking custom environment but ...
- 1
0
votes
1
answer
44
views
RL simulation: Does the Gym-like RL training solution fits the real-world environments?
I am new to the RL community and I am working on projects about using RL to control robots like drones to fly in the scenes. I have been using Nvidia's Issac Gym for some time, but I have a question ...
- 1
0
votes
0
answers
63
views
Reinforcement learning - calculating policy gradient using cross entropy loss
I am writing a program that uses reinforcement learning and the policy gradient method to play Pong. It basically extends Andrej Karpathy's version (https://gist.github.com/karpathy/...
- 1
1
vote
1
answer
111
views
Gymnasium/Petting Zoo: Creating a copy of the board/env
I'm attempting to create a Tic Tac Toe player using MCTS. For the game environment, I'm using Tic Tac Toe from the Gymnasium/Petting Zoo environment.
Running MCTS on Tic Tac Toe requires simulating ...
- 123
2
votes
0
answers
65
views
How does PPO with advantage normalization learn in MountainCar-v0 before first reaching the goal state?
I'm trying to figure out how PPO ever learns anything in a sparse environment like gymnasium's MountainCar-v0 before it first ever reaches the goal state.
Specifically was looking at stable_baselines3'...
- 121
0
votes
0
answers
38
views
DDPG input incompatible in OpenAI Gym custom environment
I have a custom OpenAI Gym Boid flocking environment using StableBAselines3 for DDPG. This error I encounter occurred previously due to wrong input size of actions. But my reset function is correct, ...
- 1
0
votes
0
answers
34
views
Also is there any way to create a gym environment that can be used without registering?
Please see the title: is there any way to create a gym environment that can be used without registering?
To be specific, since we have already created the environment class, is there any way to ...
- 21
0
votes
1
answer
418
views
Error in action space for creating gymnasium environment [closed]
While trying to use a created environment, I get the following error:
...
- 21
1
vote
1
answer
212
views
OpenAI Gym implementation of the delayed rewards
My question is about whether is it possible to implement delayed reward logic within Gym environment.
More specifically, I work on ride-pooling RL algorithm, when the action (choice of the parameters ...
0
votes
0
answers
32
views
Offline CartPole on infinitely long line?
I am tentatively exploring some RL research that involves doing offline RL on a version of the Gymnasium CartPole where the cart can move on $\mathbb R$, as opposed to the standard version (see link) ...
- 121
1
vote
1
answer
77
views
Help for my custom ENV with GYM, trying with DQN
I created a custom env to simulate a sort of blocks using gym enviroments.
My environment consists of an 8 x 8 observation space, which would be the stacks of blocks and the height of each stack.
<...
- 11
0
votes
0
answers
34
views
Deep Reinforcement Learning that takes action from two different sets
I am working on a problem where I want to schedule multiple activities (a1, a2, a3, ... aN) requiring different resource types ...
- 1
0
votes
1
answer
1k
views
My first experience with gym environment has raised many questions, and I need some guidance [closed]
As I'm new to the AI/ML field, I'm still learning from various online materials. In this particular instance, I've been studying the Reinforcement Learning tutorial by deeplizard, specifically ...
- 109
1
vote
1
answer
134
views
What kind of observation state would you give for that environment?
I'm making a new environment where I have two sphere (one above the other) in a 2D plan.
I would like some advice on what observation state I should give to my RL.
Today I have given the following:
...
- 145
0
votes
0
answers
49
views
Using deep reinforcement learning for malware detection; trained agent mostly performs the same action
I'm trying to implement this article:
Ransomware early detection using deep reinforcement learning on portable executable header
The article uses an unpublished dataset of benign and ransomware ...
3
votes
2
answers
577
views
How do you deal with movement inertia in an environment after a step?
I was wondering how can we deal with movement inertia in an environment that is constantly changing?
Imagine that you make a step on an environment that moves a ball. When you make the step, you make ...
- 145
-1
votes
1
answer
276
views
RL framework to optimize my custom multi-agent simulator [closed]
I have built a custom discrete event simulator with multiple agents and want to optimize the system using RL frameworks that support multi-agent configurations. I will use custom policies. Which ...
- 139
0
votes
1
answer
27
views
Can I add additional arguments to my custom Gym Environment? [closed]
I have a custom gym (not gymnasium) environment that I am using for research. I am currently using gym version 0.19.0 installed using conda-forge. Glossing over a lot of details, the agent is learning ...
2
votes
1
answer
4k
views
Should I make my environment with gym or gymnasium?
For context, I am looking to make my own custom Gym environment because I am more interested in trying a bunch of different architectures on this one problem than I am in seeing how a given model ...
- 155
4
votes
1
answer
102
views
Finding the true Q-values in gymnaiusm
I'm very interested in the true Q-values of state-action pairs in the classic control environments in gymnasium. Contrary to the usual goal, the ordering of the Q-values itself is irrelevant; a very ...
- 43
1
vote
1
answer
359
views
Getting always the same action on an A2C from stable_baselines3
I'm quite new to RL and have been trying to train an A2C model from stable_baselines3 to derive an integer sequence based on 3 other input sequences of floats. I have a custom gym environment that ...
- 151
0
votes
1
answer
184
views
How do you define an action space for a card game with an unlimited and variable hand size?
I'm new to the world of AI and have been primarily reading through the documentation for OpenAI's Gym/Gymnasium in hopes of training an AI to play a board game. One piece of information I haven't been ...
- 1
0
votes
0
answers
467
views
Tips for solving OpenAI/Faramas Gymnasium Car Racing Environment
Im quite new to ML and wanna solve Gyms Car Racing v2 using Q-Learning with a Q-Table.
But I am having problems approaching this. Thats why I am hoping someone more advanced in this field could give ...
1
vote
1
answer
386
views
OpeanAI Gym. Train problem: invalid values [closed]
I have a problem with my reinforcement learning model.
I am trying to simulate an electric battery storage. To keep it as simple as possible, the efficiency of charge, storage and discharge are 100%. ...
- 13
1
vote
0
answers
261
views
How to normalize input data to Reinforcement learning platform (Gym and stable-baselines)
I created a custom environment with Gym and trained it with stable baseline 3 algorithms.
The observation and space action are both continues.
The observation space includes 10 values and action space ...
- 11
1
vote
2
answers
338
views
RL solutions for OpenAI Gym environments?
Is there any place where people share their agent's settings for solving OpenAI Gym Environments?
For example, I'd like to know what are good parameters for a DDPG agent to learn the task in Reacher-...
- 41
0
votes
1
answer
734
views
Time taken to solve cartpole environment using DQN
I am trying to solve the cartpole environment (GitHub) using DQN agent. I have been building my own DQN agent by following a tutorial by Jon Krohn.
I am able to solve the environment with a maximum ...
1
vote
2
answers
1k
views
Should PPO always converge toward the global optimum?
I'm trying to "solve" the OpenAI gym environment "Humanoid-v3" using PPO. I got it to work to some degree (The NN is learning a policy and perfecting it. Average reward of about 5....
- 13
3
votes
1
answer
799
views
Why is training longer not better in reinforcement learning?
I have trained an RL agent (PPO) for 6 million steps to solve the OpenAI gym LunarLander-v2. Surprisingly, the agent performs best already after 320K steps and is getting worse after that.
In the ...
- 223
-1
votes
1
answer
789
views
what does the OpenAI ALE/Breakout-RAM-V5 observation return [closed]
I haven't been able to understand the output that OpenAI gym return for observation from this snippet
...
1
vote
0
answers
510
views
What method is better to use for a two-player reinforcement learning environment?
I want to create an RL agent for a mancala-type two-player game as my first actual project in the field. I've already completed the game itself and coded a minimax algorithm.
The question is: how ...
- 11
-1
votes
1
answer
226
views
What does the line of code "self.buffer[-1] = observation" do in this BufferWrapper class for DQN?
So the code is related to using a buffer
...
- 185
2
votes
1
answer
519
views
Delayed state observation or caching action in OpenAI gym. Can it still learn?
I am planning to use OpenAI gym for my experiment in real life.
In my experiment design, by the limits of a real-life scenario, I can only receive the state information or the rewards about 2-3 ...
- 55
1
vote
1
answer
35
views
Where to start with reinforced learning on actions and rewards sampled from slow ongoing real life system
I would like some pointers, possible projects that solve conceptually similar goals, code examples or tutorials.
I am trying to achieve a system that is able to start or stop ventilation of a given ...
- 111
1
vote
0
answers
410
views
Too slow search using MCTS in OpenAI Atari games
I'm recently using Monte Carlo Tree Search in OpenAi Gym Atari, but the result isn't satisfying.
Without render, the game lasts about 180 steps ( env.step() was called this much time ) with random ...
- 11
8
votes
1
answer
4k
views
Deep Q-Learning "catastrophic drop" reasons?
I am implementing some "classical" papers in Model Free RL like DQN, Double DQN, and Double DQN with Prioritized Replay.
Through the various models im running on ...
- 81
0
votes
0
answers
61
views
CartPoleV0 model is not getting trained in even after 1500+ episodes using deep Q-learning
I am new to deep Q learning and trying to train the open AI cartpole_V0 game using deep Q learning. Here is my code:
...
- 101
1
vote
1
answer
296
views
Why does the Atari Gym Amidar environment only move after a certain number of episodes? [closed]
When I try to run Amidar even without RL code, I cannot get the environment to move immediately. It takes about 100 steps before the game actually starts moving. I use the following simple code to ...
- 113
3
votes
0
answers
451
views
How to deal with a moving target in the Lunar Lander environment with DDPG?
I have noticed that DDPG does rather well at solving environments with a static target.
For example, the default of Lunar Lander, the flags do not change position. So the DDPG model learns how to get ...
- 131
0
votes
0
answers
199
views
Open AI Taxi - Agent fails to learn an effective policy
I'm trying to solve the openai gym taxi problem (v3) using deep q learning. I've already had some success with the q-table approach, but for the life of me cannot manage to train a NN to learn a ...
- 103
0
votes
1
answer
166
views
Is is not possible to achieve average reward of more than 20-40 with simple Q-Learning
I have implemented the simple Q-Learning based solution for AI-gym's Cartpole-v0.
However, despite changing hyper-parameters, and rechecking my code, I cannot get an average reward (N-running reward) ...
- 393
4
votes
3
answers
7k
views
How do I get started with multi-agent reinforcement learning?
Is there any tutorial that walks through a multi-agent reinforcement learning implementation (in Python) using libraries such as OpenAI's Gym (for the environment), TF-agents, and stable-baselines-3?
...
- 221
1
vote
0
answers
1k
views
Why does Q-value become negative during training of DQN, while the agent learns to play?
I have implemented a simple version of the DQN algorithm for CartPole-v0. The algorithm works fine, in the sense that achieves the highest possible scores. The ...
- 11
1
vote
0
answers
39
views
Difference in average rewards between taking random actions and following random policies
I wrote two programs that simulated 10000 episodes in gym environment CartPole-v0.
The first program takes random moves in every steps in each episode. The average reward over 10000 episodes is 22....
- 11
1
vote
0
answers
386
views
How to deal with KerasRL DDPG algorithm getting stuck in a local optima?
I am using KerasRL DDPG to try to learn a policy on my own custom environment, but the agent is stuck in a local optima although I am adding the OrnsteinUhlenbeck randomization process. I used the ...
- 36
1
vote
1
answer
1k
views
How do I create a custom gym environment based on an image?
I am trying to create my own gym environment for the A3C algorithm (one implementation is here). The custom environment is a simple login form for any site. I want to create an environment from an ...
- 21