Skip to main content

Questions tagged [gym]

For questions about OpenAI's gym library, which provides a set of APIs to access different types of environments to train reinforcement learning agents.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
21 views

RL PPO Algorithem Doesn't Learn with Fixed Sigma?

I have a problem with my PPO Implementation in Python with Jax. My Agent for example optimizes the ant-v4 environment but only if the sigmas out of my Neural Net of the Policy are be between [1e-6,1]. ...
PC Benutzer's user avatar
1 vote
1 answer
47 views

What is the right DRL algorithm to use when the goal in an environment is not fixed?

Let's take the LunarLander environment from the package Gym as an example. In this case, one can run thousand of episodes until the agent learns a good policy. However, there is a condition: the goal ...
Dave's user avatar
  • 192
0 votes
0 answers
20 views

How to design a custom openai gym environment to carry out 5G resource slicing?

PROBLEM AT HAND: I have a resource (Bandwidth) of B Hz. I have to distribute the bandwidth B to users as per their requirements. For instance, voice calls would require some amount of bandwidth while ...
ANWESA ROY's user avatar
1 vote
0 answers
24 views

Why completely two different algorithms are being used in Deep Q Learning?

I'm a new student in reinforcement learning. Recently, I've been studying about different algorithms of RL. But I'm quite surprized that there are some algorithms which are named as "same" ...
Jahid Chowdhury Choton's user avatar
0 votes
0 answers
26 views

Parametric noise over Input noise

I came across this research from 2017 that talked about using "Parametric noise" instead of input based noise. I have tried to have it in my PPO based Boid flocking custom environment but ...
Hamza's user avatar
  • 1
0 votes
1 answer
40 views

RL simulation: Does the Gym-like RL training solution fits the real-world environments?

I am new to the RL community and I am working on projects about using RL to control robots like drones to fly in the scenes. I have been using Nvidia's Issac Gym for some time, but I have a question ...
bz2000's user avatar
  • 1
0 votes
0 answers
55 views

Reinforcement learning - calculating policy gradient using cross entropy loss

I am writing a program that uses reinforcement learning and the policy gradient method to play Pong. It basically extends Andrej Karpathy's version (https://gist.github.com/karpathy/...
Blato's user avatar
  • 1
1 vote
1 answer
101 views

Gymnasium/Petting Zoo: Creating a copy of the board/env

I'm attempting to create a Tic Tac Toe player using MCTS. For the game environment, I'm using Tic Tac Toe from the Gymnasium/Petting Zoo environment. Running MCTS on Tic Tac Toe requires simulating ...
Ben G's user avatar
  • 123
2 votes
0 answers
60 views

How does PPO with advantage normalization learn in MountainCar-v0 before first reaching the goal state?

I'm trying to figure out how PPO ever learns anything in a sparse environment like gymnasium's MountainCar-v0 before it first ever reaches the goal state. Specifically was looking at stable_baselines3'...
Switch's user avatar
  • 121
0 votes
0 answers
35 views

DDPG input incompatible in OpenAI Gym custom environment

I have a custom OpenAI Gym Boid flocking environment using StableBAselines3 for DDPG. This error I encounter occurred previously due to wrong input size of actions. But my reset function is correct, ...
Hamza's user avatar
  • 1
0 votes
0 answers
33 views

Also is there any way to create a gym environment that can be used without registering?

Please see the title: is there any way to create a gym environment that can be used without registering? To be specific, since we have already created the environment class, is there any way to ...
A J's user avatar
  • 21
0 votes
1 answer
373 views

Error in action space for creating gymnasium environment [closed]

While trying to use a created environment, I get the following error: ...
A J's user avatar
  • 21
1 vote
1 answer
178 views

OpenAI Gym implementation of the delayed rewards

My question is about whether is it possible to implement delayed reward logic within Gym environment. More specifically, I work on ride-pooling RL algorithm, when the action (choice of the parameters ...
Klavdiia Bochenina's user avatar
0 votes
0 answers
29 views

Offline CartPole on infinitely long line?

I am tentatively exploring some RL research that involves doing offline RL on a version of the Gymnasium CartPole where the cart can move on $\mathbb R$, as opposed to the standard version (see link) ...
Novice's user avatar
  • 121
1 vote
1 answer
66 views

Help for my custom ENV with GYM, trying with DQN

I created a custom env to simulate a sort of blocks using gym enviroments. My environment consists of an 8 x 8 observation space, which would be the stacks of blocks and the height of each stack. <...
Betnem's user avatar
  • 11
0 votes
0 answers
34 views

Deep Reinforcement Learning that takes action from two different sets

I am working on a problem where I want to schedule multiple activities (a1, a2, a3, ... aN) requiring different resource types ...
zeeshan's user avatar
0 votes
1 answer
1k views

My first experience with gym environment has raised many questions, and I need some guidance [closed]

As I'm new to the AI/ML field, I'm still learning from various online materials. In this particular instance, I've been studying the Reinforcement Learning tutorial by deeplizard, specifically ...
Boris L.'s user avatar
  • 109
1 vote
1 answer
122 views

What kind of observation state would you give for that environment?

I'm making a new environment where I have two sphere (one above the other) in a 2D plan. I would like some advice on what observation state I should give to my RL. Today I have given the following: ...
CyDevos's user avatar
  • 145
0 votes
0 answers
46 views

Using deep reinforcement learning for malware detection; trained agent mostly performs the same action

I'm trying to implement this article: Ransomware early detection using deep reinforcement learning on portable executable header The article uses an unpublished dataset of benign and ransomware ...
soosan123's user avatar
3 votes
2 answers
577 views

How do you deal with movement inertia in an environment after a step?

I was wondering how can we deal with movement inertia in an environment that is constantly changing? Imagine that you make a step on an environment that moves a ball. When you make the step, you make ...
CyDevos's user avatar
  • 145
0 votes
0 answers
30 views

Changing Gym Environment Mid Training

I am using a custom gym environment for a research project. As my agent solves the environment, I want the task to get progressively harder. Right now, I am doing that like so: ...
user20057611's user avatar
-1 votes
1 answer
257 views

RL framework to optimize my custom multi-agent simulator [closed]

I have built a custom discrete event simulator with multiple agents and want to optimize the system using RL frameworks that support multi-agent configurations. I will use custom policies. Which ...
RookieScientist's user avatar
0 votes
1 answer
27 views

Can I add additional arguments to my custom Gym Environment? [closed]

I have a custom gym (not gymnasium) environment that I am using for research. I am currently using gym version 0.19.0 installed using conda-forge. Glossing over a lot of details, the agent is learning ...
user20057611's user avatar
2 votes
1 answer
4k views

Should I make my environment with gym or gymnasium?

For context, I am looking to make my own custom Gym environment because I am more interested in trying a bunch of different architectures on this one problem than I am in seeing how a given model ...
Justin T's user avatar
  • 155
4 votes
1 answer
102 views

Finding the true Q-values in gymnaiusm

I'm very interested in the true Q-values of state-action pairs in the classic control environments in gymnasium. Contrary to the usual goal, the ordering of the Q-values itself is irrelevant; a very ...
Mark B's user avatar
  • 43
1 vote
1 answer
326 views

Getting always the same action on an A2C from stable_baselines3

I'm quite new to RL and have been trying to train an A2C model from stable_baselines3 to derive an integer sequence based on 3 other input sequences of floats. I have a custom gym environment that ...
Jesuspc's user avatar
  • 151
0 votes
1 answer
169 views

How do you define an action space for a card game with an unlimited and variable hand size?

I'm new to the world of AI and have been primarily reading through the documentation for OpenAI's Gym/Gymnasium in hopes of training an AI to play a board game. One piece of information I haven't been ...
haec0007's user avatar
0 votes
0 answers
442 views

Tips for solving OpenAI/Faramas Gymnasium Car Racing Environment

Im quite new to ML and wanna solve Gyms Car Racing v2 using Q-Learning with a Q-Table. But I am having problems approaching this. Thats why I am hoping someone more advanced in this field could give ...
user72952's user avatar
1 vote
1 answer
365 views

OpeanAI Gym. Train problem: invalid values [closed]

I have a problem with my reinforcement learning model. I am trying to simulate an electric battery storage. To keep it as simple as possible, the efficiency of charge, storage and discharge are 100%. ...
MiPre's user avatar
  • 13
1 vote
0 answers
255 views

How to normalize input data to Reinforcement learning platform (Gym and stable-baselines)

I created a custom environment with Gym and trained it with stable baseline 3 algorithms. The observation and space action are both continues. The observation space includes 10 values and action space ...
Soprano's user avatar
  • 11
1 vote
2 answers
319 views

RL solutions for OpenAI Gym environments?

Is there any place where people share their agent's settings for solving OpenAI Gym Environments? For example, I'd like to know what are good parameters for a DDPG agent to learn the task in Reacher-...
pippo's user avatar
  • 41
0 votes
1 answer
692 views

Time taken to solve cartpole environment using DQN

I am trying to solve the cartpole environment (GitHub) using DQN agent. I have been building my own DQN agent by following a tutorial by Jon Krohn. I am able to solve the environment with a maximum ...
Rishidhar Kasam's user avatar
1 vote
2 answers
1k views

Should PPO always converge toward the global optimum?

I'm trying to "solve" the OpenAI gym environment "Humanoid-v3" using PPO. I got it to work to some degree (The NN is learning a policy and perfecting it. Average reward of about 5....
pjungk's user avatar
  • 13
2 votes
1 answer
774 views

Why is training longer not better in reinforcement learning?

I have trained an RL agent (PPO) for 6 million steps to solve the OpenAI gym LunarLander-v2. Surprisingly, the agent performs best already after 320K steps and is getting worse after that. In the ...
Martin S's user avatar
  • 213
-1 votes
1 answer
769 views

what does the OpenAI ALE/Breakout-RAM-V5 observation return [closed]

I haven't been able to understand the output that OpenAI gym return for observation from this snippet ...
Jirawat Zhou's user avatar
1 vote
0 answers
495 views

What method is better to use for a two-player reinforcement learning environment?

I want to create an RL agent for a mancala-type two-player game as my first actual project in the field. I've already completed the game itself and coded a minimax algorithm. The question is: how ...
JollyOwl's user avatar
-1 votes
1 answer
219 views

What does the line of code "self.buffer[-1] = observation" do in this BufferWrapper class for DQN?

So the code is related to using a buffer ...
user3656142's user avatar
2 votes
1 answer
508 views

Delayed state observation or caching action in OpenAI gym. Can it still learn?

I am planning to use OpenAI gym for my experiment in real life. In my experiment design, by the limits of a real-life scenario, I can only receive the state information or the rewards about 2-3 ...
Ykwk's user avatar
  • 55
1 vote
1 answer
35 views

Where to start with reinforced learning on actions and rewards sampled from slow ongoing real life system

I would like some pointers, possible projects that solve conceptually similar goals, code examples or tutorials. I am trying to achieve a system that is able to start or stop ventilation of a given ...
sanyi's user avatar
  • 111
1 vote
0 answers
401 views

Too slow search using MCTS in OpenAI Atari games

I'm recently using Monte Carlo Tree Search in OpenAi Gym Atari, but the result isn't satisfying. Without render, the game lasts about 180 steps ( env.step() was called this much time ) with random ...
Dibbla's user avatar
  • 11
8 votes
1 answer
4k views

Deep Q-Learning "catastrophic drop" reasons?

I am implementing some "classical" papers in Model Free RL like DQN, Double DQN, and Double DQN with Prioritized Replay. Through the various models im running on ...
Virus's user avatar
  • 81
0 votes
0 answers
61 views

CartPoleV0 model is not getting trained in even after 1500+ episodes using deep Q-learning

I am new to deep Q learning and trying to train the open AI cartpole_V0 game using deep Q learning. Here is my code: ...
UJM's user avatar
  • 101
1 vote
1 answer
295 views

Why does the Atari Gym Amidar environment only move after a certain number of episodes? [closed]

When I try to run Amidar even without RL code, I cannot get the environment to move immediately. It takes about 100 steps before the game actually starts moving. I use the following simple code to ...
Lennart's user avatar
  • 113
3 votes
0 answers
446 views

How to deal with a moving target in the Lunar Lander environment with DDPG?

I have noticed that DDPG does rather well at solving environments with a static target. For example, the default of Lunar Lander, the flags do not change position. So the DDPG model learns how to get ...
user1779362's user avatar
0 votes
0 answers
196 views

Open AI Taxi - Agent fails to learn an effective policy

I'm trying to solve the openai gym taxi problem (v3) using deep q learning. I've already had some success with the q-table approach, but for the life of me cannot manage to train a NN to learn a ...
Sledge's user avatar
  • 103
0 votes
1 answer
166 views

Is is not possible to achieve average reward of more than 20-40 with simple Q-Learning

I have implemented the simple Q-Learning based solution for AI-gym's Cartpole-v0. However, despite changing hyper-parameters, and rechecking my code, I cannot get an average reward (N-running reward) ...
SJa's user avatar
  • 393
4 votes
3 answers
6k views

How do I get started with multi-agent reinforcement learning?

Is there any tutorial that walks through a multi-agent reinforcement learning implementation (in Python) using libraries such as OpenAI's Gym (for the environment), TF-agents, and stable-baselines-3? ...
Rnj's user avatar
  • 221
1 vote
0 answers
1k views

Why does Q-value become negative during training of DQN, while the agent learns to play?

I have implemented a simple version of the DQN algorithm for CartPole-v0. The algorithm works fine, in the sense that achieves the highest possible scores. The ...
Arman's user avatar
  • 11
1 vote
0 answers
39 views

Difference in average rewards between taking random actions and following random policies

I wrote two programs that simulated 10000 episodes in gym environment CartPole-v0. The first program takes random moves in every steps in each episode. The average reward over 10000 episodes is 22....
artils1997's user avatar
1 vote
0 answers
374 views

How to deal with KerasRL DDPG algorithm getting stuck in a local optima?

I am using KerasRL DDPG to try to learn a policy on my own custom environment, but the agent is stuck in a local optima although I am adding the OrnsteinUhlenbeck randomization process. I used the ...
BAKYAC's user avatar
  • 36