Questions tagged [environment]

For questions related to the concept of environment in reinforcement learning and other AI sub-fields.

Filter by
Sorted by
Tagged with
10
votes
3answers
1k views

Is the optimal policy always stochastic if the environment is also stochastic?

Is the optimal policy always stochastic (that is, a map from states to a probability distribution over actions) if the environment is also stochastic? Intuitively, if the environment is ...
7
votes
1answer
1k views

How does Q-learning work in stochastic environments?

The Q function uses the (current and future) states to determine the action that gets the highest reward. However, in a stochastic environment, the current action (at the current state) does not ...
6
votes
1answer
143 views

Benchmarks for reinforcement learning in discrete MDPs

To compare the performance of various algorithms for perfect information games, reasonable benchmarks include reversi and m,n,k-games (generalized tic-tac-toe). For imperfect information games, ...
6
votes
1answer
105 views

Interesting examples of discrete stochastic games

SGs are a generalization of MDPs to multiple agents. Like this previous question on MDPs, are there any interesting examples of zero-sum, discrete SGs—preferably with small state and action spaces? I'...
5
votes
2answers
71 views

Perfect Information and Imperfect Information Games

In perfect information games, the agent can see all the moves performed in the past. Besides, it can observe the next action that will be put into practice by the opponent. In this case, can we say ...
5
votes
2answers
411 views

How can a neural network work with continuous time?

I have an ANN model that receives an input and produces an output. The output is an action that interacts with the environment and changes the input accordingly. The network has a desired environment ...
5
votes
0answers
82 views

Are all fully observable environments episodic?

According to the definition of a fully observable environment in Russell & Norvig, AIMA (2nd ed), pages 41-44, an environment is only fully observable if it requires zero memory for an agent to ...
4
votes
1answer
2k views

How to create a custom environment for reinforcement learning

I am a newbie in reinforcement learning working on a college project. The project is related to optimizing the hardware power. I am running proprietary software in Linux distribution (16.04). The goal ...
4
votes
1answer
80 views

How to assign rewards in a non-Markovian environment?

I am quite new to the Reinforcement Learning domain and I am curious about something. It seems to be the case that the majority of current research assumes Markovian environments, that is, future ...
4
votes
1answer
51 views

How to represent players in a multi agent environment so each model can distinguish its own player

So I have 2 models trained with the DQN algorithm that I want to train in a multi-agent environment to see how they react with each other. The models were trained in an environment consisting of 0's ...
3
votes
1answer
94 views

What is the advantage of using more than one environment with the advantage actor-critic?

make_env = lambda: ptan.common.wrappers.wrap_dqn(gym.make("PongNoFrameskip-v4")) envs = [make_env() for _ in range(NUM_ENVS)] Here is a code you can look at. ...
2
votes
1answer
197 views

Is there a way to train an RL agent without any environment?

Following Deep Q-learning from Demonstrations, I'd like to avoid potentially unsafe behavior during early learning by making use of supervised learning with demonstration data. However, the ...
2
votes
1answer
49 views

What should the action space for the card game Crib be?

I'm working on creating an environment for a card game, which the agent chooses to discard certain cards in the first phase of the game, and uses the remaining cards to play with. (The game is Crib if ...
2
votes
2answers
60 views

Can non-Markov environments also be deterministic?

The definition of deterministic environment I am familiar with goes as follows: The next state of the agent depends only on the current state and the action chosen by the agent. By exclusion, ...
2
votes
0answers
43 views

How to formulate normalization/probability conditions on state-action spaces in Gym?

I intend to develop a custom environment for open-ai's gym. My goal is for an agent to learn (among additional objectives) dividing a certain quantity drawn from a continous action space (i.e. spaces....
2
votes
0answers
35 views

Should the RL agent be trained in an environment with real-world data or with a synthetic model?

I want to train a reinforcement learning agent in an environment with parameters (for example, the wind speed, sun irradiation, etc.) that change over time. I have recorded a limited amount of data ...
2
votes
1answer
86 views

Can an RL algorithm trained in one environment be successful in a different one?

Can an RL algorithm trained in one environment be successful in a different one? For example, if I train a model to go through one labyrinth, could this model also go through a different but similar ...
1
vote
1answer
27 views

What are episodic and non-episodic domains in reinforcement learning?

I was reading about the temporal difference (TD) learning and I read that: TD handles continuing, non-episodic domains Assuming that continuing means non-terminating, what does non-episodic or ...
1
vote
1answer
413 views

Advice on creating a new environment using OpenAI Gym

I'm looking for some general advice here before I dive in. I'm interested in creating a new environment for OpenAI gym to provide some slightly more challenging continuous control problems than the ...
1
vote
1answer
713 views

What is the use of the seed function in the gym Environment 'Pendulum-v0'?

Why do we use the seed function in the 'Pendulum-v0' environment? https://github.com/openai/gym/blob/master/gym/envs/classic_control/pendulum.py#L25
1
vote
0answers
27 views

Do we need multiple parallel environments to train in batches an on-policy algorithm?

When using an on-policy method in reinforcement learning, like advantage actor-critic, you shouldn't use old data from an experience buffer, since a new policy requires new data. Does this mean that ...
1
vote
1answer
24 views

How to deal with the addition of a new state to the environment during training?

Let's say we have a dynamic environment: a new state gets added after 2000 episodes have been done. So, we leave room for exploration, so that it can discover the new state. When it gets to that new ...
1
vote
1answer
44 views

How do you know if an agent has learnt its environment in reinforcement learning?

I'm new to reinforcement learning and trying to understand it. If you train an agent using a reinforcement learning algorithm (discrete or continuous) on an environment (real or simulated), then how ...
1
vote
0answers
16 views

What if the rewards induced by an environment are related to the policy too?

Assume we have a policy $\pi_{\theta}$ in a classic reinforcement learning setting, and a reward function $R^{\pi}(s,a)$ that changes as long as $\pi$ changes i.e. not only is it predefined by the ...
1
vote
0answers
18 views

Why does this tutorial on reinforced learning not check whether the environment is 'game over' during training?

I am following the tutorial Train a Deep Q Network with TF-Agents. It uses the hello world environment of reinforced learning: cart pole. At the end, the agent is ...
1
vote
1answer
32 views

How to let an RL Agent move the mouse?

Gday guys, I'am building a game enviroment (picture) where an agent should position the mouse on the screen (cords upper right corner) and then click to shoot a canonball. If the goal (left) is hit. ...
1
vote
0answers
40 views

How do I know if the assumption of a static environment is made?

An important property of a reinforcement learning problem is whether the environment of the agent is static, which means that nothing changes if the agent remains inactive. Different learning methods ...
1
vote
0answers
29 views

What are a list of board game environments for RL practice?

Recently OpenAI removes their board game environments. (It may be possible to install an older version to get access to them, but I haven’t downgraded). Is there a list of repositories or resources ...
1
vote
0answers
238 views

Difficulty in balancing Pendulum using Deep Reinforcement Learning Algorithm

I am using OpenAI Gym framework for reinforcement learning where I am trying solve classic control problem of balancing an Inverted Pendulum, which is similar to the "Pendulum-v0" with some changes in ...
0
votes
1answer
20 views

Should I build an environment from scratch myself or it is not always needed?

I am inspired by the paper Neural Architecture Search with Reinforcement Learning to use reinforcement learning for optimizing a child network (learner). My meta-learner (controller or parent network) ...
0
votes
1answer
102 views

Once the environments are vectorized, how do I have to gather immediate experiences for the agent?

My main purpose right now is to train an agent using the A2C algorithm to solve the Atari Breakout game. So far I have succeeded to create that code with a single agent and environment. To break the ...
0
votes
0answers
31 views

OpenAI Gym: Multiple actions in one step

I'm trying to design an OpenAI Gym environment in which multiple users/players perform actions over time. It's round based and each user needs to take an action before the round is evaluated and the ...
-1
votes
1answer
53 views

can i use machine or deep learning in windows instead of Ubuntu? [closed]

I'm new to Machine Learning and want to start to learn it .. Does windows environment be good or i must use Ubuntu