Skip to main content
OverflowAI is here! AI power for your Stack Overflow for Teams knowledge community. Learn more

Questions tagged [observation-spaces]

For questions about observation spaces in the context of reinforcement learning and other AI sub-fields.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

How to clip actions based on state in RL environment for trading (and other tips to approach optimal trade execution)?

I’m currently trying to use RL to analyze price impact in financial markets for optimal trade execution and have coded up a custom gymnasium environment to do so. Now, I'm deciding on which RL ...
walrus's user avatar
  • 1
4 votes
1 answer
166 views

Training an RL model with an environment where some of the variables do not change as a result of the agent actions

Typically training an RL model requires an action and an observation space, and the agent learns how its actions affect the observations. Even though there are cases where the observation space ...
Jesuspc's user avatar
  • 151
0 votes
1 answer
30 views

What are the differences between loss surfaces that "derive"from different observations?

If I understand right that each observation whithin a dataset, creates a different loss surface where we want to find the global minimum. How different those surfaces one from another? Would it be ...
Igor's user avatar
  • 181
2 votes
1 answer
327 views

Variable observation space at each episode

I have an enviroment with continuous actions and state variables. Every time I reset my env, between 2 and 5 balls spawn randomly in a box of 100x100 size. One of those balls (the red one) will ...
Optical_flow_lover's user avatar
2 votes
0 answers
167 views

What to look out for when designing an environment regarding observations?

When designing an environment, what should one look out for when designing the observation space to make the environment as easy to be learnable for an agent as possible? E.g. make sure the markov ...
kitaird's user avatar
  • 115
0 votes
1 answer
324 views

When should discretization of observations be considered?

I found some literature regarding the design of action-spaces and that e.g. a discretization of continuous actions in video-game environments can be crucial for successful learning (Action Space ...
kitaird's user avatar
  • 115
4 votes
1 answer
397 views

Are multi agent or self-play environments always automatically POMDPs?

As part of my thesis, I'm working on a zero sum game with RL to train an agent. The game is a real-time game, a derivation of pong, one could imagine playing pong with both sides being foosball rods. ...
kitaird's user avatar
  • 115
3 votes
2 answers
6k views

Should I apply normalization to the observations in deep reinforcement learning?

I am new to DRL and trying to implement my custom environment. I want to know if normalization and regularization techniques are as important in RL as in Deep Learning. In my custom environment, the ...
moyukh's user avatar
  • 31
2 votes
0 answers
92 views

How do neural networks deal with inputs of different sizes that are padded in order to have them of the same size?

I am trying to create an environment for RL where the size of my input (observation space) is not fixed. As a way around it, I thought about padding the size to a maximum value and then assigning &...
user101464's user avatar
1 vote
1 answer
106 views

Scrabble rack observation with MuZero

Currently I'm trying to implement Scrabble with MuZero. The $15 \times 15$ game board observation (as input) is of size $27 \times15 \times15$ (26 letters + 1 wildcard) with a value of 0 or 1. However ...
Thrusticy's user avatar
3 votes
2 answers
402 views

What happens when the agent faces a state that never before encountered?

I have a network with nodes and links, each of them with a certain amount of resources (that can take discrete values) at the initial state. At random time steps, a service is generated, and, based on ...
krm76's user avatar
  • 155
1 vote
0 answers
311 views

Does the order in which the features are concatenated to create the state (or observation) matter?

I'm experimenting with an RL agent that interacts with the following environment. The learning algorithm is double DQN. The neural network represents the function from state to action. It's build with ...
mark mark's user avatar
  • 763