3 votes

Should I apply normalization to the observations in deep reinforcement learning?

The use of normalisation in neural networks and many other (but not all - decision trees are a notable exception) machine learning methods, is to improve the quality of the parameter space with ...
Neil Slater's user avatar
  • 32.1k
2 votes

Should I apply normalization to the observations in deep reinforcement learning?

On creating custom environments: ... always normalize your observation space when you can, i.e., when you know the boundaries (From stable-baselines) You could normalize them as part of the ...
mugoh's user avatar
  • 531
2 votes
Accepted

What happens when the agent faces a state that never before encountered?

Having too many states to actually visit is a common problem in RL. This is exactly why we often use function approximation. If you replace your q table with a good function approximator such as a ...
chessprogrammer's user avatar
2 votes

Training an RL model with an environment where some of the variables do not change as a result of the agent actions

In those cases the uncontrollable variables can be included as part of the observation space alongside other controllable ones, but that feels sub-optimal as the model will need to learn which ...
Neil Slater's user avatar
  • 32.1k
2 votes
Accepted

What are the differences between loss surfaces that "derive"from different observations?

Let's take in consideration linear regression. You have a dataset composed by $x,y$ pairs, and you assume they are linearly related, thus you model this problem with LR: $$ y = wx+b $$ Now, you want ...
Alberto's user avatar
  • 1,905
1 vote

Variable observation space at each episode

Actually, in most of these algorithms, that state is just used as input for some functions (e.g. some value or policy functions). Given the correct class of functions (e.g. recurrent neural networks), ...
Broele's user avatar
  • 561
1 vote

When should discretization of observations be considered?

I believe that discretizing the action/state space when using function approximators like NN is only acceptable when losing information is acceptable. Why would you discretize an observation, for ...
HenDoNR's user avatar
  • 81
1 vote
Accepted

Are multi agent or self-play environments always automatically POMDPs?

Generally, "perfect information" is not a formal trait of MDPs. There is a concept of the Markov property, but it only loosely coincides with "perfect information". For instance it ...
Neil Slater's user avatar
  • 32.1k
1 vote

Scrabble rack observation with MuZero

Use one hot encoding for each position, shape $(7, 27)$. Stack these two dimensions, shape $(189)$. Tile (replicate) this vector into images of the same resolution, now shape $(15, 15, 189)$ Stack ...
Uduse's user avatar
  • 136
1 vote

What happens when the agent faces a state that never before encountered?

I will try to explain this problem with the very tangible example of chess. In chess, the number of possible states is any configuration that you can make with the pieces on the board. So, the ...
Robin van Hoorn's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible