2
$\begingroup$

I have an enviroment with continuous actions and state variables. Every time I reset my env, between 2 and 5 balls spawn randomly in a box of 100x100 size. One of those balls (the red one) will receive an action (direction of movement) and will move according to some physics. This ball will always spawn.

Notice that the observation space only changes when we reset and start a new episode (when we move a step in time the observation space size remains the same). An example of my env:

enter image description here

  • Action space: the angle that we want to move the red ball
  • Observation space: the coordinates of the blue balls

The problem is that sometimes there will be only one blue ball, or two, or three... so the number of 2D coordinates in the observation will change.

Most of the reinforcement learning techniques have a fixed size state spaces, how can I deal with my dynamic size observation space and apply algorithms such as AC2, PPO, ... ? Should I consider an observation space with the maximum number of balls and make 0 the coordinates of the ones that didn't spawn ?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Actually, in most of these algorithms, that state is just used as input for some functions (e.g. some value or policy functions). Given the correct class of functions (e.g. recurrent neural networks), these algorithms do have the ability to handle dynamic size observations.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .