I am working on a problem, and I want to implement it as a reinforcement learning problem and integrate it into the OpenAI's gym.

My states are in the form of lists of length $n$, where each element is chosen from a discrete interval $[0, m]$. For example, for $n=6$ and $m=3$, this is a sample from the observation space:

[0 2 1 3 3 2]

And the possible accessible states from this space is a set of other lists that are achieved by changing a number of $k$ elements the elements in the list with a number from the same $[0, m]$. For example, for $k=1$, we can have the following states as two subsequent states of the previous state:

[0 2 2 3 3 2]


[0 3 1 3 3 2]

What is an efficient way to represent the "actions" in the OpenAI gym for such a scenario?

One way that comes to my mind is to just use the next state as the action itself, for example, if I write:

action = env.action_space.sample()

The action would be the next state (which also implicitly contains the action) and then in the env.step(action) make the state equal to the next state.

Does anyone know a better way or using the implicit action representation with the next state is the optimal way? Does anyone know a predefined gym environment that also has the same representation? What are the cons of the implicit representation of the actions that I just explained?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.