I am working on a problem, and I want to implement it as a reinforcement learning problem and integrate it into the OpenAI's gym.
My states are in the form of lists of length $n$, where each element is chosen from a discrete interval $[0, m]$. For example, for $n=6$ and $m=3$, this is a sample from the observation space:
[0 2 1 3 3 2]
And the possible accessible states from this space is a set of other lists that are achieved by changing a number of $k$ elements the elements in the list with a number from the same $[0, m]$. For example, for $k=1$, we can have the following states as two subsequent states of the previous state:
[0 2 2 3 3 2]
[0 3 1 3 3 2]
What is an efficient way to represent the "actions" in the OpenAI gym for such a scenario?
One way that comes to my mind is to just use the next state as the action itself, for example, if I write:
action = env.action_space.sample()
The action would be the next state (which also implicitly contains the action) and then in the
env.step(action) make the state equal to the next state.
Does anyone know a better way or using the implicit action representation with the next state is the optimal way? Does anyone know a predefined gym environment that also has the same representation? What are the cons of the implicit representation of the actions that I just explained?