I'm attempting to design an action space in OpenAI's gym and hitting the following roadblock. I've looked at this post which is closely related but subtly different.

The environment I'm writing needs to allow an agent to make between $1$ and $n$ sub-actions in each step. Leaving it up to the agent to decide how many sub-actions it wants to take. So, something like (sub-action-category, sub-action-id, action) where the agent can specify between $1$ and $n$ such tuples.

It doesn't seem possible to define a Box space without specifying bounds on the shape which is what I need here. I'm trying to avoid defining an action space where each sub-action is explicitly enumerated by the environment like (action) tuple with n entries for each sub-action.

Are there any other spaces I could use to dynamically scale the space?

  • $\begingroup$ Is the last element of the tuple (sub-action-category, sub-action-id, action) a discrete or continuous value? What is its dimension? gym offers the class MultiDiscrete for a similar use case. You could also write your own action type if you wanted to, but imo the straightest way to deal with this is to have actions to be instances of some suitably defined Python class you can document etc. and deal with the details of the execution to the Environment subclass. $\endgroup$ – Miquel Ramirez Feb 19 '20 at 9:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.