I would like to implement a variant of policy iteration that can choose one or more actions in each state. An example would be to heal and move in the game of Doom.

Parameterizing the power set of all single actions would be one idea, but I was wondering if somebody achieved good results on a similar problem, perhaps by simply defining some lower bound on the output layer and taking all actions with values larger than that bound (i.e. with actions and activation values {shoot=0.2, heal=0.51, move=0.6, jump=0.4} I would choose heal and move if the bound was 0.5)

Another idea was to collect these actions iteratively, i.e. choosing an action from a softmax output based on the state $s$ (taking action "healing") and then constructing and using some temporary state $s_t$ to evaluate that state to find another action (e.g. "moving"). This would require some dummy action that is just used to signal the end of that iteration procedure (i.e. choosing action $n+1$ will not add any other action to the set $\{ \text{healing}, \text{moving} \}$, but it will lead to the execution of those two actions and transition to the next state $s'$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.