0
$\begingroup$

What's the benefit of repeating an action for a consecutive number of time steps? Is there a way to tell if an agent in a given environment might perform better from repeated actions?

I came across an action repeat hyperparameter used in the experiments of this paper (Table 11.) and was wondering what the objective of this hyperparameter might be. This happens during training where the action selected by a policy at step $t$ is executed for a repeat number of timesteps $t_r$ after which the policy is then queried for another action at the current state $s_{t + t_r}$.

Improve this question
$\endgroup$
4
  • $\begingroup$ You mean hold a given action selection for more than one timestep like sticky action? $\endgroup$
    – HenDoNR
    Mar 18 at 8:57
  • $\begingroup$ I assume you're talking about behaviour policies, so during learning. It might a good idea to explicitly state this in your post, even though it seems obvious. I would also recommend that you explain why you're asking this question: just out of curiosity, or do you have something more specific that you can share with us? $\endgroup$
    – nbro
    Mar 18 at 9:34
  • $\begingroup$ Yes @HenDoNR, action selection for more than a signle timestep $\endgroup$
    – mugoh
    Mar 19 at 10:44
  • $\begingroup$ I've edited the question to include the suggested changes @nbro $\endgroup$
    – mugoh
    Mar 19 at 10:51

1 Answer 1

Reset to default
1
$\begingroup$

A primary reason to repeat an action of a series of timesteps is that your environment may require more than one timestep to process the timestep. Said another way, changing the action every timestep might prove to hamper the overall learning process significantly.

As an example, consider using RL with something from OpenAI Gym. Imagine that you are trying to train an agent to play something like Mario Brothers. If your agent takes a different random action at every timestep, how long might it take the agent to realize that it needs to keep sending "Right" to improve the x position? It might be better to configure your agent such that each action is repeated for, say, four timesteps, allowing the action to begin to take on real meaning in the environment.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .