I'm trying to teach a humanoid agent how to stand up after falling. The episode starts with the agent lying on the floor with its back touching the ground, and its goal is to stand up in the shortest amount of time.

But I'm having trouble in regards to reward shaping. I've tried multiple different reward functions, but they all end up the same way: the agent quickly learns to sit (i.e. lifting its torso), but then gets stuck on this local optimum forever.

Any ideas or advice on how to best design a good reward function for this scenario?

A few reward functions I've tried so far:

  • current_height / goal_height
  • current_height / goal_height - 1
  • current_height / goal_height - reward_prev_timestep
  • (current_height / goal_height)^N (tried multiple different values of N)
  • ...
  • $\begingroup$ In section 6.2.2 of this paper they provide some reward functions so you can try with these. $\endgroup$
    – Brale
    May 20, 2019 at 17:14

1 Answer 1


See https://gymnasium.farama.org/environments/mujoco/humanoid_standup/#rewards for a description of how to get a humanoid ragdoll to stand up!

  • $\begingroup$ Note that I have got this far so far myself... youtube.com/watch?v=ISZ_xgybazQ (it reliably rolls over onto it's front) $\endgroup$
    – ChrisSim
    Apr 16, 2023 at 11:09
  • 2
    $\begingroup$ Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community Bot
    Apr 17, 2023 at 14:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .