As the question suggests, I'm trying to see if I can solve OpenAI's hardcore version of their gym's bipedal walker using OpenAI's DDPG algorithm.

Below is a performance graph from my latest attempt, including the hyper parameters, along with some other attempts I've made. I realise it has been solved using other custom implementations (also utilising only dense layers in Tensorflow, not convolution), but I don't seem to understand why it seems so difficult to solve using OpenAI's implementation of DDPG? Can anyone please point out where I might be going wrong? Thank you so much for any help!

Latest attempt's performance: DDPG performance - Bipedal Walker Hardcore v2

  • Average score: about -75 to -80
  • Env interacts: about 8.4mil (around 2600 epochs)
  • Batch size: 64
  • Replay memory: 1000000
  • Network: 512, 256 (relu activation on inputs, tanh on outputs)
  • All other inputs left to default

Similar experiments yielded similar scores (or less), and included:

  • Network sizes of (400,300), (256,128), and (128,128,128)
  • Number of epochs ranging from 500 all the way to 100000
  • Replay memory sizes all the way up to 5000000
  • Batch sizes of 32, 64, 128, and 256
  • All of the above, with both DDPG as well as TD3

Thank you so much for any help! It would be greatly appreciated!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.