2
$\begingroup$

As the question suggests, I'm trying to see if I can solve OpenAI's hardcore version of their gym's bipedal walker using OpenAI's DDPG algorithm.

Below is a performance graph from my latest attempt, including the hyper parameters, along with some other attempts I've made. I realise it has been solved using other custom implementations (also utilising only dense layers in Tensorflow, not convolution), but I don't seem to understand why it seems so difficult to solve using OpenAI's implementation of DDPG? Can anyone please point out where I might be going wrong? Thank you so much for any help!

Latest attempt's performance: DDPG performance - Bipedal Walker Hardcore v2

  • Average score: about -75 to -80
  • Env interacts: about 8.4mil (around 2600 epochs)
  • Batch size: 64
  • Replay memory: 1000000
  • Network: 512, 256 (relu activation on inputs, tanh on outputs)
  • All other inputs left to default

Similar experiments yielded similar scores (or less), and included:

  • Network sizes of (400,300), (256,128), and (128,128,128)
  • Number of epochs ranging from 500 all the way to 100000
  • Replay memory sizes all the way up to 5000000
  • Batch sizes of 32, 64, 128, and 256
  • All of the above, with both DDPG as well as TD3

Thank you so much for any help! It would be greatly appreciated!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.