I am working on scheduling problem that has inherent randomness. The dimensions of action and state spaces are 1 and 5 respectively.
I am using DDPG, but it seems extremely unstable, and so far it isn't showing much learning. I've tried to
- adjust the learning rate,
- clip the gradients,
- change the size of the replay buffer,
- different neural net architectures, using SGD and Adam,
- change the $\tau$ for the soft-update.
So, I'd like to know what people's experience is with this algorithm, for the environments where it was tested on the paper but also for other environments. What values of hyperparameters worked for you? Or what did you do? How cumbersome was the fine-tuning?
I don't think my implementation is incorrect, because I pretty much replicated this, and every other implementation I found did exactly the same.
(Also, I am not sure this is necessarily the best website to post this kind of question, but I decided to give a shot.)