Are there any guidelines on picking hyperparameters for Deep Reinforcement Learning?

Question

I am trying to learn machine learning from Andrew NG's Machine learning specialization on Coursera. In the chapter about reinforcement learning Andrew NG said that if you do not select correct hyperparameters your model can take a long time to train.

Are there any guidelines on picking hyperparameters for Deep Reinforcement Learning?

Let's say I have an agent who has 10000 states, how many layers and units my neural network should have? If I am using mini-batches how big should each batch be? If the states are contiguous states, how far back the agent should look before taking a decision?

Answer 1

Hyperparameter is an important thing, and it happens everywhere in Machine Learning, not just Deep Reinforcement Learning.

However, to give you the current state of Deep RL, it is now widely know that Actor-Critic methods such as A2C and PPO are the standard state-of-the-art today, and people generally do not (very) bother with traditional methods anymore (I don't remember how far Andrew NG courses go). Luckily, here's a paper that studies how to choose a good hyperparam for Deel RL. The list is very exhaustive though as it's a large-scale study, but it should give you a good starting point.