How to best make use of learning rate scheduling in reinforcement learning?
To me, a low learning rate towards the end to fine-tune what you've learned with subtle updates makes sense. But I don't see why over training time this should be linearly brought down. Wouldn't this increase overfitting too, as it promotes an early adopted policy to get further and further finetuned for the rest of the training? Wouldn't it be better to keep it constant over the entire training so that when the agent finds novel experiences later, it still has a high enough learning rate to update its model?
I also don't really know how these modern deep RL papers do it. The starcraft II paper by DeepMind, and the OpenAI hide and seek paper don't mention learning rate schedules for instance.
Or are there certain RL environments where it's actually best to use something like a linear learning rate schedule?