I have a heuristic solution to a problem which works quite well when certain environmental parameters are known and unchanging. However, in a real world setting these parameters will not be known and are likely to fluctuate over the course of an episode. I'm hoping to use deep RL to develop a policy that will be similar to the heuristic, but robust to these unknowns.
My question is: does the RL agent need to be trained "from scratch" as one would typically do or is there a way to leverage the existing policy to jump start the training progress?
In the latter case, what would this looks like? I've had a couple of thoughts, but I'm not sure how well any of them would work.
Reward actions that the heuristic would take in an environment with static parameter values, then gradually make the environment more complex and set a new reward function based on what I'm actually interested in.
Instead of taking random actions in the exploration stage, take actions dictated by the heuristic.