The understanding I have is that they somehow adjust the objective to make it easier to meet, without changing the reward function.
... the observed proxy reward function is the approximate solution to a reward design problem
(source: Inverse Reward Design)
But I have trouble getting how they fit the overall reward objective and got confused by some examples of them. I had the idea of them being small reward functions (as in the case of solving for sparse rewards) eventually leading to the main goal. But the statement below, from this post, made me question that.
Typical examples of proxy reward functions include “partial credit” for behaviors that look promising; artificially high discount rates and careful reward shaping;...
What are they, and how would one go about identifying and integrating proxy rewards in an RL problem?
In the examples above, how would high discount rates form a proxy reward?
I'm also curious about how they are used as a source of multiple rewards