# What are proxy reward functions?

The understanding I have is that they somehow adjust the objective to make it easier to meet, without changing the reward function.

... the observed proxy reward function is the approximate solution to a reward design problem

(source: Inverse Reward Design)

But I have trouble getting how they fit the overall reward objective and got confused by some examples of them. I had the idea of them being small reward functions (as in the case of solving for sparse rewards) eventually leading to the main goal. But the statement below, from this post, made me question that.

Typical examples of proxy reward functions include “partial credit” for behaviors that look promising; artificially high discount rates and careful reward shaping;...

1. What are they, and how would one go about identifying and integrating proxy rewards in an RL problem?

2. In the examples above, how would high discount rates form a proxy reward?

I'm also curious about how they are used as a source of multiple rewards

• Maybe have a look at ai.stackexchange.com/q/20040/2444, although I think that you know what "proxy" usually means. – nbro Sep 22 at 11:24
• So "proxies" are intermediaries. This helped add clarity into the "what are they" part of the question – mugoh Sep 23 at 5:05