# How would you shape a reward function if there was four quantities to optimize?

I found this article quite useful on how to shape a reward function in RL. However, the example they gave is quite simple, where the goal is to minimize only two quantities (velocity and distance).

How would you formulate the reward function if you had, for instance, 4 quantities to optimize? • At AI Stack Exchange we prefer questions to be self-contained. This one requires going off site to read the article to even understand the basics of what you are asking. Could you please summarise the article in a couple of short paragraphs - what was the environment, how was the reward function shaped (give the equation, you can use LaTex inside $ like this $r = x_{t} - x_{t+1}$ which becomes$r = x_{t} - x_{t+1}\$), and how did it help? Jun 17, 2021 at 8:28
• I mentioned the article just as an example, my question is just that I need to formulate a reward function to optimize 4 parameters in the example that I sent they tried to optimize only 2 parameters as shown in the image that I provided now! Thanks for your reply. Jun 17, 2021 at 9:24
• check out this answer Jun 17, 2021 at 12:14
• @BAKYAC Could you please at least provide more details about your concrete problem? You say that you have 4 quantities to optimize. What quantities/variables are these? What problem are you trying to solve with RL? What is the action and state spaces?
– nbro
Jun 17, 2021 at 23:51

$$R = \left\{ \begin{array}{ll} d_m - 3 - \left| d_l - d_r \right| & \text{if not terminal state} \\ -100 & \text{otherwise} \end{array} \right.$$
where $$d_m$$ is the middle distance, $$d_l$$ is the left distance and $$d_r$$ is the right distance, and $$d_m, d_l, d_r \in [0, 10]$$. To get this information, the agent has a laser sensor. The agent does not directly observe these distances. Instead, it gets a real number as the reward signal that indicates how good the agent is performing an action and tries to map the camera view to it. This function is designed so that the agent should stay in the middle of the tunnel, $$-\left|d_l - d_r\right|$$, and has to avoid head-on collisions, $$d_m - 3$$. Thus, the highest possible reward is $$R = 10 - 3 - \left| 10 - 10 \right| = 7$$ 