Suppose I have a reward function $R$ that I wish to penalize w.r.t two distinct phenomenons $A$ and $B$. $A$, for example, could represent the phenomenon of the state not crossing some boundary $[s_1,s_2]$ and $B$ can represent the phenomenon that two consecutive actions shouldn't be too far apart $|a_t - a_{t+1}| < \epsilon$, for some small $\epsilon$. A trivial reward mechanism can be:
step(prev_a, a,epsilon, min_s, max_s):
s = env(action)
if not min_s < s < max_s:
r -= s
if ||a-prev_a|| > epsilon:
r -= a
As $A$ and $B$ are from different worlds (different physical units, if you will), they both have different ranges. For example, a state $s$ may obtain values that are at most $10$, though an action $a$ may have larger values like $100$. Hence, penalizing by subtracting the state or the action from the current reward may lead to the preference of the agent to only make sure the action condition is set, as this one translates to more future rewards.
How can this issue be addressed? I assume some normalization should be added, though I'm not quite sure how.
Any ideas?