Can/should a reward function depend on something other than state in a DQN

Question

Question: Is it OK to have a reward function on a DQN or any RL algorithm that depends on variables other than the enviroment state? I'm asking because, so far I'm learning from tutorials, but I've never seen a reward function working with something else than enviroment.

My Case: Imagine a trading bot. In order to get the reward, I'd neet to know the position (how much of a stock/cryptocurrency I have) to know if the bot won or lost money in the taken step. But I don't want to have current position on the enviroment's state as it is not important for calibration. Important variables in enviroment are other key market indicators.

I thought on having a separate variable with auxiliar data such as current position. So reward function would rely on new_state, previous_state and aux_data.

Is this OK? Bad practice?

Answer 1

Reward functions often depend on variables not present in the current state. Trading is the perfect example. Try as we might, we are unlikely to capture every variable necessary to perfectly predict the movement of a price and in effect the agent's reward. This is not typically a desirable property for an environment to have.

But if as you say the present positions are not important for calibration then feel free to test the hypothesis. I can drive a car without a speedometer but do not ask me to drive with a blindfold on. Not every missing variable has disastrous consequences despite their effect on the reward.