3
$\begingroup$

I have been training some kind of agent to reach a target using a Q-learning based approach, and I have tried two different types of rewards:

  1. Long-term reward: $\mathrm{reward} = - \mathrm{distance}(\mathrm{agent,target})(t+1)$

  2. Short-term reward: $\mathrm{reward} = \mathrm{distance}(\mathrm{agent,target})(t) - \mathrm{distance}(\mathrm{agent,target})(t+1)$

In the first case, I am rewarding the current progress. In the second case, I am rewarding direct progression, but this may lead to less progression in the future. My question is, what kind of reward does Q-learning need?

I understand that the $\gamma$ factor should incorporate long term rewards, so it makes more sense to reward direct progression. However, using long-term rewards gave better results for my scenario...

Improve this question
$\endgroup$
1
  • $\begingroup$ perhaps you could take a weighted average of both? or use one to regularise the other? this is just off the top of my head, it would need some thought to make sure there's no way the agent could exploit something like this to maximise the returns without achieving your objective. $\endgroup$
    – David
    Jan 18, 2021 at 15:53

0

Reset to default

You must log in to answer this question.