2
$\begingroup$

I have been training some kind of agent to reach a target using a Q-learning based approach, and I have tried two different types of rewards:

  1. Long-term reward: $\mathrm{reward} = - \mathrm{distance}(\mathrm{agent,target})(t+1)$

  2. Short-term reward: $\mathrm{reward} = \mathrm{distance}(\mathrm{agent,target})(t) - \mathrm{distance}(\mathrm{agent,target})(t+1)$

In the first case, I am rewarding the current progress. In the second case, I am rewarding direct progression, but this may lead to less progression in the future. My question is, what kind of reward does Q-learning need?

I understand that the $\gamma$ factor should incorporate long term rewards, so it makes more sense to reward direct progression. However, using long-term rewards gave better results for my scenario...

$\endgroup$
1
  • $\begingroup$ perhaps you could take a weighted average of both? or use one to regularise the other? this is just off the top of my head, it would need some thought to make sure there's no way the agent could exploit something like this to maximise the returns without achieving your objective. $\endgroup$ Jan 18 at 15:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.