# DQN, how to choose the reward fucntion?

I built a simple AI system that tries to solve the 8 puzzle using DQN. The problem is, if the agent gets only a reward greater than zero when winning, the training will take a long time, so I made a smooth reward function instead: $$R=(n/9)^3$$, where $$n$$ is the number of pieces that are in the right position.

The training became quicker but the AI chose to match 7 pieces out of 9 to get a reward of $$(7/9)^3/(1-\gamma) = 0.47/(1-\gamma) = 4.7$$, for $$\gamma=0.9$$, choosing to win and getting reward of 1 doesn't make sense to the AI, lowering $$\gamma$$ will result in the AI to choose instant reward instead of long-term reward, so that will not be very helpful; lowering rewards of non-winning stats will make the training very slow.

So, how do I choose a good reward function?

• Try using Manhattan distance between current position and supposed position for all pieces, use that as negative reward. – Brale Dec 9 '19 at 19:03
• Increase the reward for winning. The reward for winning should normally dwarf all other rewards. – Recessive Dec 10 '19 at 3:05
• Changed the reward of winning to 5, that didn't improve the results a lot somehow. i like the idea of negative rewards i will give it a go. – F0urAt Dec 10 '19 at 16:59