How are rewards calculated for episodic tasks like playing chess or tic-tac-toe?

Question

I am new to Reinforcement Learning and trying to understand the concept of reaping rewards during episodic tasks. I think in games like tic-tac-toe, rewards will be in terms of a win or lose. But does that mean we need to finish the entire game to gain the reward? I mean reward will make sense only if three of the tokens are in one line. Each game of tic-tac-toe will be different as the sequence of actions followed will be different. So does reward come into the picture only after completing the game? And what if the game is a draw?

Answer 1

There is two ways to formulate a reward function for these types of problems. First there is sparse reward:

Win +1

Loss -1

All other rewards are 0.

As the opposite there also exist dense rewards, which could be some signal for every timestep which tells you how well you're doing, i.e. give rewards in chess if you kick an opponents piece of the board.

Sparse rewards is harder to learn from since a lot of the time your reward and so your gradient is 0. On the other hand reward functions for dense reward require careful crafting by hand and leave room for human error and bias.