I'm confused as to the purpose of training a neural network (NN) for reinforcement learning (RL) tasks such as Gridworld. In RL tasks, namely q-learning, we have a q-learning update rule, which is designed to take some state and action and compute the value of that state-action pair.
Performing this process several times will eventually produce a table of states and what action will likely lead to a high reward.
In RL examples, I've seen them train a neural network to output q-values and a loss function like MSE to compute the loss between the q-learning update rule q-value and the NN's q value.
So:
(a) Q-learning update rule-> outputs target Q-values
(b) NN -> outputs Q values
MSE to compute the loss between (a) and (b)
So, given we already know what the target Q-value is from a, why do we need to train a NN?