# Why should we use TD prediction as opposed to TD control algorithms?

Consider a problem where we have a finite number of states and actions. Given a state and an action, we can easily know the reward and the next state (deterministic). The state space is really large and has no final state.

There was a paper that for a problem of this type used TD(0) by filling the value table and chose its actions by:

$$\pi(s) = \text{argmax}_a (r(s,a) + \gamma V(s_{(t+1)}))$$

I've read somewhere that is OK to use prediction algorithms when the model is well-described with the objective of choosing best actions and not only evaluating the policy.

What is the purpose, advantages and disadvantages of using TD prediction here instead of a TD control algorithms (and just saving the $$Q(s, a)$$ table)? If it was about space, you still have to store a table with all the rewards for each state-action pair, right?

I'm not sure if I was able to explain myself very well as I was trying to keep it short, if some clarification is needed please tell me.

• Hi Miguel. I have edited your question to hopefully make it clearer. Have I changed the meaning of the question? If yes, feel free to edit it again or roll it back to the previous version. – nbro May 13 '19 at 17:37
• Yep, it's good thank you :) – Miguel Saraiva May 13 '19 at 20:37