I realized that my state space is very large in size. I had planned to use tabular Q-learning (Bellman equation to update the $Q(s, a)$ after each action taken). But this 'large space' realization has now disappointed me and I read a lot of stuff on the internet. I have the following confusions.
I saw the 'approximation' term for the 'large space' scenario (for example, in this Medium blog post). But what it is exactly? I can't reduce the states I have nor can I club together different states and update the Q values. So, what is it I should do when they say 'approximate'? If it is the $Q(s,a)$ we approximate, then won't we anyway do for each state $s$ as and when it is encountered? How does this help in a 'large space' scenario?