0
$\begingroup$

I realized that my state space is very large in size. I had planned to use tabular Q-learning (Bellman equation to update the $Q(s, a)$ after each action taken). But this 'large space' realization has now disappointed me and I read a lot of stuff on the internet. I have the following confusions.

I saw the 'approximation' term for the 'large space' scenario (for example, in this Medium blog post). But what it is exactly? I can't reduce the states I have nor can I club together different states and update the Q values. So, what is it I should do when they say 'approximate'? If it is the $Q(s,a)$ we approximate, then won't we anyway do for each state $s$ as and when it is encountered? How does this help in a 'large space' scenario?

Improve this question
$\endgroup$
3
  • $\begingroup$ The only thing that is missing from your question is maybe some specific context. You say "I saw the 'approximation' term for the 'large space' scenario". Can you please quote the excerpt (and provide the link to the source) where people use the term "approximation" in this specific context. It may be necessary to answer this question more appropriately, although I think it may already be answerable. After that, I will reopen your post. $\endgroup$
    – nbro
    Nov 19, 2021 at 14:47
  • $\begingroup$ Actually, I saw like lot of sources which are more for continuous state spaces and they say approximation. For large state spaces. I saw the term 'discretization'. But I couldn't put all the details here and thought that people would eventually point this out in answers. So, if I am try to put a link here it would look like aah, it is for continuous space and you are asking for large state space. I am confused and so I asked the way it is. $\endgroup$
    – knowledge_seeker
    Nov 19, 2021 at 15:01
  • $\begingroup$ Discretisation of a continuous space is one form of approximation (one of the simplest, but can be very effective depending on the environment). $\endgroup$
    – Neil Slater
    Nov 19, 2021 at 15:24

0

Reset to default

You must log in to answer this question.