Going through the DQN paper, it said the state-space is high dimensional. I am a little bit confused here. Suppose my state is a high dimensional vector of
N length where N is a huge number. Let's say I solve this task using Q-learning and I fix my state space to 10 vectors each of
N dimensions. Q-learning can easily work with these settings as we need only a table of dimensions 10 x number of actions.
Let's say my state space can have an infinite number of vectors each of
N dimensions. In these settings Q-learning would fail as we cannot store Q-values in a table for each of these infinite vectors. Meanwhile on the other hand DQN would easily work as neural networks can generalize for any vector in the state-space.
Let's also say I have a state space of infinite vectors but each vector is now of length 2 i.e. very small dimensional vectors. Would it make sense to use DQN in these settings? Should this state-space be called high dimensional or low dimensional?