For an RL problem on a continuous state space, the states could be discretized into buckets and these buckets used in implementing the Q-table. I see that is what is done here. However, according to van Hasselt from his book, this discretization changes the problem into a partially observable MDP (POMDP), and this is understandable. And I know POMDPs require special treatment from the vanilla Q-learning we are used to (observation space, belief states, etc).
But my question is: is there a specific technical reason why a discretized-state problem (which is now POMDP) should be solved using POMDP algorithms, instead of plainly constructing a vanilla Q-table using the discretized states (i.e. the buckets from discretization)? In other words, is there a disadvantage in not using POMDP algorithms to tackle the discretized-state problem?