Can all RL algorithms learn with discrete state spaces?

Question

This question come to mind when i was planing to do a benchmark of RL algorithms to my Environment. In fact, Q-Learning, SARSA actually only handles with discrete state spaces because they are tabular methods, but Deep RL algorithms like PPO, DDPG and other algorithms that can handle continuous state spaces, can they actually handle discrete state spaces by discretizing the continuous state space into bins and then input them to the Neural Networks ?

Answer 1

Two things in this post need to be corrected. Q-learning is not married to an estimator. For instance, DQN stands for Deep Q network, a type of Q-Learning that utilizes neural networks. On the other hand, tabular utilizes a table for estimation. Second, tabular does not necessarily mean only discrete state spaces. An algorithm can discretize the space and put it into a table.

A neural network doesn't necessarily have an issue taking in a discrete input. When the input is discrete, a common form of input pre-processing is min-max scaling, which often improves network performance. This fact is especially true when some input features are on different scales.