2

Having too many states to actually visit is a common problem in RL. This is exactly why we often use function approximation. If you replace your q table with a good function approximator such as a neural network, it should be able to generelize well to states it has not yet encountered. If you do not use a function approximator but stick with a table, the ...


1

I believe that discretizing the action/state space when using function approximators like NN is only acceptable when losing information is acceptable. Why would you discretize an observation, for example, when the precise value of a continuous feature is important for making a decision? Imagine, for example, control scenarios, one of the fields that fit the ...


1

Generally, "perfect information" is not a formal trait of MDPs. There is a concept of the Markov property, but it only loosely coincides with "perfect information". For instance it is OK for there to be unknown/hidden state, provided it behaves effectively randomly (when revealed, it is drawn from a consistent distribution). An example ...


1

I will try to explain this problem with the very tangible example of chess. In chess, the number of possible states is any configuration that you can make with the pieces on the board. So, the starting position is a state, and after you did one move you are in a different state. The total number of chess states is more than $10^{100}$. It is therefore very ...


Only top voted, non community-wiki answers of a minimum length are eligible