In Q-learning, is it mandatory to know all possible states that can the agent may end up in?
I have a network with 4 source nodes, 3 sink nodes, and 4 main links. The initial state is the status network where the sink nodes have its resources at its maximum. In a random manner, I generate service requests from the source nodes to the sink nodes. These service requests are generated at random timesteps, which means that, from state to state, the network status may stay the same.
When a service request is launched, the resources from the sink nodes change, and the network status changes.
The aim of the agent is to balance the network by associating each service request to a sink node along with the path.
I know that in MDP you are supposed to have a finite set of states, my question is: if that finite set of states is supposed to be all possible states that can happen, or is just a number that you consider enough to optimize the Q-table?