# What happens when the agent faces a state that never before encountered?

I have a network with nodes and links, each of them with a certain amount of resources (that can take discrete values) at the initial state. At random time steps, a service is generated, and, based on the agent's action, the network status changes, reducing some of those nodes and links resources.

The number of all possible states that the network can have is too large to calculate, especially since there is the random factor when generating the services.

Let's say that I set the state space large enough (for example, 5000), and I use Q-Learning for 1000 episodes. Afterwards, when I test the agent ($$\max Q(s,a)$$), what could happen if the agent faces a state that did not encounter during the training phase?

I will try to explain this problem with the very tangible example of chess. In chess, the number of possible states is any configuration that you can make with the pieces on the board. So, the starting position is a state, and after you did one move you are in a different state. The total number of chess states is more than $$10^{100}$$. It is therefore very unlikely that a chess bot has seen all the states in training when playing a match.