# Picking a random move in exploitation in Q-Learning

I've been unsure about a principle of Q-Learning, I was hoping someone could clear it up.

When a new state is encountered, and thus there are no existing Q values, and that the algorithm decides to exploit, and not explore, how is the move chosen, since all the values are 0?

Is it chosen randomly? This intuitively would make sense, since after this, the state-move pair would have a value and thus the matrix would get filled up throughout the iterations. But I just want to make sure I understand this correctly...

Thanks

• Yes I think there a few issues here. The main one is the size of the state space ($10^{12}$), although the number of states that are likely to come about is much smaller. Another issue that the agent is trained against a Minimax AI, which always follows the same path, and although it is able to win against it most times, a human player is not guaranteed to play the way a Minimax algorithm responds to changes in state. Perhaps unsuitable hyperparameters? Not sure... Thanks for the input though. – Pete Apr 25 at 16:20
• Oh, so you are training on minimax and testing against human ? If that's the case it is possible that the agent overfits minimax player since MM player will usually choose the same move in the same position. When you play against human it is possible that agent visits states never met before (since there are so many states). If that happens behaviour is undefined since the agent didn't learn anything about those states before, you might get lucky or not. Have you considered "pretraining" it on a random agent ? Also $10^{12}$ states is quite a lot for tabular Q-learning to solve. – Brale_ Apr 25 at 16:37