# How to make the RL player a perfect/expert (tic-tac-toe/chess) player?

I asked a question related to tic-tac-toe playing in RL. From the answer, it seems to me a lot is dependent on the opponent (rightly so, if we write down the expectation equations).

My questions are (in the context of tic-tac-toe or chess):

• How to make the RL player a perfect/expert (tic-tac-toe/chess) player?

As far as TTT is concerned, when playing against a perfect player, an RL will become perfect, provided that the opponent is perfect.

So, will this hold true if the same RL algorithm, with its learned values, are used to play some other lesser perfect players?

• Can an RL player with pre-trained values (assume from a perfect or expert opponent) be used in any scenario with best results?

Note: The problem is more severe in chess, since experts will use some kind of opening moves which will not match with say a random player and thus finding values for those states becomes a problem, since we have not encountered it during training time.

Footnote: Any resources on Game Playing RL is appreciated.

I would say a good way to make a good agent would be making it play against itself. As you go through several episodes, with a good exploration and exploitation balance, both will gradually learn and converge to $$q_*(s,a)$$.
If you are training against a completely perfect opponent and you are not using function approximation, I believe you could get to an incomplete $$Q(s,a)$$ table, and so not be able to predict the best play when facing certain states.