1
$\begingroup$

I asked a question relating to TicTacToe playing in RL. From the answer it seems to me a lot is dependent on the opponent (rightly so, if we write down the Expectation Equations). My question is (in the context of TicTacToe or Chess):

  • How to make the RL player a perfect player (TicTactoe) or an expert (chess). As far as TTT is concerned when playing against a perfect player an RL will become perfect conditioned on the clause that the opponent is perfect. So will this hold true if the same RL algo, with its learned values are used to play some other lesser perfect players?
  • The question can be extended to the scenario, can a RL player with pre-trained values (assume from a perfect or expert opponent) be used in any scenario with best results?

Note: The problem is more severe in chess, since experts will use some kind of opening moves which will not match with say a random player and thus finding values for those states becomes a problem, since we have not encountered it during training time.

Footnote: Any resources on Game Playing RL is appreciated.

$\endgroup$
1
$\begingroup$

I would say a good way to make a good agent would be making it play against itself. As you go through several episodes, with a good exploration and exploitation balance, both will gradually learn and converge to Q*(s,a).´

So will this hold true if the same RL algo, with its learned values are used to play some other lesser perfect players?

As long as the states that are played (or approximations if you are using Function approximation methods) were simulated enough times during training, it will play well against any kind of opponent.

If you are training against a completely perfect opponent and you are not using function approximation, I believe you could get to an incomplete Q(s,a) table and as such not be able to predict the best play when facing certain states.

$\endgroup$
  • $\begingroup$ I figured out that playing against itself is a good idea, but I think it's good if the state space is small...But for big games like chess I am not sure it would work even if we use function approximation since we have to assume that some kind of underlying function is there on chess determining state value vs positions. $\endgroup$ – DuttaA May 19 at 14:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.