4 votes

Why does self-playing tic-tac-toe not become perfect?

There are lots of ways that RL agents can fail to learn properly, so you are faced with a little bit of experimentation and maybe bug hunting unfortunately. However, from the description you have ...
user avatar
  • 24.5k
3 votes
Accepted

How to fight with unstability in self play?

The AlphaZero paper mentions an "evaluation" step that seems to deal with the the problem similar to yours: ... we evaluate each new neural network checkpoint against the current best ...
user avatar
  • 1,833
3 votes

How can both agents know the terminal reward in self-play reinforcement learning?

If you are running self-play in a two player zero sum game, then you can do the following: Arbitrarily decide the reward scheme for winning, drawing, losing is +1, 0, -1 for Player A. Have Player A's ...
user avatar
  • 24.5k
3 votes

How can both agents know the terminal reward in self-play reinforcement learning?

When one agent makes a move, that move should be perceived as part of the "state transition" executed "by the environment" from the perspective of the other agent. For example, suppose that, as a "...
user avatar
  • 9,419
2 votes

How to fight with unstability in self play?

When in an environment with competing agents, from the perspective of each agent, the environment becomes non-markovian. That occurs because each agent is constantly adapting its own strategy to other'...
user avatar
1 vote
Accepted

Are multi agent or self-play environments always automatically POMDPs?

Generally, "perfect information" is not a formal trait of MDPs. There is a concept of the Markov property, but it only loosely coincides with "perfect information". For instance it ...
user avatar
  • 24.5k
1 vote

How does MuZero learn to play well for both sides of a two-player game?

Both players are represented by the exact same network with the exact same weights(similar to AplhaGO, AlphaGoZero and AlphaZero). So, they will both behave identical. Because you only have a single ...
user avatar
1 vote
Accepted

Generalising performance of Q-learning agent through self-play in a two-player game (MCTS?)

To solve this, I now train batches of models (~ 10 models per batch), which are then used in group as a new opponent, This seems quite a reasonable approach on the surface, but possibly the agents ...
user avatar
  • 24.5k
1 vote

In AlphaZero, which policy is saved in the dataset, and how is the move chosen?

However, I’m not sure which policy is saved The policy from the Monte Carlo tree search is stored, as we can get the policy estimate from the network later by passing the given state through the ...
user avatar
1 vote
Accepted

Given these two reward functions, what can we say about the optimal Q-values, in self-play tic-tac-toe?

Chapter 1 of Sutton & Barto, doesn't introduce the full version Q learning, and you are probably not expected to explain the full distribution of values at that stage. Probably what you are ...
user avatar
  • 24.5k
1 vote
Accepted

How can I oppose two AI agents with keras / tensoflow?

Keras/Tensorflow are mostly used of developing/training/deploying neural networks. For descision making problems, if you want to use machine learning, reinforcement learning is in most cases applied. ...
user avatar
  • 225

Only top scored, non community-wiki answers of a minimum length are eligible