New answers tagged deep-rl
0
The core mechanics of AlphaZero during selfplay and real tournament games are the same: something similar to Monte Carlo Tree Search is done but guided by the current neural network instead of random simulations. The network is only doing inference, it's not learning during a tree search. There's a great summary diagram here.
The differences between selfplay ...
1
I have two suggestions that you can look into. Based on my own work in RL, I believe the first one will require less work to implement.
If the observability of the environment is not an issue, then you could give the agent a relative measure (distance to the goal) as part of the observation to provide it with knowledge of how far away it is. You can also ...
2
Question 1: I don't think they ran AlphaGo or AlphaGoZero in training mode during tournament matches because the computing power required for this is really large. I don't recall if this is described in the documentary but see this quote from the AlphaZero paper (page 4):
using 5,000 first-generation TPUs (15) to generate self-play games and 64 second-...
2
You sample according to the probability distribution $\pi(a \mid s, \theta)$, so you do not always take the action with the highest probability (otherwise there would be no exploration but just exploitation), but the most probable action should be sampled the most. However, keep in mind that the policy, $\theta$, changes, so also the probability distribution....
1
there should be absolutely no problem with training an agent on any available episode roll-out data. That is because a MDP implies for an any state S, the optimal action to take is entirely dependent on the state.
The desired end-state of the trained model is that it can identify the optimal action. When comparing reinforcement learning (RL) methods, you ...
Top 50 recent answers are included