Timeline for Initialising DQN with weights from imitation learning rather than policy gradient network
Current License: CC BY-SA 4.0
5 events
when toggle format | what | by | license | comment | |
---|---|---|---|---|---|
Nov 15, 2020 at 15:17 | comment | added | calveeen | I see thank you for clarifying the terminology. I think the authors referred to the policy network as the "RL policy network" and they mentioned training it with self play. However, they did not mentioned the specific "type" of training in the sense that they could be using vanilla REINFORCE algorithm or actor critic type methods, though i highly doubt they use a vanilla policy gradient | |
Nov 15, 2020 at 11:11 | comment | added | Neil Slater | @calveen: "Policy Gradient" is not a type of network, but a type of training - there are constraints on the type of network though, such as its output must represent probabilities of taking actions. In AlphaGo the "imitation training" policy network took input as board state, and predicted where a human expert player would play. It was combined with other networks later using MCTS and to create the full AlphaGo agent. At least two of the other networks were trained using a variant of Actor-Critic reinforcement learning (a policy gradient approach). | |
Nov 15, 2020 at 5:58 | comment | added | calveeen | Thanks for the advice @Neil :-) Also with regard to the policy gradient network that alphaGo uses with initialisation from imitation learning, would you happen to know what kind of policy gradient network did they use ? I would think that if they used a sort of actor-critic network then i would think that the initial benefits of having a good policy might be "erased" by an untrained critic network | |
Nov 15, 2020 at 5:58 | vote | accept | calveeen | ||
Nov 14, 2020 at 17:53 | history | answered | Neil Slater | CC BY-SA 4.0 |