2022 Developer Survey is open! Take survey.
17 votes
Accepted

What is the difference between actor-critic and advantage actor-critic?

Actor-Critic is not just a single algorithm, it should be viewed as a "family" of related techniques. They're all techniques based on the policy gradient theorem, which train some form of critic that ...
user avatar
  • 9,326
5 votes
Accepted

What is the difference between vanilla policy gradient with a baseline as value function and advantage actor-critic?

The difference between Vanilla Policy Gradient (VPG) with a baseline as value function and Advantage Actor-Critic (A2C) is very similar to the difference between Monte Carlo Control and SARSA: The ...
user avatar
  • 23.2k
2 votes
Accepted

Policy Gradient ( Advantage actor-critic) for multiple simultaneous continuous actions

Sounds like you have several problems with the way your policy is parametrized. You don't have to use the multivariate normal distribution. It can work, and probably others have done it already (if ...
user avatar
  • 746
2 votes

What is the difference between A2C and running an agent in an environment vector?

I believe if you run a single agent in multiple parallel environments many times you will get similar actions in similar states, the reason behind multiple agents is that you will have different ...
user avatar
  • 2,226
2 votes

What is the difference between actor-critic and advantage actor-critic?

According to Sutton and Barto, they are the same thing. Note 13.5-6 (page 338) of their Reinforcement Learning: An Introduction, 2nd Edition book: Actor-critic methods are sometimes referred to as ...
user avatar
1 vote
Accepted

What is the advantage of using more than one environment with the advantage actor-critic?

What is the advantage of using more than one environment with a single agent? There are two main advantages to this approach: The dataset for training is closer to the independent, identically ...
user avatar
  • 23.2k
1 vote
Accepted

How to set the target for the actor in A2C?

In short, my last sentence was the correct answer. The "target" is a one-hot with the selected action, but there's a trick. A2C Loss Function A very crucial part of A2C implementation that I missed ...
user avatar
  • 141
1 vote
Accepted

Why I got the same action when testing the A2C?

Disclaimer: Without the full code, we can only speculate. I encourage you to post the full code on Google Colab or something like this. In the meanwhile, here is my point of view: The Problem Looks ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible