Questions tagged [advantage-actor-critic]
For questions related to the advantage actor-critic algorithms (that is, actor-critic algorithms that use the "advantage" function).
4
questions with no upvoted or accepted answers
2
votes
0answers
40 views
Problems with gradient-biased actor critic methods
To my knowledge, there are at least 6 different variants of Actor Critic:
\begin{array}{l l l l}
\text{actor gradient} & \text{critic gradient} & \text{actor gradient biased} & \text{name} ...
2
votes
0answers
367 views
Implementation of PPO - Value Loss not converging, return plateauing
Copy from my reddit post:
(Sorry if this does not fit here, please tell me and i delete it)
Help regarding
I'm working on an implementation of PPO, which i plan to use in my (Bachelors) Thesis. To ...
1
vote
0answers
31 views
What is the difference between step_model and train_model in the OpenAI implementation of the A2C algorithm?
I'm struggling a little with understanding the OpenAI implementation of A2C in the baselines (version 2.9.0) package. From my understanding, one ...
0
votes
0answers
25 views
How to deal with GAE ineffectiveness because of critic value adaptation?
I've noticed if you have a small negative reward (e.g.,-0.01) per step for idling and a series of idle steps, an agent seems to learn to trick GAE by learning a ...
