4 votes
Accepted

Where does entropy enter in Soft Actor-Critic?

In the answer I'll be using notation similar to the one from the SAC paper. If we look at the standard objective function for policy gradient methods we have \begin{align} J_\pi &= V_\pi(s_t)\\ &...
user avatar
  • 2,226
3 votes
Accepted

What is the gradient of the objective function in the Soft Actor-Critic paper?

I'll give it a go here and try to answer your question, I'm not sure if this is entirely correct, so if someone thinks that it isn't please correct me. I'll disregard expectation here to make things ...
user avatar
  • 2,226
1 vote

How to enforce action bounds between 0 & 1 in soft actor-critic algorithm?

Yes you can map the output onto [0,1] as you indicate. You should treat this as a modification to the environment. I.e. imagine that the environment takes actions in [-1, 1] instead of [0,1]. No you ...
user avatar
  • 856
1 vote

What is the gradient of the objective function in the Soft Actor-Critic paper?

This is more meant like a comment to the previous answer. I also originally thought that $$ \nabla_{\theta}\log \pi_{\theta}(f_{\theta}(\varepsilon, s)\mid s) = \nabla_{a}\log\pi_{\theta}(a\mid s)\...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible