# Tag Info

Accepted

### Where does entropy enter in Soft Actor-Critic?

In the answer I'll be using notation similar to the one from the SAC paper. If we look at the standard objective function for policy gradient methods we have \begin{align} J_\pi &= V_\pi(s_t)\\ &...
• 2,286
Accepted

### What is the gradient of the objective function in the Soft Actor-Critic paper?

I'll give it a go here and try to answer your question, I'm not sure if this is entirely correct, so if someone thinks that it isn't please correct me. I'll disregard expectation here to make things ...
• 2,286
Accepted

### Can off-policy algorithms benefit from the parallelization?

From the point of view of someone developing an in-house DRL lib and working on extremely CPU-intensive environments (usually large finite element-based simulations that can require several hours to ...
• 136
1 vote

### How to enforce action bounds between 0 & 1 in soft actor-critic algorithm?

Yes you can map the output onto [0,1] as you indicate. You should treat this as a modification to the environment. I.e. imagine that the environment takes actions in [-1, 1] instead of [0,1]. No you ...
• 1,111
1 vote

### What is the gradient of the objective function in the Soft Actor-Critic paper?

This is more meant like a comment to the previous answer. I also originally thought that  \nabla_{\theta}\log \pi_{\theta}(f_{\theta}(\varepsilon, s)\mid s) = \nabla_{a}\log\pi_{\theta}(a\mid s)\...
• 141

Only top scored, non community-wiki answers of a minimum length are eligible