Question is regarding Deep Reinforcement Learning using Policy Gradients.
Cutting edge policy gradients algorithms are TRPO (Trusted Region Policy Optimization) and PPO (Proximal Policy Optimization).
When using single continuous action then normally you would use some random distribution (for example Gaussian) for the loss function. The rough version is:
$L(\theta) = log(P(a_1)) * A$
Where $A$ is the advantage of rewards $P(a_1)$ is characterized by $\mu$ and $\sigma^2$ that comes out of neural network like in Pendulum environment here: https://github.com/leomzhong/DeepReinforcementLearningCourse/blob/69e573cd88faec7e9cf900da8eeef08c57dec0f0/hw4/main.py
Problem is that I cannot find any paper on 2+ continuous actions using policy gradients (not Actor-critic methods that use a different approach by transferring gradient from Q-Function).
Do you know how to do this using TRPO for 2 continuous actions in LunarLander environment? https://gym.openai.com/envs/LunarLanderContinuous-v2/
Is following approach correct for policy gradient loss function?
$L(\theta) = (log(P(a_1))+log(P(a_2)))*A$