Skip to main content
2 of 4
link to PPO paper added + title
nbro
  • 41.4k
  • 12
  • 114
  • 205

How are continuous actions sampled (or generated) from the policy network in PPO?

I am trying to understand and reproduce the Proximal Policy Optimization (PPO) algorithm in detail. One thing that I find missing in the paper introducing the algorithm is how exactly actions $a_t$ are generated given the policy network $\pi_\theta(a_t|s_t)$.

From the source code, I saw that discrete actions get sampled from some probability distribution (which I assume to be discrete in this case) parameterized by the output probabilities generated by $\pi_\theta$ given the state $s_t$.

However, what I don't understand is how continuous actions are sampled/generated from the policy network. Are they also sampled from a (probably continuous) distribution? In that case, which type of distribution is used and which parameters are predicted by the policy network to parameterize said distribution?

Also, is there any official literature that I could cite which introduces the method by which PPO generates its action outputs?

Daniel B.
  • 825
  • 1
  • 6
  • 14