In Sutton's 2020 Reinforcement Learning text (in chapter 13.7 Policy Parameterization for Continuous Actions) it's stated

actions [may be] chosen from a normal (Gaussian) distribution.

However, I can't seem to find the justification for choosing a Gaussian distribution. It seems somewhat presumptive. I understand we choose a distribution we can sample from to model a continuous action space but not why we choose a specific distribution.

Why not fit a more complicated distribution instead of just learning the mean and std. dev. of the Gaussian?

I've noticed throughout different implementations and papers, the distribution is assumed Gaussian.

  • $\begingroup$ you may find this unsatisfactory but the answer is down to simplicity. the gaussian has lots of nice properties and is well understood. personally I think one of the main reasons is for the ease in implementing a re-parameterisation trick if needed, which is typically done in Stochastic Actor Critic. It is also nice to be able to learn directly a parameter of a distribution which corresponds to the mean IMO (but of course this isn't unique to the Gaussian distribution) $\endgroup$ Jan 7 at 11:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.