I try to solve a navigation problem with PPO; my actions space have three-part:

  1. robot linear velocity that is in [-3,3] range (getting from a tanh activation func)
  2. robot linear angular that is in [-pi/6, pi/6] range (getting from a tanh activation func)
  3. robot step-time duration that is from [0.2, 0.5, 0.8] (getting from a softmax activation func)

The problem that I face is how to calculate the ratio of probability from this separate disturbing? Mean or sum? or was there another way to calculate log_prob from different distributions? Something like log_prob from multivariable distribution!


It depends on how you have modelled the probability distributions that you are sampling from when you turn the neural network output into a specific action.

If your three action dimensions are sampled independently (which would be a standard way to model a policy like this), then the probabilities or probability densities multiply. Which means the log probabilities add.

  • $\begingroup$ I got my actions from 3 different output layers of actor-network, and each one has its own mean and std. So I must use the sum function to add log_prob and save it in the memory. $\endgroup$
    – m031n
    Aug 20 at 17:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.