I'm trying to implement the proximal policy optimization (PPO) algorithm. I'm confused on how to make it work with continuous action space.
For discrete action space, the output of the network is the probability for every available action, then I choose the next action based on this probability. The ratio, in the objective function, is the ratio between the action probability of the new policy and the action probability between the old policy.
For continuous action space, from what I understand, the output of the network should be the action itself. How should this ratio (or the objective function itself) look like in that case?
