The PPO objective may include a value function error term when parameters are shared between the policy and value functions. How does this help, and when to use a neural network architecture that shares parameters between the policy and value functions, as opposed to two neural networks with separate parameters?
Think of the network as a feature extractor followed by a policy head and a value function head. The feature extractor compresses the inputs into a lower dimensional feature vector that we hypothesize will be useful to both the policy and the value function. Then you train the policy and value function with those learned features as inputs (which is in theory much easier than training both with higher dimensional inputs). This way you save a bunch of parameters and hopefully accelerate training.