In Proximal Policy Optimization Algorithms (2017), Schulman et al. write
With this scheme, we only ignore the change in probability ratio when it would make the objective improve, and we include it when it makes the objective worse.
I don't understand why the clipped surrogate objective works. How can it work if it doesn't take into account the objective improvements?