Given that policies are probability distributions, in principle, you can use any metric or measure of distance that can be used to compare two probability distributions. (Note that notions of distance are not necessarily metrics in a mathematical sense).
A common measure is the Kullback–Leibler divergence (which is not a metric, in a mathematical sense, given that it does not satisfy certain required conditions for being a metric). For example, in section 4 of the PPO paper, the KL divergence is used as a regulariser (which is actually quite common, for instance, in the context of variational Bayesian neural networks). The TRPO also uses the KL divergence.
The Wasserstein metric has also been used in RL, for instance, in distributional RL (but, in this case, not to compare policies but distributions over value functions).
You can find more info about statistical distances here. The specific distance that you use may depend on the problem that you want to solve and the properties that you want your distance to have. For example, the KL divergence is unbounded above, so, if that's not desirable, you could choose another one. The paper On choosing and bounding probability metrics (2002, by Gibbs and Su) may also be useful. Here I also talk about the KL divergence and total variation.