Does anyone use Statistical Energy to monitor generative AI training?

Question

Statistical Energy (Szekely & Rizzo, 2013 or Aslan & Zech, 2005) can be used as a statistical test of whether two distributions are the same or different. It works particularly well on high dimensional datasets where other methods like the Kolmogorov–Smirnov test fail. It seems to me that this would be a good way of evaluating whether samples generated using Generative Adversarial Networks or Variational Autoencoders have similar statistical distributions to training or validation datasets. This may not be a replacement for conventional loss functions but could be a good single metric for monitoring training progress.

Is anyone doing this? If not, are there reasons this is a bad idea?

References:

Aslan, B., and Gunter Zech. "Statistical energy as a tool for binning-free, multivariate goodness-of-fit tests, two-sample comparison and unfolding." Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 537.3 (2005): 626-636.

Székely, Gábor J., and Maria L. Rizzo. "Energy statistics: A class of statistics based on distances." Journal of statistical planning and inference 143.8 (2013): 1249-1272.

Answer 1

Fréchet inception distance (FID) is a more specialized metric to evaluate performance of image generative models including GANs and VAEs, while energy statistic such as Székely's energy distance is indeed a more general-purpose metric which can be applied across different types of raw data due to main advantages you've rightly mentioned in your question.

FID is also essentially a distance between real and generated images' assumed multivariate-Gaussian feature distributions via the Inception network indirectly, which directly captures both the means (quality) and covariance (diversity) of the two distributions and is thus highly computationally efficient especially in high-dimensional feature space for image data along with its Gaussianity assumption.

On the other hand, Székely's energy distance is non-parametric requiring computing distances between all pairs of samples which is computationally expensive especially for high-dimensional big datasets. Also energy distance works on the raw data instead of abstract features for natural images thus is not practical and popular for image generation tasks. Having said that, there're some research work to leverage the built-in convexity of energy distance systematically in VAEs called RS-VAE instead of the usual sliced Wasserstein distance or KL-divergence as referenced by Turinici's Radon–Sobolev Variational Auto-Encoders, Neural Networks, Volume 141, September 2021, Pages 294-305.

Finally generally speaking loss functions of image generative models such as the common ELBO (reconstruction error and KL-Divergence) do employ ideas of statistical energy inspired from statistical mechanics, in fact ELBO is also known as negative variational free energy, a term borrowed from physics.