If we know the joint distribution, can we simply derive the evidence from it?

Question

I'm struggling to understand one specific part of the formalism of the free energy principle.

My understanding is that the free energy principle can be derived from considering statistical dynamics of a system that is coupled with its environment in some non-trivial way. To use a more specific instance, we can think of sensory perception as estimating a latent state given some sensory observation. Thus, the problem would be straightforwardly interpreted as Bayesian inference as the estimation of the quantity:

$$P(x \mid o) = \frac{P(o \mid x)p(x)}{P(o)}$$

The denominator $P(o)$ is intractable as it involves integrating $P(o|x) dx$ over all latent variable states $x$.

The problem is then solved as an optimization problem of fitting the variational posterior $Q(x \mid o)$ to the true posterior $P(x \mid o)$ by minimizing the KL divergence between the distributions.

The free energy then is simply the upper bound on this divergence which is the KL divergence $D(Q(x \mid o) || P(o, x))$. So our upper bound -- the free energy -- is the KL divergence between variational posterior and the joint distribution.

What confuses me is that it seems like knowing the joint distribution allows one to calculate the value $P(o)$ directly avoiding this whole minimization using the KL divergence, since $P(o, x) = P(o \mid x)P(x)$.

For reference: https://arxiv.org/abs/2107.12979 (pages 7 and 8 contain the derivation I'm referring to)

Answer 1

If the full joint distribution $p(o,x)=p(o|x)p(x)$ (usually assumed to be the product of two Gaussians for free energy principle of predictive coding, and in many practical situations you can only learn the full joint distribution alongside the variational posterior on the fly via EM algo) is allowed to calculate $p(o)$ directly, then you'd first have to know the posterior function $p(x|o)$ which is exactly what the variational inference here aims to approximate using a variational posterior $q(x|o; φ)$ with parameters $φ$ instead of the convergence slower or inefficient MCMC method, and you've already mentioned $p(o)$ is intractable directly as it requires an integration over all latent variable states.