In the original diffusion model paper by Sohl-Dickstein et al., they explain very little about calculating the loss and training and network to learn the diffusion process. They did publish a repository with code here, which gives a few more clues.
Now there is one thing I don't particularly understand, and that is that although the KL divergence is taken over $t=2..T$,
in the code, they sample $t=1..T-1$ and say in the comments,
# choose a timestep in [1, self.trajectory_length-1].
# note the reverse process is fixed for the very
# first timestep, so we skip it.
Now if I understand correctly, with the first timestep of the reverse process is just $t=T$, which is just the isotropic gaussian, but what I don't understand is, why do they sample from $t=1$ instead of $t=2$ like in the KL divergence.
Also, if you were indeed to sample from $t=2$, how do you learn $f_{\mu}(x^1,1),f_{\Sigma}(x^1,1)$ so that you can reverse the last step? I would expect that to come form the entropy $H_q(X^{(1)}|X^{(0)})$, but from the code we see that they replace that with the entropy of a Gaussian with $\sigma=\sqrt{1 - a_1} = \sqrt{b_1}$, $$ H_q(X^{(1)}|X^{(0)}) = \frac{1}{2}(\log 2 \pi + 1) + \frac{1}{2}\log b_1 $$
In the reverse process, you also see that they don't handle the last step of the reverse process any differently.
Summarising, how can they sample $t \in 1..T-1$, and calculate the KL divergence, while the equation specifies that $t \in 2..T$?
EDIT: After analysing the code further, it looks like they are not learning the last denoising step, and are only generating $x^{(1)}$.
