0
$\begingroup$

In the 2015 paper "Deep Unsupervised Learning using Nonequilibrium Thermodynamics" by Sohl-Dickstein et al. on diffusion for generative models, Figure 1 shows the forward trajectory for a 2-d swiss-roll image using Gaussian diffusion. The thin lines are gradually blurred into wider and fuzzier lines, and eventually into an identity-covariance Gaussian. Table App.1 gives the diffusion kernel as:

$$ q(\mathbf{x}^{(t)} \mid \mathbf{x}^{(t-1)}) = \mathcal{N}(\mathbf{x}^{(t)} ; \mathbf{x}^{(t-1)} \sqrt{1 - \beta_t}, \mathbf{I} \beta_t ) $$

The covariance of the diffusion kernel is diagonal, so each component $x_i^{(t)}$ (i.e., each pixel in the image at time step $t$) is independently sampled from a 1-d Gaussian based on the prior time step's pixel value at the same x-y location in the image. So a given pixel should NOT diffuse into neighboring pixels; instead, the action of the diffusion step is a linear Gaussian 1-d transformation of the number held in the pixel, with the mean slightly reduced and some noise added.

Question: This seems inconsistent with Figure 1? Instead of the blurred line (wider and fuzzier line), we should have a line that has the same width, but exhibits more noise? In order to have a pixel diffuse into neighboring pixels, we would need a diffusion kernel with a non-diagonal covariance, so that there is nonzero covariance between components?

$\endgroup$

1 Answer 1

0
$\begingroup$

Figure 1. The proposed modeling framework trained on 2-d swiss roll data

The data is actually 2d coordinates, plotted on a scatter-plot. It doesn't represent an $n \times n$ image with individual pixels. And the added noise is 2d, well 1d to the x-coordinate and 1d to the y-coordinate, if you consider them separately.

But when the process is applied to images, as seen in Figure 3, it behaves as you described and expected the result to look like.

$\endgroup$
1
  • $\begingroup$ I did not realize that "2-d swiss roll data" is actually represented as a (x,y) scatter plot, thanks very much!! $\endgroup$ Apr 14, 2023 at 5:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .