Why don't we use diffusion for non-graph CNNs?

Question

I'm pretty new to graph neural networks, so please forgive me if this is a silly question.

Diffusion is a method used to improve graph CNNs, however it seems to me that general CNNs can also benefit from taking into consideration diffusion-like processes (for example, in CV, one may want information to diffuse from a small neighborhood, then to a bigger neighborhood, etc.). Also, PDE-based methods have been used in CV traditionally (before deep learning became a thing), so there's at least some evidence that this might work. But I did search extensively, and I can't find any information on using diffusion for general CNNs!

Therefore, what I'm curious about is: why didn't I find any information about this? Is it already explored under a different name? Or does it not really work because of some reason I don't know (or didn't think about)? Or maybe it does work, but there are some technical difficulties?

Answer 1

Just for completeness, here is one simple formalization of a diffusion GCN (Gasteiger et al.):

$\text{D-GCN}(X) = \sum_{k=1}^K A^k X W_k$

You have a diffusion factor $k \in [1 .. K]$ and you apply different GCNs with parameters $W_k$ to the $1$-hop, $2$-hop, .., $K$-hop neighborhood and sum the results together. $A^k$ denotes the $k^{th}$ power of the adjacency matrix.

There are two principles in visual CNNs that come to my mind that are comparable to the diffusion applied in graph models.

Firstly, dilated convolutions apply a kind of diffusion (they are also referred to as Atrous convolution). Specifically, the convolutional kernel is spread out based on a dilation factor $d$. The dilation factor determines the number of pixels that are skipped when applying the kernel. For example, if you have a 3x3 kernel, and apply a dilation rate of $d=1$, you get the standard convolution, a dilation rate of $d=2$ applies the 3x3 kernel to a 5x5 area of the image by using only every 2nd pixel (see the Image below). So in that case $d$ determines which pixel neighborhood the kernel is applied to, which is comparable to the $k$ in the diffusion GCN. Dilated convolution has a history in segmentation models (see Yu et al. and Chen et al.). Apparently, it helps with segmentation, because the dilated convolutions can capture the context of pixels a bit better, so segmentation maps become cleaner. It has also been applied with 1-dimensional convolutions in the WaveNet architecture. In that case dilation was used to cover greater receptive fields without increasing the number of parameters, i.e. model complexity.

The green is the output, blue is the input and the grayed pixels are the pixels where kernel weights are applied to. Image taken from here

Second, the Inception architecture by Szegedy et al. applies something that can be interpreted as a diffusion as well: Inception computes convolutions of different kernel sizes in parallel and stacks their feature maps (this blog post has a nice visualization). Here, the different kernel sizes would correspond to the $k$ and stacking is used instead of a sum.

I think these two examples are definitely related to the idea of diffusion GCN.