Is there any use of using 3D convolutions for traditional images (like cifar10, imagenet)?

Question

I am curious if there is any advantage of using 3D convolutions on images like CIFAR-10/100 or ImageNet. I know that they are not usually used on this data set, though they could because the channel could be used as the "depth" channel.

I know that there are only 3 channels, but let's think more deeply. They could be used deeper in the architecture despite the input image only using 3 channels. So, we could have at any point in the depth of the network something like $(C_F,H,W)$ where $C_F$ is dictated by the number of filters and then apply a 3D convolution with kernel size less than $C_F$ in the depth dimension.

Is there any point in doing that? When is this helpful? When is it not helpful?

I am assuming (though I have no mathematical proof or any empirical evidence) that if the first layer aggregates all input pixels/activations and disregards locality (like a fully connected layer or conv2D that just aggregates all the depth numbers in the feature space), then 3D convolutions wouldn't do much because earlier layers destroyed the locality structure in that dimension anyway. It sounds plausible but lacks any evidence or theory to support it.

I know Deep Learning uses empirical evidence to support its claims so perhaps there is something that confirms my intuition?

Any ideas?

---

Similar posts:

• https://forums.fast.ai/t/are-there-any-successful-vision-models-that-use-3d-convolutions/52503/2

• When should I use 3D convolutions?

• Which neural network architectures are there that perform 3D convolutions?

Answer 1

As far as I've seen, there is no use of 3D CNNs for traditional image classification tasks.

The reason I think is that, while these images do have multiple channels, there is no spatial information in those channels for the 3D convolution to extract. On the other hand, it makes more sense to take the weighted sum of those pixels along that dimension (as the 2D convolution does).

3D CNNs have been used, as far as I know, only for applications where you have volumetric data, i.e. the images are sequential and, when combined, form a large 3D image.