# Tag Info

Accepted

### When should I use 3D convolutions?

3D convolutions are used when you want to extract features in 3 dimensions or establish a relationship between 3 dimensions. Essentially, it's the same as 2D convolutions, but the kernel movement is ...
• 1,419
Accepted

### How can the convolution operation be implemented as a matrix multiplication?

To show how the convolution (in the context of CNNs) can be viewed as matrix-vector multiplication, let's suppose that we want to apply a $3 \times 3$ kernel to a $4 \times 4$ input, with no padding ...
• 40.9k
Accepted

### Do convolutional neural networks perform convolution or cross-correlation?

Short answer Theoretically, convolutional neural networks (CNNs) can either perform the cross-correlation or convolution: it does not really matter whether they perform the cross-correlation or ...
• 40.9k

### When should I use 3D convolutions?

3D convolutions should be used when you want to extract spatial features from your input on 3 dimensions. For computer vision, they are typically used on volumetric images, which are 3D. Some examples ...
• 201

### What is the difference between graph convolution in the spatial vs spectral domain?

Spectral Convolution In a spectral graph convolution, we perform an Eigen decomposition of the Laplacian Matrix of the graph. This Eigen decomposition helps us in understanding the underlying ...
Accepted

### What is a filter in the context of graph convolutional networks?

Short answer Check out the paper of Shuman et al. [1], it provides some background on Graph Signal Processing, including answers to your questions in sections II.C and III.A Long Answer Question 1 Yes,...

### Is there anything that ensures that convolutional filters don't end up the same?

No, nothing really prevents the weights from being different. In practice though they end up almost always different because it makes the model more expressive (i.e. more powerful), so gradient ...
• 1,600

### What does 'input planes' mean in the phrase 'input signal/image composed of several input planes'?

Yes, it is a bit misleading. What it really means is input channels, so it would be: nn.Conv2d: Applies a 2D convolution over an input signal composed of several input channels. So, why don't just use ...
• 1,148

### Why would you implement the position-wise feed-forward network of the transformer with convolution layers?

I'm going to post another guess to this question - it won't be a complete answer, but hopefully it'll provide some direction towards finding a more legitimate answer. The feed-forward networks as ...
• 91

### Does each filter in each convolution layer create a new image?

For a 3 channel image (RGB), each filter in a convolutional layer computes a feature map which is essentially a single channel image. Typically, 2D convolutional filters are used for multichannel ...
• 156
Accepted

### Does each filter in each convolution layer create a new image?

You are partially correct. On CNNs the output shape per layer is defined by the amount of filters used, and the application of the filters (dilation, stride, padding, etc.). CNNs shapes In your ...
• 156

### Does each filter in each convolution layer create a new image?

About the images inside the CNN layers: I really recommend this article since there is no one short answer to this question and it probably will be better to experiment with it. About the RGB input ...
Accepted

### How is the convolution layer is usually implemented in practice?

I don't think that to understand convolution you need to dig into the nested code of huge libraries, since the code becomes quickly really hard to understand and convoluted (ba dum tsss!). Joking ...
• 5,348
Accepted

### When should we use separable convolution?

Context of the question This is a link to the text cited in the question. It refers to the usage of SeparableConv2D (tf, keras name). A related question on StackOverflow is "What is the ...
• 1,293

### Are there any advantages of the local attention against convolutions?

It is true that when using local attention with a window of size 5, the "receptive field" is the same as a CNN with kernel size 5 (or two CNN layers with kernel size 3). However, there is a ...
• 161
Accepted

### Can I shuffle image channel data as a form of data augmentation?

As a rule of thumb for image data augmentation, look at the augmented images: Can you correctly classify or measure your target label from the augmented images? Could something similar to the ...
• 32.7k

### Can CNNs be applied to non-image data, given that the convolution and pooling operations are mainly applied to imagery?

You can use CNN for time-series data. The Convolutional Recurrent Neural Network (RCNN) is one of the examples. Convolutional layers basically extract features from images. It is not related to time-...
Accepted

### Can CNNs be applied to non-image data, given that the convolution and pooling operations are mainly applied to imagery?

Usually, you need to ensure that your convolutions are causal, meaning that there is no information leakage from the future into the past. You could start by looking at this paper, which compares ...
• 1,148
Accepted

### How do I optimize the number of filters in a convolution layer?

Am I right in thinking that because there are only newImageX * newImageY patterns in the 32 x 32 image, that the maximum amount ...
• 40.9k

### Why would you implement the position-wise feed-forward network of the transformer with convolution layers?

1) The math is the exact same, so from an optimization or mathematical perspective there is no difference 2) Here are my guesses to a possible answer. Habit: People may just call one over the ...
• 2,379

### Wouldn't convolutional neural network models work better without flattening the input in any stages?

I have had similar thoughts about neural networks before. Convolution layers are layers of two dimensional nodes effectively passing the spacial data so why don't we use two dimensional hidden layers ...
Accepted

• 953

### What is the use of the regular convolutional layer in expansion path of U-Net?

The point is that in the expansive path you have two forms of information: the information from the contracting path, which includes all high-level features extracted from the original image. the ...
• 3,193

### What is the difference between same convolution and full convolution in terms of feature map size?

What happens to the size of output feature map in case of full convolution? It increases. First one is valid padding: the blue square is not padded, so the green square is smaller. Third one is ...
• 149
Accepted

### How is the convolution operation connected to neural networks?

The convolution operation performed by most CNNs that you will find (on the web) assumes that the signals/functions are discrete and 2-dimensional (e.g. images can be viewed as 2-dimensional discrete ...
• 40.9k
Accepted

### Is down-sampling the only purpose of using stride?

The general purpose of stride (along with padding) is to determine the spatial dimensions of the output. So, with appropriate stride (and padding), you can also make the spatial dimensions of the ...
• 40.9k