Questions tagged [convolutional-layers]

For questions related to convolutional layers, which are layers that perform the convolution (or cross-correlation) operation.

Filter by
Sorted by
Tagged with
0 votes
1 answer
31 views

How is a filter actually applied to all input channels in a ConvLayer2D

I was studying Convolutional Layers and some of their variations and I came across this post which says: 'For rgb vs greyscale, think about channels as feature maps for input layer and a filter gets ...
0 votes
0 answers
26 views

What are the benefits of using multiple convolutions, as opposed to one, before the pooling layer in a U-Net?

I have seen U-Nets that use a single convolution before the pooling operator and some that use two or more. My question is, what is better? Or what are the benefits of using more or less convolutions?
0 votes
0 answers
16 views

How are CNN kernels trained when using FFT for convolutions?

CNNs (convolutional neural networks) are adept at processing images, as their construction is based on the biological neural networks found in the human eye. "Kernels", sometimes called &...
2 votes
1 answer
34 views

Is this aggregation of multiple convolutions of the same input a type of attention or dynamic convolution?

Are there any examples of people performing multiple convolutions at a single depth and then performing feature max aggregation as a convex combination as a form of "dynamic convolutions"? ...
  • 165
1 vote
0 answers
25 views

Order of multiple Convolutional and Pooling layers in generated CNNs

I am reading this article: https://www.sciencedirect.com/science/article/pii/S2210650221000249 There, a multi layered particle swarm optimization of CNN parameters is presented. First step (layer) is ...
  • 111
0 votes
2 answers
81 views

What do people refer to when they use the word 'dimensionality' in the context of convolutional layer?

In practical applications, we generally talk about three types of convolution layers: 1-dimensional convolution, 2-dimensional convolution, and 3-dimensional convolution. Most popular packages like ...
  • 3,241
1 vote
1 answer
153 views

How do convolutional layers of basic Graph Convolutional Networks work?

I was reading the following article on Towards Data Science (here) and it says the following, regarding the calculation of convolutional layers: So the overall steps are: Transform the graph into ...
0 votes
1 answer
198 views

How do CNNs handle inputs of different sizes and shapes?

I am new to deep learning so feel free to correct me where I am wrong. Imagine this scenario where we have a 7 * 7 input. We want to slide a 3 * 3 filter with a stride of 3 and padding of zero over ...
2 votes
2 answers
81 views

How can equivariance to translation be a benefit of a CNN?

I just learnt about the properties of equivariance and invariance to translation and other transformations. Being invariant to translation is clearly an advantage, as even if the input gets shifted, ...
  • 235
1 vote
1 answer
97 views

Given an input of shape $(3, 32, 32)$, which is convolved with a $(3 \times 3)$ kernel, how do I calculate the FLOPS?

I have an input tensor of shape $\mathbf{(3, 32, 32)}$ consisting of 3 channels, 16 rows, and 16 columns. I want to convolve the input tensor using $\mathbf{(3 \times 3)}$ kernel/filter. How can I ...
0 votes
0 answers
43 views

What is the significance behind having small kernel sizes over having one large kernel size that covers the entire input in a CNN?

I have hardly ever seen anyone cover the entire input image with a filter of the same dimensions. I was wondering why that is the case, and if the performance in say, an image detection application ...
  • 223
0 votes
2 answers
66 views

Why do we lose detail of an image as we go deeper into a ConvNet?

I was reading this research paper titled 'Image Style Transfer using Convolutional Neural Networks' which as the title suggests was based on Neural Style Transfer. I came across this line which didn't ...
0 votes
1 answer
112 views

Is "kernel" different from "filter" in convolutional neural networks?

Recently I asked a question on how a convolution 2d layer changes an RGB image into a grayscale image. Assume that our task is to convert an RGB image into a grayscale image. I use to believe that ...
  • 3,241
1 vote
1 answer
215 views

Why do we add 1 in the formula to calculate the shape of the output of the convolution?

In the formula to calculate output shape of tensor after convolution operation $$ W_2 = (W_1-F+2P)/S + 1, $$ where: $W_2$ is the output shape of the tensor $W_1$ is the input shape $F$ is the filter ...
0 votes
1 answer
312 views

Confusion about conversion of RGB image to grayscale image using a convolutional layer with 2-dimensional filters

Let us imagine $x$ as a tensor containing 1000 RGB images, each of size $64 \times 32$. ...
  • 3,241
1 vote
0 answers
111 views

Is there any animation that illustrates the "fold" and "unfold" operations of convolutional layers?

There are fourteen convolution layers in PyTorch. Among them six are related to convolution, another six are related to transposed convolution. The remaining two are fold and unfold operations. The ...
  • 3,241
2 votes
1 answer
293 views

What gets optimized in convolutional neural network?

In a convolutional neural network, the hyperparameters such as number of kernels and stride, kernel size, etc are determined. After some combination of convolutions, ReLU and pooling layer there is ...
4 votes
1 answer
248 views

Are these visualisations the filters of the convolution layer or the convolved images with the filters?

There are several images related to convolutional networks on the Internet, an example of which I have given below My question is: are these images the weights/filters of the convolution layer (the ...
2 votes
0 answers
39 views

How to implement a (3 + 2)-dimensional convolutional layer where the 2d space is "internal"?

I am trying to train a CNN to learn 5D (kind of) data. The data is structured as follows. It has three spatial dimensions [x, y, z], but it also has two "...
  • 21
1 vote
0 answers
219 views

Convolutional Layer Multichannel Backpropagation Implementation

I have been working on coding a CNN in python from scratch using numpy as a semester project and I think I have successfully implemented it up to backpropagation in the MaxPool Layers. However, my ...
2 votes
0 answers
107 views

Error in MobileNet V1 Architecture?

From the architecture table of the first MobileNet paper, a depthwise convolution with stride 2 and an input of 7x7x1024 is followed by a pointwise convolution with the same input dimensions, 7x7x1024....
1 vote
1 answer
161 views

How do you pass the image from one convolutional layer to another in a CNN?

I am currently trying to write a CNN from scratch, but I don't understand how to feed the information from a max-pooling layer to the next convolutional layer. Specifically, I don't know what to do ...
3 votes
1 answer
156 views

What is the use of the regular convolutional layer in expansion path of U-Net?

I was going through the paper on U-Net. U-net consists of a contracting path followed by an expanding path. Both the paths use a regular convolutional layer. I understand the use of convolutional ...
2 votes
0 answers
34 views

Do filters have as many layers as the depth of the input in CNNs? [duplicate]

Firstly as an example here is the architecture of YOLOv2 I am trying to understand the depth of an output of a convolutional layer. For example, the first convolutional layer has the shape 3x3x32. So ...
1 vote
0 answers
96 views

Is the 3d convolution associative given that it can be represented as matrix multiplication?

I'm trying to understand if a 3D convolution of the sort performed in a convolutional layer of a CNN is associative. Specifically, is the following true: $$ X \otimes(W \cdot Q)=(X \otimes W) \cdot Q, ...
2 votes
2 answers
970 views

What is the need for so many filters in a CNN?

Consider the following coding line related to CNNS Conv2D(64, (3,3), strides=(2, 2), padding='same') It is a convolution layer with filter size $3 \times 3$ and ...
  • 3,241
2 votes
3 answers
85 views

DQN not learning and step not stepping towards target

I am trying to create a simple Deep Q-Network with 2d convolutional layers. I can't figure out what I am doing wrong, and the only thing I can see that doesn't seem right is when I get the model ...
2 votes
1 answer
399 views

What is the intuition behind the number of filters/channels for each convolutional layer?

After having chosen the number of layers for a convolutional neural network, we must also choose the number of filters/channels for each convolutional layer. The intuition behind the filter's spatial ...
  • 23
3 votes
2 answers
587 views

Is the stride applied both in the horizontal and vertical directions in convolutional neural networks?

In the convolutional layer for CNNs, when you specify the stride of a filter, typical notes show some examples of this but only for the horizontal panning. Is this same stride applied for the vertical ...
3 votes
0 answers
52 views

What does "convolve k filters" mean in the AlphaGo paper?

On page 27 of the DeepMind AlphaGo paper appears the following sentence: The first hidden layer zero pads the input into a $23 \times 23$ image, then convolves $k$ filters of kernel size $5 \times 5$ ...
4 votes
1 answer
135 views

Do all filters of the same convolutional layer need to have the same dimensions and stride?

In Convolutional Neural Networks, do all filters of the same convolutional layer need to have the same dimensions and stride? If they don't, then it would seem the channel produced by each filter ...
  • 283
1 vote
0 answers
83 views

How can the FCNN reduce the dimensions of the input from $1048 \times 100$ to $523 \times 100$ with max-pooling?

I am trying to implement a paper on Image tempering detection and localization, the paper is Image Manipulation Detection and Localization Based on the Dual-Domain Convolutional Neural Networks, I was ...
  • 21
1 vote
0 answers
50 views

Why does the number of channels in the PointNet increase as we go deeper?

For example, in PointNet, you see the 1D convolutions with the following channels 64 -> 128 -> 1024. Why not e.g. ...
  • 530
1 vote
1 answer
116 views

Can fully connected layers be used for feature detection?

I need help in understanding something basic. In this video, Andrew Ng says, essentially, that convolutional layers are better than fully connected (FC) layers because they use fewer parameters. But I'...
  • 31
1 vote
0 answers
87 views

When to use convolutional layers as opposed to fully connected layers?

I am still new to CNNs, but I would like to check my understanding between when to use convolutional layers versus fully connected layers. From what I have read, we can use convolutional layers with ...
8 votes
2 answers
331 views

What is the point of using 1D and 2D convolutions with a kernel size of 1 and 1x1 respectively?

I understand the gist of what convolutional neural networks do and what they are used for, but I still wrestle a bit with how they function on a conceptual level. For example, I get that filters with ...
5 votes
1 answer
2k views

How to add a dense layer after a 2d convolutional layer in a convolutional autoencoder?

I am trying to implement a convolutional autoencoder with a dense layer at the bottleneck to do some dimensional reduction. I have seen two approaches for this, which aren't particularly scalable. The ...
5 votes
2 answers
6k views

How to calculate the number of parameters of a convolutional layer?

I was recently asked at an interview to calculate the number of parameters for a convolutional layer. I am deeply ashamed to admit I didn't know how to do that, even though I've been working and using ...
  • 185
1 vote
1 answer
504 views

How is the depth of the filters of convolutional layers determined? [duplicate]

I am a bit confused about the depth of the convolutional filters in a CNN. At layer 1, there are usually about 40 3x3x3 filters. Each of these filters outputs a 2d array, so the total output of the ...
1 vote
0 answers
25 views

Wasserstein GAN with non-negative weights in the critic

I want to train a WGAN where the convolution layers in the critic are only allowed to have non-negative weights (for a technical reason). The biases, nonetheless, can take both +/- values. There is no ...
  • 41
9 votes
2 answers
5k views

When should I use 3D convolutions?

I am new to convolutional neural networks, and I am learning 3D convolution. What I could understand is that 2D convolution gives us relationships between low-level features in the X-Y dimension, ...
2 votes
1 answer
220 views

Why do the inputs and outputs of a convolutional layer usually have the same depth?

Here's the famous VGG-16 model. Do the inputs and outputs of a convolutional layer, before pooling, usually have the same depth? What's the reason for that? Is there a theory or paper trying to ...
2 votes
1 answer
237 views

How do you go from the last convolutional layer to your first fully connected layer?

I'm implementing a neural network framework from scratch in C++ as a learning exercise. There is one concept I don't see explained anywhere clearly: How do you go from your last convolutional or ...
1 vote
1 answer
84 views

If I wanted to calculate multiple feature maps in a convolutional layer, should the filters be trained individually?

Assume I have an input of size $32 \times 32 \times 3$ and pass it to a convolution layer. Now, if my kernel size were to be $5 \times 5 \times 3$ and the depth of my convolution layer were to be 1, ...
0 votes
3 answers
417 views

How are the dimensions of the feature maps produced by the convolutional layer determined in VGG-16?

I'm trying to understand how the dimensions of the feature maps produced by the convolution are determined in a ConvNet. Let's take, for instance, the VGG-16 architecture. How do I get from 224x224x3 ...
4 votes
2 answers
2k views

How is the depth of the input related to the depth of the output of a convolutional layer? [duplicate]

Let's suppose I have an image with 16 channels that goes to a convolutional layer, which has 3 trainable $7 \times 7$ filters, so the output of this layer has depth 3. How does the convolutional layer ...
7 votes
3 answers
2k views

How to compute the derivative of the error with respect to the input of a convolutional layer when the stride is bigger than 1?

I read that to compute the derivative of the error with respect to the input of a convolution layer is the same to make of a convolution between deltas of the next layer and the weight matrix rotated ...