Hot answers tagged

25 votes

In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?

The following picture that you used in your question, very accurately describes what is happening. Remember that each element of the 3D filter (grey cube) is made up of a different value (...
user avatar
15 votes

In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?

In a convolutional neural network, is there a unique filter for each input channel or are the same new filters used across all input channels? The former. In fact there is a separate kernel defined ...
user avatar
  • 24.5k
7 votes

In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?

I'm following up on the answers above with a concrete example in the hope to further clarify how the convolution works with respect to the input and output channels and the weights, respectively: Let ...
user avatar
  • 199
5 votes
Accepted

Are filters fixed or learned?

What are filters in image processing? In the context of image processing (and, in general, signal processing), the kernels (also known as filters) are used to perform some specific operation on the ...
user avatar
  • 34.3k
5 votes
Accepted

How are the kernels initialized in a convolutional neural network?

The kernels are usually initialized at a seemingly arbitrary value, and then you would use a gradient descent optimizer to optimize the values, so that the kernels solve your problem. There are many ...
user avatar
5 votes
Accepted

How to calculate the number of parameters of a convolutional layer?

What are the parameters in a convolutional layer? The (learnable) parameters of a convolutional layer are the elements of the kernels (or filters) and biases (if you decide to have them). There are ...
user avatar
  • 34.3k
4 votes

Is there anything that ensures that convolutional filters don't end up the same?

No, nothing really prevents the weights from being different. In practice though they end up almost always different because it makes the model more expressive (i.e. more powerful), so gradient ...
user avatar
  • 1,250
3 votes
Accepted

What is the need for so many filters in a CNN?

Then how do each filter differ by? Is it in hovering over the input matrix? Or is it in the values contained by filter itself? Or differs in both hovering and content? The filters (aka kernels) are ...
user avatar
  • 34.3k
3 votes
Accepted

Why is the number of output channels 16 in the hidden layer of this CNN?

I understand your question as: "How did the author select the number of neurons in their hidden layer?" The number of neurons in the hidden layer is how you can control the complexity of the function ...
user avatar
  • 450
3 votes
Accepted

Can neurons in MLP and filters in CNN be compared?

tl;dr The equivalent to a neuron in a Fully-Connected (FC) layer is the kernel (or filter) of a Convolution layer Differences The neurons of these two types of layers have two key differences. These ...
user avatar
  • 3,103
3 votes

How to calculate the number of parameters of a convolutional layer?

For a standard convolution layer, the weight matrix will have a shape of (out_channels, in_channels, kernel_sizes). In addition, you will need a vector of shape <...
user avatar
  • 1,098
3 votes

What is the difference between Attention Gate and CNN filters?

CNNs work by applying filters over the entire image. The same filter is applied at every pixel in the image. That is, the same weights are used at every pixel. Note, when I say "at every pixel&...
user avatar
3 votes
Accepted

What are some references that describe known filters (or kernels) and how we can create new ones?

I'd suggest you better understand edge detectors such as Robert or Sobel operators first to understand better how convolution operation on images extract features by constant value kernels. Would ...
user avatar
  • 1,359
3 votes

Are these visualisations the filters of the convolution layer or the convolved images with the filters?

Only the first convolutional layer, with filters that process the input [colour] channels directly, can be rendered directly as image patches in the same domain as the input. The left-most panel in ...
user avatar
  • 24.5k
2 votes

How do we choose the kernel size depending on the problem?

Take a look at this article. It give tools to actually understand what your filters have learn and show what you can do next to optimize your hyper-parameters. Also check more recent articles that ...
user avatar
  • 131
2 votes
Accepted

How is the depth of a convolutional layer determined?

The $96$ is the number of feature maps, which is equal to the number of filters/kernels. The choice of the number of kernels is not fully arbitrary, although there is no equation or exact rule ...
user avatar
  • 256
2 votes
Accepted

How can I make the kernels non-learnable and set them manually?

In most modern neural network frameworks, the update rules for training can be selectively applied to some parameters and not others. How to do that is dependent on the framework. Some will have the ...
user avatar
  • 24.5k
2 votes
Accepted

How to compute the number of weights of a CNN?

Calculating the number of parameters in a CNN is very straightforward. A CNN is composed of different filters, which are essentially 3d tensors. CNN weights are shared, meaning they are used multiple ...
user avatar
  • 1,715
2 votes
Accepted

Is it a good idea to use different width and height of the kernel in a CNN?

It depends on your application. In case of text recognition, non-uniform kernels are used since the information about text is less on the horizontal axis and more on the vertical axis. If in your case ...
user avatar
2 votes
Accepted

Is "kernel" different from "filter" in convolutional neural networks?

The term "filter" is (usually) a synonym for "kernel" in the context of convolutional neural networks and image processing. The reason why the ...
user avatar
  • 34.3k
2 votes

How to calculate number of connected neurons with filter

Firstly, step size and padding have no influence on the number of parameters, they just determine where to apply your parameters, i.e. the convolution operator. My first thought was 32 because there ...
user avatar
  • 671
1 vote

What is the need for so many filters in a CNN?

All filters move across the same area, but the filter values (also called filter kernels) are different for each filter. This makes it possible to "filter out" different features.
user avatar
  • 163
1 vote

Does replacing 3x3 filters with 3x1 and 1x3 filters improve the performance?

If the filter is separable, that is, the NxM kernel can be mathematically equal to the convolution of a Nx1 filter and a 1xM filter, there are a very important increase in performance. Using separable ...
user avatar
1 vote
Accepted

How to construct input dependent convolutional filter?

Here is one way of achieving this. This network is an autoencoder, with extra auxiliary_convs. The active convolution depends on the input image's class, since each ...
user avatar
  • 345
1 vote
Accepted

What is the intuition behind the number of filters/channels for each convolutional layer?

The channel sizes 32, 128, etc. are used because of memory and efficiency. There is nothing holy about these numbers. The intuition behind choosing the number of channels is as follows- The initial ...
user avatar
1 vote

Do all filters of the same convolutional layer need to have the same dimensions and stride?

It seems that a similar question has been raised here: https://stackoverflow.com/questions/57438922/different-size-filters-in-the-same-layer-with-tensorflow-2-0 Like answered in the link above, you ...
user avatar
1 vote
Accepted

Does the number of parameters in a convolutional neuronal network increase if the input dimension increases?

If I have a convolutional neuronal network, does the input dimension change the number of parameters? And if yes, why? If the convolutional neural network (CNN) only uses convolutional layers, then ...
user avatar
  • 34.3k
1 vote
Accepted

How does the math behind heat map filters work?

About your question concerning ColorMaps: A cv2 ColorMap is basically just a lookup table which directly maps the intensity values of the input image to a predefined RGB color. In its essences it is ...
user avatar
  • 126
1 vote

What is the point of using 1D and 2D convolutions with a kernel size of 1 and 1x1 respectively?

Traditional CNNs used for image classification (and related tasks) are composed of 1 or more fully connected layers (FCs), after the convolutional and pooling layers, which take as input the features ...
user avatar
  • 34.3k
1 vote
Accepted

How is the depth of the filters of convolutional layers determined?

Does the next convolutional filter have a depth of 40? So, would the filter dimensions be 3x3x40? Yes. The depth of the next layer $l$ (which corresponds to the number of feature maps) will be 40. If ...
user avatar
  • 34.3k

Only top scored, non community-wiki answers of a minimum length are eligible