Skip to main content
Share Your Experience: Take the 2024 Developer Survey
27 votes

In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?

The following picture that you used in your question, very accurately describes what is happening. Remember that each element of the 3D filter (grey cube) is made up of a different value (...
Mohsin's user avatar
  • 982
16 votes

In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?

In a convolutional neural network, is there a unique filter for each input channel or are the same new filters used across all input channels? The former. In fact there is a separate kernel defined ...
Neil Slater's user avatar
  • 32.5k
8 votes

In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?

I'm following up on the answers above with a concrete example in the hope to further clarify how the convolution works with respect to the input and output channels and the weights, respectively: Let ...
Lukas Z.'s user avatar
  • 209
6 votes
Accepted

How to calculate the number of parameters of a convolutional layer?

What are the parameters in a convolutional layer? The (learnable) parameters of a convolutional layer are the elements of the kernels (or filters) and biases (if you decide to have them). There are ...
nbro's user avatar
  • 40.8k
6 votes
Accepted

Are filters fixed or learned?

What are filters in image processing? In the context of image processing (and, in general, signal processing), the kernels (also known as filters) are used to perform some specific operation on the ...
nbro's user avatar
  • 40.8k
5 votes
Accepted

How are the kernels initialized in a convolutional neural network?

The kernels are usually initialized at a seemingly arbitrary value, and then you would use a gradient descent optimizer to optimize the values, so that the kernels solve your problem. There are many ...
Mustafa Radha's user avatar
5 votes
Accepted

Why is the number of output channels 16 in the hidden layer of this CNN?

I understand your question as: "How did the author select the number of neurons in their hidden layer?" The number of neurons in the hidden layer is how you can control the complexity of the function ...
JahKnows's user avatar
  • 470
5 votes

Is there anything that ensures that convolutional filters don't end up the same?

No, nothing really prevents the weights from being different. In practice though they end up almost always different because it makes the model more expressive (i.e. more powerful), so gradient ...
user3667125's user avatar
  • 1,570
4 votes
Accepted

Can neurons in MLP and filters in CNN be compared?

tl;dr The equivalent to a neuron in a Fully-Connected (FC) layer is the kernel (or filter) of a Convolution layer Differences The neurons of these two types of layers have two key differences. These ...
Djib2011's user avatar
  • 3,193
4 votes

Are these visualisations the filters of the convolution layer or the convolved images with the filters?

Only the first convolutional layer, with filters that process the input [colour] channels directly, can be rendered directly as image patches in the same domain as the input. The left-most panel in ...
Neil Slater's user avatar
  • 32.5k
3 votes

How to calculate the number of parameters of a convolutional layer?

For a standard convolution layer, the weight matrix will have a shape of (out_channels, in_channels, kernel_sizes). In addition, you will need a vector of shape <...
razvanc92's user avatar
  • 1,148
3 votes
Accepted

What are some references that describe known filters (or kernels) and how we can create new ones?

I'd suggest you better understand edge detectors such as Robert or Sobel operators first to understand better how convolution operation on images extract features by constant value kernels. Would ...
ashenoy's user avatar
  • 1,419
3 votes

In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?

I recommend chapter 2.2.1 of my masters thesis as an answer. To add to the remaining answers: Keras is your friend to understand what happens: ...
Martin Thoma's user avatar
  • 1,055
3 votes
Accepted

How is the depth of a convolutional layer determined?

The $96$ is the number of feature maps, which is equal to the number of filters/kernels. The choice of the number of kernels is not fully arbitrary, although there is no equation or exact rule ...
daniel451's user avatar
  • 266
3 votes
Accepted

What is the need for so many filters in a CNN?

Then how do each filter differ by? Is it in hovering over the input matrix? Or is it in the values contained by filter itself? Or differs in both hovering and content? The filters (aka kernels) are ...
nbro's user avatar
  • 40.8k
3 votes

What is the difference between Attention Gate and CNN filters?

CNNs work by applying filters over the entire image. The same filter is applied at every pixel in the image. That is, the same weights are used at every pixel. Note, when I say "at every pixel&...
a crazy Minion's user avatar
2 votes

How is the depth of a convolutional layer determined?

Let's say you have an image with $3$ channels and you have $10$ filters, where each filter has the shape $5 \times 5 \times 3$. The depth of the convolutional layer after having applied this filter to ...
GPrathap's user avatar
  • 121
2 votes

How do we choose the kernel size depending on the problem?

Take a look at this article. It give tools to actually understand what your filters have learn and show what you can do next to optimize your hyper-parameters. Also check more recent articles that ...
Robin's user avatar
  • 131
2 votes
Accepted

If I wanted to calculate multiple feature maps in a convolutional layer, should the filters be trained individually?

Each feature map (or kernel) is independent of each other. If you had $3$ of these filters, your output shape would be $(28, 28, 3)$ (given the appropriate amount of padding and stride) with a total ...
mshlis's user avatar
  • 2,369
2 votes

How is the number of parameters reduced in the group convolution?

The number of parameters is filters*input_channels*output_channels Groups are formed among input and output channels. So instead of ...
j314erre's user avatar
  • 121
2 votes
Accepted

How can I make the kernels non-learnable and set them manually?

In most modern neural network frameworks, the update rules for training can be selectively applied to some parameters and not others. How to do that is dependent on the framework. Some will have the ...
Neil Slater's user avatar
  • 32.5k
2 votes
Accepted

How to compute the number of weights of a CNN?

Calculating the number of parameters in a CNN is very straightforward. A CNN is composed of different filters, which are essentially 3d tensors. CNN weights are shared, meaning they are used multiple ...
Clement's user avatar
  • 1,745
2 votes
Accepted

How many weights does the max-pooling layer have?

A max-pooling layer doesn't have any trainable weights. It has only hyperparameters, but they are non-trainable. The max-pooling process calculates the maximum value of the filter, which consists of ...
Clement's user avatar
  • 1,745
2 votes
Accepted

What is the intuition behind the number of filters/channels for each convolutional layer?

The channel sizes 32, 128, etc. are used because of memory and efficiency. There is nothing holy about these numbers. The intuition behind choosing the number of channels is as follows- The initial ...
cybershiptrooper's user avatar
2 votes
Accepted

Is "kernel" different from "filter" in convolutional neural networks?

The term "filter" is (usually) a synonym for "kernel" in the context of convolutional neural networks and image processing. The reason why the ...
nbro's user avatar
  • 40.8k
2 votes

How to calculate number of connected neurons with filter

Firstly, step size and padding have no influence on the number of parameters, they just determine where to apply your parameters, i.e. the convolution operator. My first thought was 32 because there ...
Chillston's user avatar
  • 1,748
2 votes
Accepted

Which face filter algorithms can work on CPU or integrated GPU?

Face filters works by first detecting and localizing the face, then predicting the so called facial landmarks (a set of points that depict the geometry of the face, like its contour, shape of eyes, ...
Luca Anzalone's user avatar
1 vote
Accepted

How to construct input dependent convolutional filter?

Here is one way of achieving this. This network is an autoencoder, with extra auxiliary_convs. The active convolution depends on the input image's class, since each ...
NikoNyrh's user avatar
  • 777
1 vote

Do all filters of the same convolutional layer need to have the same dimensions and stride?

It seems that a similar question has been raised here: https://stackoverflow.com/questions/57438922/different-size-filters-in-the-same-layer-with-tensorflow-2-0 Like answered in the link above, you ...
Marco Prata's user avatar
1 vote
Accepted

Does the number of parameters in a convolutional neuronal network increase if the input dimension increases?

If I have a convolutional neuronal network, does the input dimension change the number of parameters? And if yes, why? If the convolutional neural network (CNN) only uses convolutional layers, then ...
nbro's user avatar
  • 40.8k

Only top scored, non community-wiki answers of a minimum length are eligible