Questions tagged [filters]

For questions related to the filters (also known as kernels) of a convolutional layer (of a convolutional neural network) or, in general, used in a convolution operation.

Filter by
Sorted by
Tagged with
1
vote
1answer
24 views

Graph Convolutional Networks: why are non-parametric filters not localized in space?

I was reading the following paper here about some of the groundwork in graph deep learning. On page 3, in the bit entitled Polynomial parameterization for localized filters, it states that non-...
0
votes
0answers
19 views

How to select dimensions of kernel and stride for pooling?

Consider a tensor of size $512 \times 512$. I need to reduce it to $32 \times 32$. There are several ways to do it. There are a lot of possibilities. Each possibility has its own kernel dimensions and ...
1
vote
0answers
28 views

FCNs: Questions about the filter rarefaction in the CVPR paper [Long et al., 2015]

I am reading the paper about the fully convolutional network (FCN). I had some questions about the part where the authors discuss the filter rarefaction technique (I guess this is roughly equivalent ...
1
vote
1answer
27 views

Is it possible to have different channel dimensions in a CNN?

Let's say I have two channels that I wish to feed into a CNN. One of the channel contains 4 traces and has a width of 512. Stacking them on top of each other therefore yields an image with dimensions (...
1
vote
1answer
35 views

Given an input of shape $(3, 32, 32)$, which is convolved with a $(3 \times 3)$ kernel, how do I calculate the FLOPS?

I have an input tensor of shape $\mathbf{(3, 32, 32)}$ consisting of 3 channels, 16 rows, and 16 columns. I want to convolve the input tensor using $\mathbf{(3 \times 3)}$ kernel/filter. How can I ...
0
votes
0answers
16 views

What is the significance behind having small kernel sizes over having one large kernel size that covers the entire input in a CNN?

I have hardly ever seen anyone cover the entire input image with a filter of the same dimensions. I was wondering why that is the case, and if the performance in say, an image detection application ...
0
votes
1answer
58 views

Is "kernel" different from "filter" in convolutional neural networks?

Recently I asked a question on how a convolution 2d layer changes an RGB image into a grayscale image. Assume that our task is to convert an RGB image into a grayscale image. I use to believe that ...
1
vote
0answers
73 views

What is the fundamental difference between max pooling and adaptive max pooling used in PyTorch

PyTorch provides max pooling and adaptive max pooling. Both, max pooling and adaptive max pooling, is defined in three dimensions: 1d, 2d and 3d. For simplicity, I am discussing about 1d in this ...
0
votes
2answers
35 views

Is "Pruning" only applicable to CNNs?

What Is Neural Network Pruning And Why Is It Important Today? The above article only talks about Convolutional Neural Networks: One of the first methods of pruning is pruning entire convolutional ...
3
votes
1answer
102 views

Are these visualisations the filters of the convolution layer or the convolved images with the filters?

There are several images related to convolutional networks on the Internet, an example of which I have given below My question is: are these images the weights/filters of the convolution layer (the ...
2
votes
1answer
108 views

Is it a good idea to use different width and height of the kernel in a CNN?

I always see that the width and height of the kernel are the same. But is it a good idea to use different numbers? Recently I tried to use GoogLeNet (which expects images to be 224x224) on my images (...
4
votes
0answers
52 views

Visualizing the Loss Landscape of Neural Nets: Meaning of the word 'filter'?

I found myself scratching my head when I read the following phrase in the paper Visualizing the Loss Landscape of Neural Nets: To remove this scaling effect, we plot loss functions using filter-wise ...
2
votes
0answers
29 views

Do filters have as many layers as the depth of the input in CNNs? [duplicate]

Firstly as an example here is the architecture of YOLOv2 I am trying to understand the depth of an output of a convolutional layer. For example, the first convolutional layer has the shape 3x3x32. So ...
2
votes
0answers
56 views

How will the filter size affect the transpose convolution operation?

After a series of convolutions, I am up-sampling a compressed representation, I was curious what is the methodology I should follow to choose an optimum kernel size for up-sampling. How will the ...
4
votes
1answer
92 views

Is there anything that ensures that convolutional filters don't end up the same?

I trained a simple model to recognize handwritten numbers from the mnist dataset. Here it is: ...
0
votes
1answer
403 views

What is the difference between Attention Gate and CNN filters?

Attention models/gates are used to focus/pay attention to the important regions. According to this paper, the authors describe that a model with Attention Gate (AG) can be trained from scratch. Then ...
0
votes
1answer
69 views

In CNNs, why do we sum the filter derivatives w.r.t the loss function to get the final gradient?

In a Convolutional Neural Network, unlike the fully connected layers, the same filter is used multiple times on the input while convolving - so during backpropagation, we get multiple derivatives for ...
2
votes
2answers
496 views

What is the need for so many filters in a CNN?

Consider the following coding line related to CNNS Conv2D(64, (3,3), strides=(2, 2), padding='same') It is a convolution layer with filter size $3 \times 3$ and ...
1
vote
2answers
170 views

Does replacing 3x3 filters with 3x1 and 1x3 filters improve the performance?

Recently I have come up with a VGG16 model for my binary classification task. I have relatively simple signal images Therefore (maybe?) other deeper models like ...
1
vote
1answer
60 views

How to construct input dependent convolutional filter?

I am constructing a convolutional variational autoencoder for images, starting out with mnist digits. Typically I would specify convolutional layers in the following way: ...
2
votes
1answer
139 views

What is the intuition behind the number of filters/channels for each convolutional layer?

After having chosen the number of layers for a convolutional neural network, we must also choose the number of filters/channels for each convolutional layer. The intuition behind the filter's spatial ...
3
votes
0answers
35 views

What does "convolve k filters" mean in the AlphaGo paper?

On page 27 of the DeepMind AlphaGo paper appears the following sentence: The first hidden layer zero pads the input into a $23 \times 23$ image, then convolves $k$ filters of kernel size $5 \times 5$ ...
0
votes
1answer
48 views

How can I implement 2D CNN filter with channelwise-bound kernel weights?

I would like to bind kernel parameters through channels/feature-maps for each filter. In a conv2d operation, each filter consists of HxWxC parameters I would like to have filters that have HxW ...
3
votes
1answer
66 views

Do all filters of the same convolutional layer need to have the same dimensions and stride?

In Convolutional Neural Networks, do all filters of the same convolutional layer need to have the same dimensions and stride? If they don't, then it would seem the channel produced by each filter ...
1
vote
0answers
43 views

Why does the number of channels in the PointNet increase as we go deeper?

For example, in PointNet, you see the 1D convolutions with the following channels 64 -> 128 -> 1024. Why not e.g. ...
2
votes
1answer
51 views

Does the number of parameters in a convolutional neuronal network increase if the input dimension increases?

If I have a convolutional neuronal network, does the input dimension change the number of parameters? And if yes, why? If the sizes and lengths of the filters are still the same, how can the number of ...
0
votes
1answer
197 views

How does the math behind heat map filters work?

I am working on an app that generates heat/ thermal map given a picture. i have been able to get what i expected using python opencv builtin function ...
7
votes
1answer
197 views

What is the point of using 1D and 2D convolutions with a kernel size of 1 and 1x1 respectively?

I understand the gist of what convolutional neural networks do and what they are used for, but I still wrestle a bit with how they function on a conceptual level. For example, I get that filters with ...
0
votes
1answer
200 views

What happens to the channels after the convolution layer?

I wonder what happens to the 'channels' dimension (usually 3 for RGB images) after the first convolution layer in CNNs? In books and other sources, it is always said that the depth of the output ...
4
votes
2answers
259 views

Can neurons in MLP and filters in CNN be compared?

I know they are not the same in working, but an input layer sends the input to $n$ neurons with a set of weights, based on these weights and the activation layer, it produces an output that can be fed ...
0
votes
1answer
123 views

YOLOv3 Model Structure: Why is filters = (classes + coords + 1) * num?

Here's a tutorial about doing custom training of YOLO (Darknet): https://medium.com/@manivannan_data/how-to-train-yolov3-to-detect-custom-objects-ccbcafeb13d2 The tutorial guides how to set values in ...
2
votes
1answer
179 views

How is the depth of filters of hidden layers determined?

I am a bit confused about the depth of the convolutional filters in a CNN. At layer 1, there are usually about 40 3x3x3 filters. Each of these filters outputs a 2d array, so the total output of the ...
1
vote
1answer
182 views

How can I make the kernels non-learnable and set them manually?

I'm a newbie in Convolutional Neural Networks. I have found out that kernels in convolutional layers are usually learned while training. Suppose I have a kernel that is very good to extract the ...
2
votes
1answer
55 views

What are some references that describe known filters (or kernels) and how we can create new ones?

I'm pursuing a master's degree in Artificial Intelligence. My final work is about Convolutional Neural Networks. I was looking for information about filters (or kernels) of the convolutional layers. I ...
3
votes
2answers
152 views

Are filters fixed or learned?

No matter what I google or what paper I read, I can't find an answer to my question. In a deep convolutional neural network, let's say AlexNet (Krizhevsky, 2012), filters' weights are learned by means ...
1
vote
1answer
47 views

If I wanted to calculate multiple feature maps in a convolutional layer, should the filters be trained individually?

Assume I have an input of size $32 \times 32 \times 3$ and pass it to a convolution layer. Now, if my kernel size were to be $5 \times 5 \times 3$ and the depth of my convolution layer were to be 1, ...
3
votes
1answer
571 views

How is the number of parameters reduced in the group convolution?

I think I don't understand group convolutions well. Say you have 2 groups. This means that the number of parameters would be reduced in half. So, assuming you have an image and 100 channels, with a ...
3
votes
1answer
2k views

How are the kernels initialized in a convolutional neural network?

I am currently learning about CNNs. I am confused about how filters (aka kernels) are initialized. Suppose that we have a $3 \times 3$ kernel. How are the values of this filter initialized before ...
8
votes
2answers
6k views

How do we choose the kernel size depending on the problem?

Obviously, finding suitable hyper-parameters for a neural network is a complex task and problem or domain-specific. However, there should be at least some "rules" that hold most times for the size of ...