Skip to main content
Share Your Experience: Take the 2024 Developer Survey
8 votes
Accepted

Is pooling a kind of dropout?

Dropout and Max-pooling are performed for different reasons. Dropout is a regularization technique, which affects only the training process (during evaluation, it is not active). The goal of dropout ...
Mark.F's user avatar
  • 446
8 votes
Accepted

Can non-differentiable layer be used in a neural network, if it's not learned?

It is not possible to backpropagate gradients through a layer with non-differentiable functions. However, the pooling layer function is differentiable*, and usually trivially so. For example: If an ...
Neil Slater's user avatar
  • 32.5k
6 votes
Accepted

Is a non-linear activation function needed if we perform max-pooling after the convolution layer?

Let's first recapitulate why the function that calculates the maximum between two or more numbers, $z=\operatorname{max}(x_1, x_2)$, is not a linear function. A linear function is defined as $y=f(x) =...
nbro's user avatar
  • 40.8k
5 votes
Accepted

What is the effect of using pooling layers in CNNs?

Pooling has multiple benefits Robust feature detection. Makes it computationally feasible to have deeper CNNs Robust Feature Detection Think of max-pooling (most popular) for understanding this. ...
Kaivalya Swami's user avatar
4 votes

Is max-pooling really bad?

In addition to JCP's answer I would like to add some more detail. At best, max pooling is a less than optimal method to reduce feature matrix complexity and therefore over/under fitting and improve ...
hisairnessag3's user avatar
3 votes
Accepted

Is down-sampling the only purpose of using stride?

The general purpose of stride (along with padding) is to determine the spatial dimensions of the output. So, with appropriate stride (and padding), you can also make the spatial dimensions of the ...
nbro's user avatar
  • 40.8k
3 votes

Can CNNs be applied to non-image data, given that the convolution and pooling operations are mainly applied to imagery?

You can use CNN for time-series data. The Convolutional Recurrent Neural Network (RCNN) is one of the examples. Convolutional layers basically extract features from images. It is not related to time-...
Mahir Mahbub's user avatar
3 votes
Accepted

Can CNNs be applied to non-image data, given that the convolution and pooling operations are mainly applied to imagery?

Usually, you need to ensure that your convolutions are causal, meaning that there is no information leakage from the future into the past. You could start by looking at this paper, which compares ...
razvanc92's user avatar
  • 1,148
3 votes

Is it effective to concatenate the results of mean-pooling and max-pooling?

I haven't seen it as you describe and I don't think it would be much useful. Pooling layers are being gradually phased out of networks, because they don't seem to be that useful anymore. With the ...
Javier's user avatar
  • 131
3 votes
Accepted

In which scenario would you want to have two adjacent pooling layers?

Assuming you're not referring to any particular type of pooling operation, it's possible that you could have, for example, a mean pool followed by a max or min pool. What this could do is combine the ...
juicedatom's user avatar
2 votes

Why do we have to dot product in the Low-rank Bilinear Pooling?

The paper (which I was not familiar with) proposes a new pooling method, which they name "Low-rank Bi-linear Pooling". Because it is a pooling method, its output should be a single value ...
Mark.F's user avatar
  • 446
2 votes
Accepted

How many weights does the max-pooling layer have?

A max-pooling layer doesn't have any trainable weights. It has only hyperparameters, but they are non-trainable. The max-pooling process calculates the maximum value of the filter, which consists of ...
Clement's user avatar
  • 1,745
2 votes

Is pooling a kind of dropout?

I think we would consider regularization and downsampling better in this way: dropout it puts some input value (neuron) for the next layer as 0, which makes the current layer a sparse one. So it ...
Bs He's user avatar
  • 121
2 votes
Accepted

How can we compute the gradient of max pooling with overlapping regions?

When gradients in a neural network can follow multiple paths to same parameter, the different gradient values from the sources can often be added together, because the operations in the forward ...
Neil Slater's user avatar
  • 32.5k
2 votes

Does a fully convolutional network share the same translation invariance properties we get from networks that use max-pooling?

Neural networks are not invariant to translations, but equivariant, Invariance vs Equivariance Suppose we have input $x$ and the output $y=f(x)$ of some map between spaces $X$ and $Y$. We apply ...
spiridon_the_sun_rotator's user avatar
2 votes

Is there any reason behind bias towards max pooling over avg pooling?

I've found out rather a good explanation on Quora. Max pooling extracts the most salient features - edges, cusps, whatever. Average pooling operates smoothly - collects features, that are relevant to ...
spiridon_the_sun_rotator's user avatar
2 votes
Accepted

Is there any recommended way to perform pooling in this context?

This one is a bit crazy: pool1 = nn.AvgPool3d(kernel_size = (361, 1, 1), stride= 1) because it averages large numbers of the features at once. Very little ...
Neil Slater's user avatar
  • 32.5k
2 votes
Accepted

Is there a correct order of "conv2d", "batchnorm2d", "ReLU/LeakyReLU", "MaxPool2d" for UNet like architectures?

I suggest to follow the official U-NET implementation. To me, the second option ...
Luca Anzalone's user avatar
1 vote
Accepted

In Fast R-CNN, how are input RoIs mapped to the respective RoIs in the feature map before RoI pooling?

The ROIs in the input space are mapped to the feature map space, by dividing it by the net stride at that layer. Say, in a network, after a sequence of four 2x2 pooling layers, your image is reduced ...
Arjun Ashok's user avatar
1 vote

How do you pass the image from one convolutional layer to another in a CNN?

The application of 1 kernel (aka filter) to an input (with a 2d convolution) is a matrix (a 2d array), which is often known as a feature map (aka activation map). The application of $k$ kernels to the ...
nbro's user avatar
  • 40.8k
1 vote
Accepted

Is average pooling equivalent to a strided convolution with a specific constant kernel?

Is average pooling equivalent to a strided convolution with a specific constant kernel? Yes. Why then are explicit pooling layers needed if they can be realized by convolutions? It is probably ...
nbro's user avatar
  • 40.8k
1 vote

Does a fully convolutional network share the same translation invariance properties we get from networks that use max-pooling?

FCNs can and typically have downsampling operations. For example, u-net has downsampling (more precisely, max-pooling) operations. The difference between an FCN and a regular CNN is that the former ...
nbro's user avatar
  • 40.8k
1 vote

How can max-pooling be applied to find features in words?

In an image you are pooling usually over some (n x n) set of positions which lets you maintain spatial correlation but on the other hand most 1D CNNs used for language modeling pool over the temporal ...
mshlis's user avatar
  • 2,369
1 vote

What are the benefits of using max-pooling in convolutional neural networks?

MaxPooling pools together information. Imagine you have 2 convolutional layers $(F_1, F_2)$ respectively, each with a 3x3 kernel and a stride of $1$. Also, imagine your input is $I$ is of shape $(w,h)$...
mshlis's user avatar
  • 2,369
1 vote

Is max-pooling really bad?

Max pooling isn't bad, it just depends of what are you using the convnet for. For example if you are analyzing objects and the position of the object is important you shouldn't use it because the ...
JCP's user avatar
  • 173
1 vote

In the inception neural network, how is an image of shape $224 \times 224 \times 3$ converted into one of shape $112 \times 112 \times 64$?

The padding is not size zero* in the inception CNN layers. In fact it is deliberately chosen to pad so that the convolution by itself would produce an image the same size as the original. I.e. $p=(f−1)...
Neil Slater's user avatar
  • 32.5k

Only top scored, non community-wiki answers of a minimum length are eligible