For example, AFAIK, the pooling layer in a CNN is not differentiable, but it can be used because it's not learning. Is it always true?


1 Answer 1


It is not possible to backpropagate gradients through a layer with non-differentiable functions. However, the pooling layer function is differentiable*, and usually trivially so.

For example:

  • If an average pooling layer has inputs $z$ and outputs $a$, and each output is average of 4 inputs then $\frac{da}{dz} = 0.25$ (if pooling layers overlap it gets a little more complicated, but you just add things up where they overlap).

  • A max pooling layer has $\frac{da}{dz} = 1$ for the maximum z, and $\frac{da}{dz} = 0$ for all others.

A pooling layer usually has no learnable parameters, but if you know the gradient of a function at its outputs, you can assign gradient correctly to its inputs using the chain rule. That is essentially all that back propagation is, the chain rule applied to the functions of a neural network.

To answer your question more directly:

Can non-differentiable layer be used in a neural network, if it's not learned?


There is one exception: If this layer appears directly after the input, then as it has no parameters to learn, and you generally do not care about the gradient of the input data, so you can have a non-differentiable function there. However, this is just the same as transforming your input data in some non-differentiable way, and training the NN with that transformed data instead.

* Technically there are some discontinuities in the gradient of a max function (where any two inputs are equal). However, this is not a problem in practice, as the gradients are well behaved close to these values. When you can safely do this or not is probably the topic of another question.

  • 1
    $\begingroup$ To describe a function as differentiable means that it is differentiable at all points in its domain, and this is not true of the max-pooling function. If $f(x) = \max_i f_i(x)$, then $f$ is typically not a differentiable function, even if each function $f_i$ is. If we look at the graph of $f$ we will see some sharp corners. $\endgroup$
    – littleO
    Jan 8, 2019 at 17:00
  • $\begingroup$ @littleO: Those sharp points are not important to gradient descent by backpropagation in practice, and you can take whichever of the two overlapping values you prefer as the value of the "gradient" - everything will work fine. I will try to add that detail somehow without changing the flow of the answer. $\endgroup$ Jan 8, 2019 at 20:08
  • $\begingroup$ This answer is related to the math.stackexchange question here: math.stackexchange.com/questions/2837737/… $\endgroup$
    – NicNic8
    Jan 8, 2019 at 20:09
  • $\begingroup$ @NicNic8: Yes, but note that the question here is not really about the maths - the OP thought that the function was not differentiable at all, whilst it is entirely possible to use the chain rule in domains of the input functions that are differentiable. Further to that, it is not even very important in this case if we hit a non-differentiable point, we can safely patch it. That may not be the case for all functions or for all situations where you want to use the chain rule . . . but that's a different question $\endgroup$ Jan 8, 2019 at 20:19
  • 1
    $\begingroup$ Thanks. I think it's worth being careful about this point, because there are examples where gradient descent (with exact like search) fails to converge to a local minimizer despite never encountering a nondifferentiable point along the way. (Such an example can be found in Vandenberghe's 236c notes.) So, it seems to me that there is a serious theory question about how well we can expect gradient descent to perform when the objective is nondifferentiable, even if the objective is differentiable almost everywhere. If the goal is to grock deep learning, I think this point should be grappled with. $\endgroup$
    – littleO
    Jan 8, 2019 at 21:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .