Return to Question

In the GradCAM paper section 3 they implicitly propose that two things are needed to understand which areas of an input image contribute most to the output class (in a multi-label classification problem). That is:

• $A^k$ the final feature maps
• $\alpha_k^c$ the average pooled partial derivatives of the output class scores $y^c$ with respect to the the final feature maps $A_k$.

The second point is clear to me. The stronger the derivative, the more important the $k$th channel of the final feature maps is.

The first point is not, because the implicit assumption is that non-zero activations have more significance than activations close to zero. I know it's tempting to take that as a given, but for me it's not so obvious. After all, neurons have biases, and a bias can arbitrarily shift the reference point, and hence what 0 means. We can easily transform two neurons [0, 1] to [1, 0] with a linear transformation.

So why should it matter which regions of the final feature maps are strongly activated?

In the GradCAM paper section 3 they implicitly propose that two things are needed to understand which areas of an input image contribute most to the output class (in a multi-label classification problem). That is:

• $A^k$ the final feature maps
• $\alpha_k^c$ the average pooled partial derivatives of the output class scores $y^c$ with respect to the the final feature maps $A_k$.

The second point is clear to me. The stronger the derivative, the more important the $k$th channel of the final feature maps is.

The first point is not, because the implicit assumption is that non-zero activations have more significance than activations close to zero. I know it's tempting to take that as a given, but for me it's not so obvious. After all, neurons have biases, and a bias can arbitrarily shift the reference point, and hence what 0 means. We can easily transform two neurons [0, 1] to [1, 0] with a linear transformation.

So why should it matter which regions of the final feature maps are strongly activated?

EDIT

To address a comment further down, this table explains why I'm thinking about magnitude rather than sign of the activations.

It comes from thinking about the possible variations of

$$ L_{Grad-CAM}^c = ReLU\bigl( \sum_k \alpha_k^c A^k \bigr) $$

EDIT 2

I've changed this question from the general version below to a more specific one.

In the GradCAM paper section 3 they implicitly propose that two things are needed to understand which areas of an input image contribute most to the output class (in a multi-label classification problem). That is:

• $A^k$ the final feature maps
• $\alpha_k^c$ the average pooled partial derivatives of the output class scores $y^c$ with respect to the the final feature maps $A_k$.

The second point is clear to me. The stronger the derivative, the more important the $k$th channel of the final feature maps is.

The first point is not, because the implicit assumption is that non-zero activations have more significance than activations close to zero. I know it's tempting to take that as a given, but for me it's not so obvious. After all, neurons have biases, and a bias can arbitrarily shift the reference point, and hence what 0 means. We can easily transform two neurons [0, 1] to [1, 0] with a linear transformation.

So why should it matter which regions of the final feature maps are strongly activated?

Original question

I've noticed there are two ways to interpret 0. They are seemingly at odds with each other so my question is around how to reconcile these two viewpoints. And if both interpretations are correct depending on context, how so?

Interpretation #1

A 0 is just as meaningful as any non-zero number. Examples and reasoning:

• When we normalize images for input to a CNN we might divide by 255, subtract the mean, and divide by the standard deviation. There will be many pixels close to 0 but they are just as meaningful as other pixels in the image.

• We can do the classic digits MNIST exercise with white digits on a black background, or black digits on a white background. It doesn't matter.

• 0 shouldn't be special because the bias component of a neuron can always shift the 0-point.

Interpretation #2

A 0 or something close to 0 is a weakly activated neuron which therefore doesn't contribute much to what happens downstream in the network. Examples and reasoning:

• We use dropout to randomly set neurons to 0 as a means of regulation.

• The deeper layers in the forward pass of a well trained network tend to have a few strong activations with most activations being close to 0.

• When interpreting a CNN we can look at the last set of feature maps and observe strongly activated pixels. For instance GradCAM relies on this idea to visualise which part of an image contributed to a prediction.

EDIT

I want to clarify my question by making it a little less philosophical. Here are some practical reasons I might care about the answer:

• Under Interpretation 2, I sometimes worry about the way I normalize my inputs. If 0 has significance, then maybe I care about which parts of my input image are close to 0, or whether or not I have white digits on a black background vs black digits on a white background. Should it matter?

• Under Interpretation 1, the GradCAM technique I mentioned above doesn't seem so sensible. Without understanding the answer to my main question, I can't really understand how GradCAM works.

In the GradCAM paper section 3 they implicitly propose that two things are needed to understand which areas of an input image contribute most to the output class (in a multi-label classification problem). That is:

• $A^k$ the final feature maps
• $\alpha_k^c$ the average pooled partial derivatives of the output class scores $y^c$ with respect to the the final feature maps $A_k$.

The second point is clear to me. The stronger the derivative, the more important the $k$th channel of the final feature maps is.

The first point is not, because the implicit assumption is that non-zero activations have more significance than activations close to zero. I know it's tempting to take that as a given, but for me it's not so obvious. After all, neurons have biases, and a bias can arbitrarily shift the reference point, and hence what 0 means. We can easily transform two neurons [0, 1] to [1, 0] with a linear transformation.

So why should it matter which regions of the final feature maps are strongly activated?