1
$\begingroup$

I am trying to understand Intagrated Gradients, but have difficulty in understanding the authors' claim (in section 3, page 3):

For most deep networks, it is possible to choose a baseline such that the prediction at the baseline is near zero ($F(x') \approx 0$). (For image models, the black image baseline indeed satisfies this property.)

They are talking about a function $F : R^n \rightarrow [0, 1]$ (in 2nd paragraph of section 3), and if you consider a deep learning classification model, the final layer would be a softmax layer. Then, I suspect for image models, the prediction at the baseline should be close to $1/k$, where $k$ is the number of categories. For CIFAR10 and MNIST, this would equal to $1/10$, which is not very close to $0$. I have a binary classification model on which I am interested in applying the Integrated Gradients algorithm. Can the baseline output of $0.5$ be a problem?

Another related question is, why did they choose a black image as the baseline in the first place? The parameters in image classification models (in a convolution layer) are typically initialized around $0$, and the input is also normalized. Therefore, image classification models do not really care about the sign of inputs. I mean we could multiply all the training and test inputs with $-1$, and the model would learn the task equivalently. I guess I can find other neutral images other than a black one. I suppose we could choose a white image as the baseline, or maybe the baseline should be all zero after normalization?

$\endgroup$
  • 1
    $\begingroup$ Regarding your last paragraph, in remark 1 (in the paper), they talk about the rationale behind a baseline and what it represents. $\endgroup$ – nbro Jun 21 at 13:46
  • $\begingroup$ @nbro I can see that a black image does imply "the absence of the cause", and it also might be "neutral". I still think a white image or random noise for example also could represent something neutral. In fact, I asked this question because the medical images I am working on have both positive and negative pixels, and the offset is kind of arbitrary. Maybe I will see mean pixel value and use it for the baseline. $\endgroup$ – Taro Kiritani Jun 22 at 4:24
1
$\begingroup$

You are right that the baseline score is near zero only when there are a large number of label classes, i.e., when k is large. We should have qualified this line in the paper more carefully.

In this sense, formally, the technique explains the *difference in prediction between the input score and the baseline score, as is made clear elsewhere in the paper (see Remark 1 and Proposition 1 for instance.)

| improve this answer | |
$\endgroup$
  • $\begingroup$ I see that non 0 baseline does not affect the calculation of IG. Thanks for your clarification. $\endgroup$ – Taro Kiritani Jul 12 at 23:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.