3
$\begingroup$

I am trying to understand Intagrated Gradients, but have difficulty in understanding the authors' claim (in section 3, page 3):

For most deep networks, it is possible to choose a baseline such that the prediction at the baseline is near zero ($F(x') \approx 0$). (For image models, the black image baseline indeed satisfies this property.)

They are talking about a function $F : R^n \rightarrow [0, 1]$ (in 2nd paragraph of section 3), and if you consider a deep learning classification model, the final layer would be a softmax layer. Then, I suspect for image models, the prediction at the baseline should be close to $1/k$, where $k$ is the number of categories. For CIFAR10 and MNIST, this would equal to $1/10$, which is not very close to $0$. I have a binary classification model on which I am interested in applying the Integrated Gradients algorithm. Can the baseline output of $0.5$ be a problem?

Another related question is, why did they choose a black image as the baseline in the first place? The parameters in image classification models (in a convolution layer) are typically initialized around $0$, and the input is also normalized. Therefore, image classification models do not really care about the sign of inputs. I mean we could multiply all the training and test inputs with $-1$, and the model would learn the task equivalently. I guess I can find other neutral images other than a black one. I suppose we could choose a white image as the baseline, or maybe the baseline should be all zero after normalization?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Regarding your last paragraph, in remark 1 (in the paper), they talk about the rationale behind a baseline and what it represents. $\endgroup$
    – nbro
    Jun 21, 2020 at 13:46
  • $\begingroup$ @nbro I can see that a black image does imply "the absence of the cause", and it also might be "neutral". I still think a white image or random noise for example also could represent something neutral. In fact, I asked this question because the medical images I am working on have both positive and negative pixels, and the offset is kind of arbitrary. Maybe I will see mean pixel value and use it for the baseline. $\endgroup$
    – Taro Kiritani
    Jun 22, 2020 at 4:24

1 Answer 1

Reset to default
2
$\begingroup$

You are right that the baseline score is near zero only when there are a large number of label classes, i.e., when k is large. We should have qualified this line in the paper more carefully.

In this sense, formally, the technique explains the *difference in prediction between the input score and the baseline score, as is made clear elsewhere in the paper (see Remark 1 and Proposition 1 for instance.)

Improve this answer
$\endgroup$
1
  • $\begingroup$ I see that non 0 baseline does not affect the calculation of IG. Thanks for your clarification. $\endgroup$
    – Taro Kiritani
    Jul 12, 2020 at 23:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .