0
$\begingroup$

I was having looking at this lecture by Ian Goodfellow and my doubt is around 18:00 timestamp where he explains generation of adversarial examples using FGSM.

He mentions that the there is a linear relationship between the input to the model and the output as the activation functions are piece-wise linear with a small number of pieces. I'm not very clear what he means by input and output. Is he referring to inputs and outputs of a single layer or the input image and final output?

He states that the relation between the parameters (weights) of a model and the output are non-linear which is what makes it difficult to train a neural network, thus it is much easier to find an adversarial example.

Could someone explain what is linear in what? and how linearity helps in adversarial example construction?

EDIT: As per my understanding FGSM method relies on the linearity of the loss function with respect to the input image. It constructs an adversarial example by perturbing the input in the direction of the gradient of the loss function w.r.t image. I am not able to understand why this works?

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
1
$\begingroup$

input to output is linear refers to the input X i.e image and output is the output logits/softmax from the network.

So how does linearity help in constructing adversarial examples? Imagine a simple logistic regressor and a simple 2D space. There is a definite boundary beyond which the label that the model(i.e logisitc regressor in this case) changes. So if we move perpendicular to the boundary (i.e the line represented by the model in this case) we can get to another class's space. So if we perturb the input in this direction, the model outputs wrong class. { Refer the slide with the title Adversarial Examples from Excessive Linearity for diagram }

Now imagine the neural network trained on imagenet, there are so many boundaries and a small change can just shift change the class the model would predict. Now it is important to note that these subspaces of the image remain nearly the same if we train VGG or ResNet etc. So this explains how an adversarial example on one network effects another.

You may ask how such small change can effect. this is because the vectors we deal it is not 2d or 3d, it is very large and small changes add up.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Computing the derivative of the loss function w.r.t the image would involve similar steps as of back-prop just that we would have the propagate the gradients from the 1st hidden layer to the input. In that case doesn't the gradient w.r.t image involve similar amount of non-linearity as the weights itself? $\endgroup$
    – ANIRUDH BUVANESH
    Commented Jan 1, 2021 at 10:09
  • $\begingroup$ Let's take a simple nn, 1st we multiply W.x, now multiplication is linear. Next say we apply ReLUs, they are linear too in the majority of the domain. Even sigmoid can be nearly considered linear in the non-saturating region. So we say dnn are very piecewise linear. i.e overall we can assume they're linear. Yes some amount of nonlinearity is present but experiments have been done to show they act linearly. Note that I am only talking about the linearity of the network, not that of loss function. $\endgroup$
    – Hrushi
    Commented Jan 1, 2021 at 10:21
  • $\begingroup$ I get what you're saying, but the FGSM takes a step in the direction of the gradient of the loss w.r.t input image. And as you said if the inter-class decision boundary is linear then on moving farther in this direction would yield an adversarial example. Doesn't FGSM rely on linearity of gradients w.r.t image and not gradients of outputs w.r.t input [which is linear somewhat by your above explaination]. $\endgroup$
    – ANIRUDH BUVANESH
    Commented Jan 1, 2021 at 10:44
  • $\begingroup$ The input image is like a reference, we don't take any gradient with respect to it, rather with the perturbation that is added to the input image. By the way, your question in title and OP are different. It would help if you can be a bit clear. $\endgroup$
    – Hrushi
    Commented Jan 1, 2021 at 12:50
  • $\begingroup$ Have edited the question. $\endgroup$
    – ANIRUDH BUVANESH
    Commented Jan 1, 2021 at 13:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .