5
$\begingroup$

I while ago I read that you can make subtle changes to an image that will ensure a good CNN will horribly misclassify the image. I believe the changes must exploit details of the CNN that will be used for classification. So we can trick a good CNN into classifying an image as a picture of a bicycle when any human would say it's an image of a dog. What do we call that technique, and is there an effort to make image classifiers robust against this trick?

$\endgroup$
2
  • $\begingroup$ This seems to be at least a partial duplicate of this: ai.stackexchange.com/q/15820/2444 (how can we generate inputs to an AI to fool them? is there any research on this?) or this ai.stackexchange.com/q/6800/2444 (in what ways could you fool an "AI", so not just a neural network, so this question seems to be too broad), or this ai.stackexchange.com/q/92/2444 (how is that possible that you can fool a NN), or this ai.stackexchange.com/q/6892/2444 (what tools are used to deal with AML). $\endgroup$
    – nbro
    Feb 18, 2021 at 19:43
  • 1
    $\begingroup$ However, currently, none of these seem to be exact duplicates, so I will leave this open for a while, because your question focuses on CNNs, and I hope that the answers below understand that. $\endgroup$
    – nbro
    Feb 18, 2021 at 20:07

1 Answer 1

5
$\begingroup$

These are known as adversarial attacks, and the specific examples that are misclassified are known as adversarial examples.

There is a reasonably large body of work on finding adversarial examples, and on making CNNs more robust (i.e. less prone to these attacks). An example is the DeepFool algorithm, which can be used to find perturbations of data which would cause the label to change.

There are several techniques in the literature which are used to fight against adversarial attacks. Here is a selection:

  1. Augmenting the training data with various random perturbations. This is intended to make the model more robust to the typical adversarial attack where random noise is added to an image. An example of this approach is discussed in [1].

  2. Constructing some sort of model to "denoise" input before feeding into the CNN. An example of this is Defense-GAN [2], which uses a generative adversarial model which models the "true" image distribution and finds an approximation of the input closer to the real distribution.

References

[1] Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy. Explaining and Harnessing Adversarial Examples. ICLR (2015). URL.

[2] Pouya Samangouei, Maya Kabkab, Rama Chellappa. Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models. ICLR (2018). URL.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .