8
$\begingroup$

The problem of adversarial examples is known to be critical for neural networks. For example, an image classifier can be manipulated by additively superimposing a different low amplitude image to each of many training examples that looks like noise but is designed to produce specific misclassifications.

enter image description here

Since neural networks are aapplied to some safety-critical problems (e.g. self-driving cars), I have the following question

What tools are used to ensure safety-critical applications are resistant to the injection of adversarial examples at training time?

Laboratory research aimed at developing defensive security for neural networks exists. These are a few examples.

However, do industrial-strength, production-ready defensive strategies and approaches exist? Are there known examples of applied adversarial-resistant networks for one or more specific types (e.g. for small perturbation limits)?

There are already (at least) two questions related to the problem of hacking and fooling of neural networks. The primary interest of this question, however, is whether any tools exist that can defend against some adversarial example attacks.

$\endgroup$
  • $\begingroup$ Dear Douglas, I'm sorry if I said something rude or like that. I have nothing against your answer. It will be great if you restore the answer. It's good enough, in some sense, but I just thought that it could be even better. I respect your point of view, and just wanted to understand it more. Maybe my notes seemed to be too critical, but I just wanted to add any links that support your opinion: surveys, blogs, papers etc. It's evident that someone has already published these ideas, if they are really relevant to the current situation of this research area. $\endgroup$ – Ilya Palachev Jul 25 '18 at 13:57
  • 1
    $\begingroup$ I have subscribed Google.Scholar to papers related with this area of research, and almost every week I receive new papers published on this topic on arXiv or something else. This is why I said that the literature is not thin. $\endgroup$ – Ilya Palachev Jul 25 '18 at 14:03
2
$\begingroup$

However, do industrial strength, production ready defensive strategies and approaches exist? Are there known examples of applied adversarial-resistant networks for one or more specific types (e.g. for small perturbation limits)?

I think it's difficult to tell whether or not there are any industrial strength defenses out there (which I assume would mean that they'd be reliable against all or most known methods of attacking). Adversarial Machine Learning is indeed a highly active, and growing, area of research. Not only are new approaches for defending being published quite regularly, but there is also active research into different approaches for "attacking". With new attack methods being discovered frequently, it's unlikely that anyone can already claim to have approaches that would work reliably against them all.

The primary interest of this question, however, is whether any tools exist that can defend against some adversarial example attacks.

The closest thing to a ready-to-use "tool" that I've been able to find is IBM's Adversarial Robustness Toolbox, which appears to have various attack and defense methods implemented. It appears to be in active development, which is natural considering the area of research itself is also highly active. I've never tried using it, so I can't vouch personally for the extent to which it's easily usable as a tool for industry, or if it's maybe really only still suitable for research.


Based on comments by Ilya, other frameworks that may be useful to consider are Cleverhans and Foolbox.

$\endgroup$
  • 1
    $\begingroup$ I'd like to add CleverHans and Foolbox, which are the competitors of IBM Adversarial Robustness toolbox. Do you know how these three solutions can be compared? $\endgroup$ – Ilya Palachev Jul 25 '18 at 16:21
  • $\begingroup$ @IlyaPalachev From a quick glance through their descriptions, I got the impression that those repositories only contain attack methods, not defense methods. I'm not familiar enough with them to tell for sure if that's correct, that's just the impression I got. Obviously, such repositories would still be used for benchmarking defense methods, so searching for places that cite usage of these repositories may lead to interesting defense methods. $\endgroup$ – Dennis Soemers Jul 25 '18 at 17:02
  • 1
    $\begingroup$ CleverHans guys are implementing defense sub-framework now. In recent survey about this area they also claim CleverHans to contain "reference implementations of several attack and defense procedures" (p. 85). $\endgroup$ – Ilya Palachev Jul 25 '18 at 17:43
  • 1
    $\begingroup$ @IlyaPalachev Ah I see. For the sake of completeness, I edited them into my answer, where they might be easier to see for future visitors of the site than in the comments. $\endgroup$ – Dennis Soemers Jul 25 '18 at 17:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.