How can neural networks that extract many features be fooled by adversarial images?

Question

I have been reading a bit about networks where deep layers able to deal with a bunch of features (be it edges, colours, whatever).

I am wondering: how can possibly a network based on this 'specialised' layers be fooled by adversarial images? Wouldn't the presence of specialised feature detectors be a barrier to this? (as in: this image of a gun does share one feature with 'turtles' but it lacks 9 others so: no, it ain't a turtle). thanks!

Answer 1

The link in DuttA's comment provides a good answer to the more general question of how adversarial images are generated. As to why it's possible at all, the key is that these specialized feature detectors really are just picking up features, and it's possible to construct an image that has those features, without resembling the target object much at all.

For example, we might imagine that the key features that predict an image of a gun are regions like the trigger guard and perhaps the muzzle. The features that are extracted really will just be "round-ish thing with a curved bit in it" and "perfectly circular opening". These might predict a gun really well, but it's also easy to imagine that you could place shapes satisfying those criteria the correct distance apart without including any of the other components that we'd expect. This is what you actually see in the adversarial images that are generated. Consider the bagel or crossword puzzle images below. While they are not actually pictures of the objects in question, it is fairly easy to see what the important features were for the network: a bagel is a round shape that's shadowed in the center and lighter between the center and rim. It's a bit orange. A crossword puzzle is a bunch of black and white squares distributed in a rough grid. Possibly the copyright symbol or the letter C was also important (it is found on many crosswords!).