I have read (*) that a common technique to attack a black box AI system based on a neural network is to use it to train a surrogate model to make the same classifications as the black box one.

Once this is done, one can look for adversarial examples on the surrogate model (on which the attacker has access to all the weights and can compute gradients).

The key property that makes such attacks successful is transferability: an adversarial example on the surrogate model is likely to be an adversarial example on the black box model.

Question: Do we know why such transferability properties hold and under which conditions?

(*) Although I don't remember where I first read it, this is mentioned in the book Not with a bug but with a sticker although it doesn't get into technical details (this is a book for the general public).

  • $\begingroup$ A reference or link to where you have read this information would help a lot for anyone attempting an answer $\endgroup$ Commented Feb 22 at 10:09
  • $\begingroup$ @NeilSlater Right, I added a reference in my question. $\endgroup$
    – Weier
    Commented Feb 22 at 10:19

1 Answer 1


As far as I know, there are only hypothesis on this (https://simons.berkeley.edu/talks/new-perspective-adversarial-perturbations) which pretty much relies on the idea that there are "spurious" correlation in the dataset used for training (check the video in the link for more informations, they have also some papers referenced to read more about it)


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .