I am working on an AI system that classifies images of cats and dogs. I am concerned about the possibility of adversarial attacks, where an attacker can make small changes to an image to fool the AI system into misclassifying it. What are some techniques that I can use to make my AI system more robust to adversarial attacks?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Adversarial training, however any AI system is going to be with some degree vulnerable to adversarial attacks
However, if your attacker is only fed with the "final answer" instead of the probability distribution made by the system, it will have a very hard time understanding if the perturbation performed is effective or not
Adversarial training is nothing more than an additional data augmentation step where you perform some gradient ascent in the loss of the model of some training data points, in order to simulate the attack, and then used such augmented data as training data points
