Expected behavior of adversarial attacks on deep NN?

Question

I am trying adversarial attack (AA) for a simple CNNs. Instead of the clean image, my simple CNN is trained with attacked images as suggested by some papers. As the training goes on, I am not sure if the training is well going or something is wrong.

Here is what I observed:

When the epsilon value is large, the classification performance of the model from the adversarial training is low. I understand if the attacked image is given to the model, then the performance is poor. Although the model is from the adversarial training, because the epsilon is large, the model is poorly perform. However, when an clean image is given, the performance of the model is still low. Performance on the clean images are higher than the performance of the attacked images, but not as high as the baseline model without adversarial training.

So, I wonder if the adversarial training also degrades the performance of the model on the clean images. When I read papers, I only see the results on the adversarial Images, not clean images. If you have any experience, it will be very helpful to check if my training code is working well or not.

When the epsilon is very large, the accuracy of the model on clean image is around 15%. The model without the adversarial training is around 81%.

Some details. I use PGD attack with 5-iterations and epsilon is one of eps = [0.0001, 0.0005, 0.001, 0.005, 0.01, 0.03, 0.05, 0.07]. Step size is eps/3. Only one epsilon is selected and the adversarial training is conducted. So there are 8 different models trained with different epsilons. I use natural image Dataset.

Answer 1

This seems to be a known problem, and intuitively seems reasonable. You might be interested in the paper Adversarial Training Can Hurt Generalization.

The authors suggest that this might be because training on the perturbed data requires the model to learn more robust features, which means more samples are required to obtain performance comparable to a model that is not adversarially trained.

You could try collecting or generating additional samples to see if this leads to an improvement. They also mention that in their experiments on the MNIST dataset, using Xavier initialisation led to a significant benefit, so you could experiment with that too.