Questions tagged [adversarial-ml]
For questions related to adversarial machine learning, which is a branch of machine learning focused on the study of adversarial examples, which are malicious inputs designed to fool machine learning models.
37
questions
2
votes
1
answer
66
views
Why do adversarial attack transfer well?
I have read (*) that a common technique to attack a black box AI system based on a neural network is to use it to train a surrogate model to make the same classifications as the black box one.
Once ...
- 131
1
vote
1
answer
52
views
What constitutes a 'backdoor' attack in machine learning models?
I've recently come across the term "backdoor attack" in the context of machine learning and I'm trying to understand its precise definition and characteristics. From what I gather, backdoor ...
- 3,840
0
votes
0
answers
13
views
adversarial training on convnext shows a very strange curve
i am currently working on a research project where I have to train some models for adversarial robustness. I have implemented the algorithm used by a research paper called adversarial training for ...
- 1
1
vote
1
answer
39
views
Is "The Dimpled Manifold Hypothesis" correct to say this about autoencoders?
This quite famous paper states page 3 that:
The (well-known) fact which underlies the new conceptual framework is that all the natural images are located on or near some low-dimensional manifold (as ...
- 13
0
votes
1
answer
86
views
Why don't people use their own random noise to counter adversarial attacks on computer vision systems?
Why couldn't you take the image an AI is given and apply several different random noise filters to the image and take the democratically most common response and use that for the output of the AI. As ...
- 103
0
votes
0
answers
35
views
Can we naively merge source and target datasets to train for the same task instead of performing domain adaptation?
I have seen from literature that models such as DANN or ADDA are typical in the field of Domain Adaptation, a branch of transductive learning. I know that these methods are extremely useful especially ...
- 1
2
votes
0
answers
53
views
Why do adversarial attacks work on CNNs if they classify images as humans do?
A common illustration on how CNN works is as follows: https://www.researchgate.net/figure/Learned-features-from-a-Convolutional-Neural-Network_fig1_319253577. It seems to suggest that CNN in ...
- 185
1
vote
1
answer
60
views
Do adversarial samples violate the i.i.d. assumption?
I am trying to understand why adversarial attacks work in theory.
I have read, that the image is perturbed by a special perturbation $X_{adv}=X_1+p$, but i can't find any reference on that ...
- 351
0
votes
0
answers
1k
views
What is the "attack success rate" of an Adversarial Attack?
For a typical adversarial attack, a sample $x_{0}$ is chosen from a training set belonging to class $C_{i}$ and a transformation $A$ is applied such that the adversarial example $x=A(x_{0})$ would be ...
1
vote
0
answers
68
views
Where do the objective functions proposed in this paper by Carlini-Wagner attack come from?
I'm trying to understand the paper by Carlini and Wagner on deep neural networks adversarial attacks. On page 44, in Section V-A, it is explained how the loss function to the described problem was ...
- 111
2
votes
2
answers
135
views
Why adversarial images are not the mainstream for captchas?
In order to check, whether the visitor of the page is a human, and not an AI many web pages and applications have a checking procedure, known as CAPTCHA. These tasks are intended to be simple for ...
- 2,664
1
vote
0
answers
28
views
How could an attacker poision the training data?
I came across the following definition of Backdoor attack (in this paper):
These attacks are accomplished in two steps. First, special patterns are embedded in the targeted model during the training ...
- 11
1
vote
0
answers
18
views
How could poisoning attacks be prevented in adversarial Machine Learning
How we could prevent poisoning attacks in adversarial Machine Learning?
I read it from this link and other sources. As per my understanding, poisoning could be done after the ML algorithm has been ...
- 11
1
vote
1
answer
51
views
Expected behavior of adversarial attacks on deep NN?
I am trying adversarial attack (AA) for a simple CNNs. Instead of the clean image, my simple CNN is trained with attacked images as suggested by some papers. As the training goes on, I am not sure if ...
- 113
5
votes
1
answer
592
views
Can CNNs be made robust to tricks where small changes cause misclassification?
I while ago I read that you can make subtle changes to an image that will ensure a good CNN will horribly misclassify the image. I believe the changes must exploit details of the CNN that will be used ...
- 153
2
votes
1
answer
232
views
Is there a taxonomy of adversarial attacks?
I am a medical doctor working on methodological aspects of health-oriented ML. Reproducibility, replicability, generalisability are critical in this area. Among many questions, some are raised by ...
- 23
1
vote
1
answer
102
views
Can the addition of unnoticeable noise to images be used to create subliminals?
I was reading this report: https://www.theverge.com/2017/4/12/15271874/ai-adversarial-images-fooling-attacks-artificial-intelligence
Researchers used noise to trick machine learning algorithms to ...
- 147
1
vote
0
answers
28
views
How can transition models in RL be trained adversarially?
To give a little background, I've been reading the COBRA paper, and I've reached the section that talks about the exploration policy, in particular. We figure that a uniformly random policy won't do ...
- 1,141
2
votes
1
answer
113
views
Adversarial Q Learning should use the same Q Table?
I'm creating a RF Q-Learning agent for a two player fully-observable board game and wondered, if I was to train the Q Table using adversarial training, should I let both 'players' use, and update, the ...
- 613
2
votes
0
answers
39
views
To perform a white-box adversarial attack, would the use of a numerical gradient suffice?
I am trying to perform a white-box attack on a model.
Would it be possible to simply use the numerical gradient of the output wrt input directly rather than computing each subgradient of the network ...
1
vote
0
answers
42
views
How do I decide which norm to use for placing a constraint on my adversarial perturbation?
I am performing an adversarial machine learning attack on a neural network for network traffic classification. For adding adversarial perturbations in features such as packet interarrival times and ...
- 125
1
vote
1
answer
293
views
How do you perform a gradient based adversarial attack on an SVM based model?
I have an SVM currently and want to perform a gradient based attack on it similar to FGSM discussed in Explaining And Harnessing
Adversarial Examples.
I am struggling to actually calculate the ...
3
votes
1
answer
160
views
How do I poison an SVM with manifold regularization?
I'm working on Adversarial Machine Learning, and have read multiple papers on this topic, some of them are mentioned as follows:
Poisoning Attacks on SVMs: https://arxiv.org/pdf/1206.6389.pdf
...
- 125
2
votes
1
answer
75
views
Can a trained object detection model deal with variations of the input?
Suppose an object detection algorithm is good at detecting objects and people when an object and person is close to a camera and upright. If the person walks farther away from the camera and is "...
- 21
6
votes
1
answer
142
views
What is the relationship between robustness and adversarial machine learning?
I have been reading a lot of articles on adversarial machine learning and there are mentions of "best practices for robust machine learning".
A specific example of this would be when there are ...
- 125
3
votes
1
answer
2k
views
What are causative and exploratory attacks in Adversarial Machine Learning?
I've been researching Adversarial Machine Learning and I know that causative attacks are when an attacker manipulates training data. An exploratory attack is when the attacker wants to find out about ...
- 125
5
votes
1
answer
264
views
Isn't deep fake detection bound to fail?
Deep fakes are a growing concern: the ability to credibly alter a video may have great (negative) impacts on our society. It is so much of a concern, that the biggest tech companies launched a ...
- 252
2
votes
3
answers
146
views
In adversarial machine learning, how does an attacker have access to the test and training dataset in order to poison it?
In the field of adversarial machine learning, machine learning models are vulnerable to attacks both on the test and training data set. However, how does the attacker get access to these datasets? How ...
- 125
1
vote
1
answer
198
views
How do you game an automatic trading system by messing with data, as opposed to hacking the algorithm itself?
There was a recent question on adversarial AI applications, which led me to start digging around.
Here my interest is not general, but specific:
How do you game an automatic trading system by messing ...
- 6,227
29
votes
8
answers
4k
views
Is there any research on the development of attacks against artificial intelligence systems?
Is there any research on the development of attacks against artificial intelligence systems?
For example, is there a way to generate a letter "A", which every human being in this world can recognize ...
- 433
1
vote
1
answer
44
views
Is any classifier not subject (or less susceptible) to fooling?
Is any classifier not subject to fooling as in here?
My question is related to this, but not an exact duplicate.
What I wanted to ask is that any classifiers inherently do not subject (or less prone) ...
- 101
2
votes
1
answer
136
views
Is there any research on models that make predictions by also taking into account the previous predictions?
With the recent revelation of severe limitations in some AI domains, such as self-driving cars, I notice that neural networks behave with the same sort of errors as in simpler models, i.e. they may be ...
- 203
4
votes
3
answers
323
views
What is an adversarial attack?
I'm reading this really interesting article CycleGAN, a Master of Steganography. I understand everything up until this paragraph:
we may view the CycleGAN training procedure as continually mounting ...
- 41
5
votes
2
answers
516
views
Can artificial intelligence applications be hacked? [duplicate]
Can artificial intelligence (or machine learning) applications or agents be hacked, given that they are software applications, or are all AI applications secure?
- 115
11
votes
2
answers
895
views
What tools are used to deal with adversarial examples problem?
The problem of adversarial examples is known to be critical for neural networks. For example, an image classifier can be manipulated by additively superimposing a different low amplitude image to each ...
- 299
31
votes
7
answers
5k
views
Is artificial intelligence vulnerable to hacking? [closed]
The paper The Limitations of Deep Learning in Adversarial Settings explores how neural networks might be corrupted by an attacker who can manipulate the data set that the neural network trains with. ...
- 495
87
votes
9
answers
8k
views
How is it possible that deep neural networks are so easily fooled?
The following page/study demonstrates that the deep neural networks are easily fooled by giving high confidence predictions for unrecognisable images, e.g.
How this is possible? Can you please ...
- 10.5k