61
votes
Accepted
How is it possible that deep neural networks are so easily fooled?
First up, those images (even the first few) aren't complete trash despite being junk to humans; they're actually finely tuned with various advanced techniques, including another neural network.
The ...
- 2,599
29
votes
Accepted
Is there any research on the development of attacks against artificial intelligence systems?
Yes, there is some research on this topic, which can be called adversarial machine learning, which is more an experimental field.
An adversarial example is an input similar to the ones used to train ...
- 40.5k
28
votes
How is it possible that deep neural networks are so easily fooled?
The images that you provided may be unrecognizable for us. They are actually the images that we recognize but evolved using the Sferes evolutionary framework.
While these images are almost impossible ...
- 1,072
21
votes
Is artificial intelligence vulnerable to hacking?
AI is vulnerable from two security perspectives the way I see it:
The classic method of exploiting outright programmatic errors to achieve some sort of code execution on the machine that is running ...
17
votes
How is it possible that deep neural networks are so easily fooled?
All answers here are great, but, for some reason, nothing has been said so far on why this effect should not surprise you. I'll fill the blank.
Let me start with one requirement that is absolutely ...
- 1,957
12
votes
Is there any research on the development of attacks against artificial intelligence systems?
Sometimes if the rules used by an AI to identify characters are discovered, and if the rules used by a human being to identify the same characters are different, it is possible to design characters ...
- 363
11
votes
Is there any research on the development of attacks against artificial intelligence systems?
Yes there are, for instance one pixel attacks described in
Su, J.; Vargas, D.V.; Kouichi, S. One pixel attack for fooling deep
neural networks. arXiv:1710.08864
One pixels attacks are attacks in ...
- 211
11
votes
How is it possible that deep neural networks are so easily fooled?
An important question that does not yet have a satisfactory answer in neural network research is how DNNs come up with the predictions they offer. DNNs effectively work (though not exactly) by ...
- 1,315
7
votes
Is artificial intelligence vulnerable to hacking?
Programmer vs Programmer
It's a "infinity war": Programmers vs Programmers. All thing can be hackable. Prevention is linked to the level of knowledge of the professional in charge of security and ...
- 568
6
votes
Is artificial intelligence vulnerable to hacking?
How we can prevent it?
There are several works about AI verification. Automatic verifiers can prove the robustness properties of neural networks. It means that if the input X of the NN is perturbed ...
- 299
5
votes
Is there any research on the development of attacks against artificial intelligence systems?
Here's an example:
How to hack your face to dodge the rise of facial recognition tech
In his recent book The Fall, Stephenson wrote about smartglasses that that project a pattern over the facial ...
- 6,237
5
votes
How is it possible that deep neural networks are so easily fooled?
How is it possible that deep neural networks are so easily fooled?
Deep neural networks are easily fooled by giving high confidence predictions for unrecognizable images. How is this possible? Can you ...
- 632
5
votes
Can artificial intelligence applications be hacked?
To answer your question, it really depends on the purpose of the Artificial Intelligence program.
For example, 4Chan has hacked a number of "Artificial Intelligent" bots, most notably was Microsoft's ...
- 151
5
votes
Accepted
Isn't deep fake detection bound to fail?
Not necessarily it depends on the function of the problem space for both the GANs.
A real world example: a batter's reaction time and a pitchers max speed are actual bounded values based on genetics ...
- 555
5
votes
Accepted
What are causative and exploratory attacks in Adversarial Machine Learning?
When someone is able to do a causative attack it means there is a mechanism by which they are able to input data into the network. Maybe a website where people can input their images and it outputs a ...
- 555
5
votes
Accepted
Can CNNs be made robust to tricks where small changes cause misclassification?
These are known as adversarial attacks, and the specific examples that are misclassified are known as adversarial examples.
There is a reasonably large body of work on finding adversarial examples, ...
- 1,010
4
votes
Accepted
What tools are used to deal with adversarial examples problem?
However, do industrial strength, production ready defensive strategies and approaches exist? Are there known examples of applied adversarial-resistant networks for one or more specific types (e.g. for ...
- 10.3k
4
votes
Is artificial intelligence vulnerable to hacking?
Is Artificial Intelligence Vulnerable to Hacking?
Invert your question for a moment and think:
What would make AI at less of a risk of hacking compared to any other
kind of software?
At the end ...
- 210
4
votes
Is artificial intelligence vulnerable to hacking?
I believe it is, no system is safe, however I am not sure if I can still say this after 20-30 years of AI development/evolution. Anyways, there are articles that showed humans fooling AI (Computer ...
- 41
4
votes
How is it possible that deep neural networks are so easily fooled?
Can't comment(due to that required 50 rep), but I wanted to make a response to Vishnu JK and the OP. I think you guys are skipping the fact that the neural network only really is saying truly from a ...
- 51
4
votes
Is there any research on the development of attacks against artificial intelligence systems?
There are many insightful comments and answers so far. I want to illustrate my idea of "color blindness test" more. Maybe it's a hint to lead us to the truth.
Imagine there are two people here. One ...
- 433
4
votes
Is there any research on the development of attacks against artificial intelligence systems?
Isn't that essentially what chess does? For example, A human can recognize that a Ruy exchange offers white great winning chances (because of pawn structure) by move 4 while an engine would take ...
- 41
4
votes
Accepted
Is there any research on models that make predictions by also taking into account the previous predictions?
What you're describing is called a recurrent neural network. There are a large number of designs in this family that all have the ability to remember recent inputs and use them in the processing of ...
- 9,267
4
votes
Accepted
What is the relationship between robustness and adversarial machine learning?
A robust ML model is one that captures patterns that generalize well in the face of the kinds of small changes that humans expect to see in the real world.
A robust model is one that generalizes ...
- 9,267
3
votes
Is there any research on the development of attacks against artificial intelligence systems?
Here's a live demo: https://www.labsix.org/physical-objects-that-fool-neural-nets/
Recall that neural nets are trained by feeding in the training data, evaluating the net, and using the error between ...
- 131
3
votes
How is it possible that deep neural networks are so easily fooled?
The neural networks can be easily fooled or hacked by adding certain structured noise in image space (Szegedy 2013, Nguyen 2014) due to ignoring non-discriminative information in their input.
For ...
- 10.5k
2
votes
How is it possible that deep neural networks are so easily fooled?
Neural networks are easily fooled, provided you know how to fool them.
Consider a linear network with an input layer and an output layer, which has an error function E (we don't need hidden layers to ...
2
votes
How is it possible that deep neural networks are so easily fooled?
There is already many good answers, I will just add to those that came before mine:
This type of images you are referring to are called adversarial perturbations, (see 1, and it is not limited to ...
- 840
2
votes
Is there any research on the development of attacks against artificial intelligence systems?
There are some research at least on the "foolability" of neural networks, that gives insight on potential high risk of neural nets even when they "seem" 99.99% acurate.
A very good paper on this is ...
- 141
2
votes
What is an adversarial attack?
Adversarial examples are inputs to machine learning models that an attacker has intentionally designed to cause the model to make a mistake; they’re like optical illusions for machines.
Source: ...
- 632
Only top scored, non community-wiki answers of a minimum length are eligible