60

First up, those images (even the first few) aren't complete trash despite being junk to humans; they're actually finely tuned with various advanced techniques, including another neural network. The deep neural network is the pre-trained network modeled on AlexNet provided by Caffe. To evolve images, both the directly encoded and indirectly encoded images, ...


28

Yes, there is some research on this topic, which can be called adversarial machine learning, which is more an experimental field. An adversarial example is an input similar to the ones used to train the model, but that leads the model to produce an unexpected outcome. For example, consider an artificial neural network (ANN) trained to distinguish between ...


28

The images that you provided may be unrecognizable for us. They are actually the images that we recognize but evolved using the Sferes evolutionary framework. While these images are almost impossible for humans to label with anything but abstract arts, the Deep Neural Network will label them to be familiar objects with 99.99% confidence. This result ...


21

AI is vulnerable from two security perspectives the way I see it: The classic method of exploiting outright programmatic errors to achieve some sort of code execution on the machine that is running the AI or to extract data. Trickery through the equivalent of AI optical illusions for the particular form of data that the system is designed to deal with. ...


16

All answers here are great, but, for some reason, nothing has been said so far on why this effect should not surprise you. I'll fill the blank. Let me start with one requirement that is absolutely essential for this to work: the attacker must know neural network architecture (number of layers, size of each layer, etc). Moreover, in all cases that I examined ...


12

Sometimes if the rules used by an AI to identify characters are discovered, and if the rules used by a human being to identify the same characters are different, it is possible to design characters that are recognized by a human being but not recognized by an AI. However, if the human being and AI both use the same rules, they will recognize the same ...


11

Yes there are, for instance one pixel attacks described in Su, J.; Vargas, D.V.; Kouichi, S. One pixel attack for fooling deep neural networks. arXiv:1710.08864 One pixels attacks are attacks in which changing one pixel in input image can strongly affect the results.


11

An important question that does not yet have a satisfactory answer in neural network research is how DNNs come up with the predictions they offer. DNNs effectively work (though not exactly) by matching patches in the images to a "dictionary" of patches, one stored in each neuron (see the youtube cat paper). Thus, it may not have a high level view of the ...


7

Programmer vs Programmer It's a "infinity war": Programmers vs Programmers. All thing can be hackable. Prevention is linked to the level of knowledge of the professional in charge of security and programmers in application security. eg There are several ways to identify a user trying to mess up the metrics generated by Sentiment Analysis, but there are ...


6

How we can prevent it? There are several works about AI verification. Automatic verifiers can prove the robustness properties of neural networks. It means that if the input X of the NN is perturbed not more that on a given limit ε (in some metric, e.g. L2), then the NN gives the same answer on it. Such verifiers are done by: Stanford: https://arxiv.org/...


5

Here's an example: How to hack your face to dodge the rise of facial recognition tech In his recent book The Fall, Stephenson wrote about smartglasses that that project a pattern over the facial features to foil recognition algorithms (which seems not only feasible but likely;) Here's an article from our sponsors, Adversarial AI: As New Attack Vector ...


5

How is it possible that deep neural networks are so easily fooled? Deep neural networks are easily fooled by giving high confidence predictions for unrecognizable images. How is this possible? Can you please explain ideally in plain English? Intuitively, extra hidden layers ought to make the network able to learn more complex classification functions, and ...


5

To answer your question, it really depends on the purpose of the Artificial Intelligence program. For example, 4Chan has hacked a number of "Artificial Intelligent" bots, most notably was Microsoft's Twitter bot Tay. The general purpose of the bot was to parse what was tweeted at it and respond in kind, learning and evolving with each and every interaction. ...


5

These are known as adversarial attacks, and the specific examples that are misclassified are known as adversarial examples. There is a reasonably large body of work on finding adversarial examples, and on making CNNs more robust (i.e. less prone to these attacks). An example is the DeepFool algorithm, which can be used to find perturbations of data which ...


5

Those examples are called Adversarial Examples. I think it is important to understand why a CNN can be "tricked" like that: We often expect human-like behavior when a model has a human-like performance. That is similar for CNNs. We expect they decide like we do, i.e. we look for the shape of objects. However, as various experiments on common CNN ...


4

However, do industrial strength, production ready defensive strategies and approaches exist? Are there known examples of applied adversarial-resistant networks for one or more specific types (e.g. for small perturbation limits)? I think it's difficult to tell whether or not there are any industrial strength defenses out there (which I assume would mean that ...


4

Is Artificial Intelligence Vulnerable to Hacking? Invert your question for a moment and think: What would make AI at less of a risk of hacking compared to any other kind of software? At the end of the day, software is software and there will always be bugs and security issues. AIs are at risk to all the problems non-AI software is at risk to, being AI ...


4

I believe it is, no system is safe, however I am not sure if I can still say this after 20-30 years of AI development/evolution. Anyways, there are articles that showed humans fooling AI (Computer Vision). https://www.theverge.com/2018/1/3/16844842/ai-computer-vision-trick-adversarial-patches-google https://spectrum.ieee.org/cars-that-think/transportation/...


4

Can't comment(due to that required 50 rep), but I wanted to make a response to Vishnu JK and the OP. I think you guys are skipping the fact that the neural network only really is saying truly from a programmatic standpoint that "this is most like". For example, while we can list the above image examples as "abstract art", they definitively are most like was ...


4

There are many insightful comments and answers so far. I want to illustrate my idea of "color blindness test" more. Maybe it's a hint to lead us to the truth. Imagine there are two people here. One is colorblind (AI) and another one is non-colorblind (human). If we show them a normal number "6", both of them can easily recognize it as number 6. Now, if we ...


4

Isn't that essentially what chess does? For example, A human can recognize that a Ruy exchange offers white great winning chances (because of pawn structure) by move 4 while an engine would take several hours of brute force calculation to understand the same idea.


4

Not necessarily it depends on the function of the problem space for both the GANs. A real world example: a batter's reaction time and a pitchers max speed are actual bounded values based on genetics and physics. If the max speed a pitcher can pitch is greater than the max reaction time a human needs to effectively hit against them they will permanently be ...


4

When someone is able to do a causative attack it means there is a mechanism by which they are able to input data into the network. Maybe a website where people can input their images and it outputs a guess on what is in the picture and then you click if it got it right or not. If you continue to input images and lie to it it will obviously get worse and ...


4

A robust ML model is one that captures patterns that generalize well in the face of the kinds of small changes that humans expect to see in the real world. A robust model is one that generalizes well from a training set to a test or validation set, but the term also gets used to refer to models that generalize well to, e.g. changes in the lighting of a ...


3

Here's a live demo: https://www.labsix.org/physical-objects-that-fool-neural-nets/ Recall that neural nets are trained by feeding in the training data, evaluating the net, and using the error between the observed and the intended output to adjust the weights and bring the observed output closer to the intended. Most attacks have been on the observation that ...


3

The neural networks can be easily fooled or hacked by adding certain structured noise in image space (Szegedy 2013, Nguyen 2014) due to ignoring non-discriminative information in their input. For example: Learning to detect jaguars by matching the unique spots on their fur while ignoring the fact that they have four legs.2015 So basically the high ...


2

Neural networks are easily fooled, provided you know how to fool them. Consider a linear network with an input layer and an output layer, which has an error function E (we don't need hidden layers to show how to fool a network). For a given input image x, E measures the (squared) difference between the network's output y and the desired (correct) output. ...


2

There is already many good answers, I will just add to those that came before mine: This type of images you are referring to are called adversarial perturbations, (see 1, and it is not limited to images, it has been shown to apply to text too, see Jia & Liang, EMNLP 2017. In text, the introduction of an irrelevant sentence which doesn't contradict the ...


2

I don't have a proper source for this, as i've only read this from an online forum: everything i say is just hearsay and I am very uneducated on the subject. With that being said... As you may know algorithmic trading relies on strategies, i.e. I trade a certain way once I see certain indexes move in a certain way. If your procedure is known by other ...


2

There are some research at least on the "foolability" of neural networks, that gives insight on potential high risk of neural nets even when they "seem" 99.99% acurate. A very good paper on this is in Nature: https://www.nature.com/articles/d41586-019-03013-5 In a nutshell: It shows diverse exemples of fooling neural networks/AIs, for exemple one where a ...


Only top voted, non community-wiki answers of a minimum length are eligible