28 votes
Accepted

Is there any research on the development of attacks against artificial intelligence systems?

Yes, there is some research on this topic, which can be called adversarial machine learning, which is more an experimental field. An adversarial example is an input similar to the ones used to train ...
user avatar
  • 33k
21 votes

Is artificial intelligence vulnerable to hacking?

AI is vulnerable from two security perspectives the way I see it: The classic method of exploiting outright programmatic errors to achieve some sort of code execution on the machine that is running ...
user avatar
12 votes

Is there any research on the development of attacks against artificial intelligence systems?

Sometimes if the rules used by an AI to identify characters are discovered, and if the rules used by a human being to identify the same characters are different, it is possible to design characters ...
user avatar
  • 353
11 votes

Is there any research on the development of attacks against artificial intelligence systems?

Yes there are, for instance one pixel attacks described in Su, J.; Vargas, D.V.; Kouichi, S. One pixel attack for fooling deep neural networks. arXiv:1710.08864 One pixels attacks are attacks in ...
user avatar
8 votes
Accepted

How would one debug, understand or fix the outcome of a neural network?

If the observation that the neural network saw was recorded, then yes the prediction can be explained. There was a paper written fairly recently on this topic called "Why Should I Trust You?": ...
user avatar
7 votes

Is artificial intelligence vulnerable to hacking?

Programmer vs Programmer It's a "infinity war": Programmers vs Programmers. All thing can be hackable. Prevention is linked to the level of knowledge of the professional in charge of security and ...
user avatar
6 votes

Is artificial intelligence vulnerable to hacking?

How we can prevent it? There are several works about AI verification. Automatic verifiers can prove the robustness properties of neural networks. It means that if the input X of the NN is perturbed ...
user avatar
5 votes

Is there any research on the development of attacks against artificial intelligence systems?

Here's an example: How to hack your face to dodge the rise of facial recognition tech In his recent book The Fall, Stephenson wrote about smartglasses that that project a pattern over the facial ...
user avatar
  • 6,077
5 votes

Can artificial intelligence applications be hacked?

To answer your question, it really depends on the purpose of the Artificial Intelligence program. For example, 4Chan has hacked a number of "Artificial Intelligent" bots, most notably was Microsoft's ...
user avatar
5 votes
Accepted

What will happen when you place a fake speedsign on a highway?

https://www.technologyreview.com/s/530276/hidden-obstacles-for-googles-self-driving-cars/ Google’s cars can detect and respond to stop signs that aren't on its map, a feature that was introduced to ...
user avatar
  • 249
4 votes
Accepted

What tools are used to deal with adversarial examples problem?

However, do industrial strength, production ready defensive strategies and approaches exist? Are there known examples of applied adversarial-resistant networks for one or more specific types (e.g. for ...
user avatar
  • 9,326
4 votes

Is artificial intelligence vulnerable to hacking?

Is Artificial Intelligence Vulnerable to Hacking? Invert your question for a moment and think: What would make AI at less of a risk of hacking compared to any other kind of software? At the end ...
user avatar
  • 210
4 votes

Is artificial intelligence vulnerable to hacking?

I believe it is, no system is safe, however I am not sure if I can still say this after 20-30 years of AI development/evolution. Anyways, there are articles that showed humans fooling AI (Computer ...
user avatar
  • 41
4 votes
Accepted

Why did a Tesla car mistake a truck with a bright sky?

Tesla's technology is assistive, as Alexey points out, so this is not a case of an autonomous system (e.g. an AGI) doing some fatal stunt (the product name AutoPilot is famously misleading). Now on ...
user avatar
  • 1,480
4 votes

Is there any research on the development of attacks against artificial intelligence systems?

There are many insightful comments and answers so far. I want to illustrate my idea of "color blindness test" more. Maybe it's a hint to lead us to the truth. Imagine there are two people here. One ...
user avatar
  • 403
4 votes

Is there any research on the development of attacks against artificial intelligence systems?

Isn't that essentially what chess does? For example, A human can recognize that a Ruy exchange offers white great winning chances (because of pawn structure) by move 4 while an engine would take ...
user avatar
3 votes

Is there any research on the development of attacks against artificial intelligence systems?

Here's a live demo: https://www.labsix.org/physical-objects-that-fool-neural-nets/ Recall that neural nets are trained by feeding in the training data, evaluating the net, and using the error between ...
user avatar
  • 131
3 votes

Why did a Tesla car mistake a truck with a bright sky?

As far as I know, Tesla cars autopilot is not a 100% AI pilot, it's an assitant: as it detects hands off wheel it slows down, so it's incorrect to speak about AI mistake: it is not trained/designed to ...
user avatar
2 votes

Which government agencies oversee development of new AI?

May not be quite what you’re looking for, but nonetheless helpful, I hope. The White House a year ago commissioned a report on AI that touches briefly on policy issues.
user avatar
2 votes
Accepted

How can genetic programming be used to prevent cyberterrorism?

They treated it as a classification problem. While it's common to use some variety of Neural Nets (NNs) to build classifiers, Genetic Programming (GP) can also be used for this purpose. In contrast to ...
user avatar
2 votes
Accepted

How to source training data in ML for information security?

You have implicitly assumed that supervised learning is being used, given the assumption that labels are needed. But this might lead to the following potential problems: Log file data tends to be ...
user avatar
  • 391
2 votes
Accepted

During neural network training, can gradients leak sensitive information in case training data fed is encrypted (homomorphic)?

It will recover the encrypted inputs. The algorithm starts with dummy data and dummy labels, and then iteratively optimizes the dummy gradients to be close as to the original. This makes the dummy ...
user avatar
  • 314
2 votes

Is there any research on the development of attacks against artificial intelligence systems?

There are some research at least on the "foolability" of neural networks, that gives insight on potential high risk of neural nets even when they "seem" 99.99% acurate. A very good paper on this is ...
user avatar
2 votes
Accepted

How to design a classifier while the patterns of positive data are changing rapidly?

The phenomenon where the prediction targets (in your case, behaviour) change over time is referred to as "concept drift". If you search for that term, you'll find that there have been many ...
user avatar
  • 9,326
1 vote

How do I encrypt and decrypt my model when I run inference on it?

While @Oliver Mason's comment is correct, and your proposed method won't provide perfect security, you can still protect your models at rest, so that they are stored encrypted in the memory, and your ...
user avatar
  • 314
1 vote

Can artificial intelligence applications be hacked?

Everything can be hacked. The solutions found by artificial intelligence can be much more efficient than human solutions, but they can also be confused because of the diversity and immensity of ...
user avatar
1 vote

What tools are used to deal with adversarial examples problem?

Another point of view - In safety-critical real world systems, this attack should be evaluated from other aspects as well. In many systems the attack is somewhat mitigated to physical attacks only - ...
user avatar
  • 314
1 vote

Could a CNN hear the difference between sound of a pet moving, and a person?

It is an interesting application. It is possible. You can interpret sound as histogram (2D image) and apply same image processing techniques (CNN) to extract information. Alternatively, you can keep ...
user avatar
1 vote

Is artificial intelligence vulnerable to hacking?

I concur with Akio that no system is completely safe, but the take away is AI systems are less prone to attacks when comparing with the old systems because of the ability to constantly improve. As ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible