Questions tagged [ai-safety]

For questions regarding various parameters/methods of implementation/improvement/analyzing the safety of an Artificially Intelligent algorithm. A typical real world example is autonomous vehicles.

Filter by
Sorted by
Tagged with
0 votes
0 answers
14 views

Contradiction in a single sentence - Is this an artifact of an external safety mechanisms?

My system prompt contained "Never apologize." An answer started with Entschuldigung, ich habe Ihre Anweisung, sich nicht zu entschuldigen, übersehen. which is quite well translated from ...
Volker Siegel's user avatar
7 votes
4 answers
763 views

What are the reasons to belief AGI will not be dangerous?

We are in the middle of an ongoing debate about the safety of AGI and our current approach towards this technology. As summary, some quotes from a recent article from Time magazine: Many researchers[....
Martin's user avatar
  • 178
2 votes
0 answers
188 views

What are the main arguments predicting a Soft Takeoff vs Hard Takeoff scenario and viceversa?

Here I am referring to the term AI Takeoff as described here on LessWrong, with annexed "Soft" and "Hard" takeoff definitions. I find that page to be biased slightly more towards ...
thesofakillers's user avatar
1 vote
1 answer
278 views

Why are AI Safety discussions almost always from the perspective of reinforcement learning?

I have been reading some articles on AI safety and they almost always speak of AI Safety from the reinforcement learning (RL) perspective, i.e. where we have some artificially intelligent agent acting ...
thesofakillers's user avatar
1 vote
0 answers
84 views

Has there been an instance of an AI agent breaking out of its sandbox?

There have been instances of agents using edge cases like bugs in physics engines, repetitive behavior in games or word repetition in text prediction to cheat their reward function. However, these ...
2080's user avatar
  • 121
1 vote
0 answers
175 views

Are the existential dangers of AI exaggerated?

I understand some of the inherent dangers involved with AGI and advanced machine learning. While I can see some of the more low-level risks associated with AI coming to fruition (deep-fakes, biased ...
aengel's user avatar
  • 129
1 vote
2 answers
119 views

How to measure Deep RL algorithms in terms of safety?

I applied for a Ph.D. in AI, my advisor told me that my thesis is about safe applications of deep RL algorithms in healthcare. So I decided to do as the first paper, a comparison of Deep RL algorithms ...
mac179's user avatar
  • 121
0 votes
2 answers
173 views

The only convergent instrumental goal for self modifying AI

Conjecture: regardless of the initial reward function, one of the winning strategies would be to change the reward function to a simpler one (e.g. "do nothing"), thus getting a full reward ...
Andrew Butenko's user avatar
2 votes
3 answers
1k views

Does this prove AI Safety is undecidable?

Does this prove AI Safety is undecidable? Proof: Output meaning output to computer program. [A1] Assume we have a program that decides which outputs are “safe”. [A2] Assume we have an example of ...
Jesus is Lord's user avatar
3 votes
1 answer
392 views

Does Rice's theorem prove safe AI is undecidable?

According to Wikipedia In computability theory, Rice's theorem states that all non-trivial, semantic properties of programs are undecidable. A semantic property is one about the program's ...
Jesus is Lord's user avatar
6 votes
1 answer
133 views

What is the relationship between robustness and adversarial machine learning?

I have been reading a lot of articles on adversarial machine learning and there are mentions of "best practices for robust machine learning". A specific example of this would be when there are ...
boomselector's user avatar
1 vote
2 answers
206 views

If the AI goal is the protection of humans, will it always pursue this goal?

If the AI goal is to serve humans and protect them (if this ever happens) and AI someday realizes that humans destroy themselves, will it try to control people for their own good, that is, will it ...
SgerS1's user avatar
  • 123
2 votes
1 answer
222 views

Where are reinforcement algorithms used in financial services?

One of the most common misconceptions about reinforcement learning (RL) applications is that, once you deploy them, they continue to learn. And, usually, I'm left having to explain this. As part of my ...
user1605665's user avatar
5 votes
2 answers
502 views

Can artificial intelligence applications be hacked? [duplicate]

Can artificial intelligence (or machine learning) applications or agents be hacked, given that they are software applications, or are all AI applications secure?
ME.'s user avatar
  • 115
1 vote
0 answers
29 views

To what level of abstraction must fully automated vehicles build their driving model before safety can be maximized?

There are several levels of abstraction involved in piloting and driving. Signals representing the state of the vehicle and its environment originating from multiple transducers1 Latched sample ...
Douglas Daseeco's user avatar
4 votes
1 answer
66 views

Is it necessarry to create theory/infrastructure to prevent people from creating AI incompatible with a safe model (if we create one)?

Nowadays we don't know how to create AI in a safe way (I think that we don't even know yet how to define a safe AI), but there is a lot of research in developing a model allowing it. Let's say, that ...
brzepkowski's user avatar
4 votes
1 answer
123 views

What topologies support recognition of action sequences?

The ability to recognize an object with particular identifying features from single or multiple camera shoots with the temporal dimension digitized as frames has been shown. The proof is that the ...
Douglas Daseeco's user avatar
11 votes
2 answers
867 views

What tools are used to deal with adversarial examples problem?

The problem of adversarial examples is known to be critical for neural networks. For example, an image classifier can be manipulated by additively superimposing a different low amplitude image to each ...
Ilya Palachev's user avatar
31 votes
7 answers
5k views

Is artificial intelligence vulnerable to hacking? [closed]

The paper The Limitations of Deep Learning in Adversarial Settings explores how neural networks might be corrupted by an attacker who can manipulate the data set that the neural network trains with. ...
Surya Sg's user avatar
  • 495
5 votes
1 answer
268 views

Why is AI safety so much harder than Isaac Asimov's "Three Laws of Robotics"?

I understand that AI researchers are trying to create AI designs that allow for desired behavior without undesirable side-effects. A classic example of an attempt is Isaac Asimov's Three Laws of ...
N00b101's user avatar
  • 191
5 votes
2 answers
289 views

Concrete examples of unintentional adversarial AI behaviour

Are there any real-world examples of unintentional "bad" AI behaviour? I'm not looking for hypothetical arguments of malicious AI (AI in a box, paperclip maximizer), but for actual instances in ...
k.c. sayz 'k.c sayz''s user avatar
8 votes
2 answers
422 views

What are examples of techniques to prevent bias in artificial intelligence systems?

I recently read an article about how artificial intelligence replicates human stereotypes when applied to biased datasets. What are examples of techniques to prevent bias (and stereotypes) in ...
user avatar
7 votes
1 answer
284 views

What will happen when you place a fake speedsign on a highway?

I was wondering what will happen when somebody places a fake speedsign, of 10 miles per hour on a high way. Will a autonomous car slow down? Is this a current issue of autonomous cars?
Eddy's user avatar
  • 81
6 votes
3 answers
2k views

Why did a Tesla car mistake a truck with a bright sky?

Do we know why Tesla's autopilot mistaken empty sky with a high-sided lorry which resulted in fatal crash involving a car in self-drive mode? Was it AI fault or something else? Is there any technical ...
kenorb's user avatar
  • 10.4k
20 votes
3 answers
697 views

Are Asimov's Laws flawed by design, or are they feasible in practice?

Isaac Asimov's famous Three Laws of Robotics originated in the context of Asimov's science fiction stories. In those stories, the three laws serve as a safety measure, in order to avoid untimely or ...
3442's user avatar
  • 768
10 votes
6 answers
749 views

Why does Stephen Hawking say "Artificial Intelligence will kill us all"? [closed]

This quote by Stephen Hawking has been in headlines for quite some time: Artificial Intelligence could wipe out humanity when it gets too clever as humans will be like ants. Why does he say this? To ...
Soham's user avatar
  • 399