8 votes
Accepted

What is the Control Problem?

The Control Problem is, in short, the idea that AI will eventually be much better decision-makers than humans. If we don't set things up correctly beforehand, we won't get a chance to fix it ...
user avatar
6 votes
Accepted

Would an AI with human intelligence have the same rights as a human under current legal frameworks?

Yes, to some of what you propose. No to some. Today corporations are granted rights: to own property, earn income, pay taxes, contribute to political campaigns, offer opinion in public, ad more. ...
user avatar
  • 671
6 votes

Is overfitting always a bad thing?

Typically the ramification of overfitting is poor performance on unseen data. If you're confident that overfitting on your dataset will not cause problems for situations not described by the dataset, ...
user avatar
  • 61
5 votes
Accepted

Is policy learning and online system identification the same?

From the book Reinforcement Learning, An Introduction (R. Sutton, A. Barto): The term system identification is used in adaptive control for what we call model-learning (e.g., Goodwin and Sin, ...
user avatar
4 votes

Could artificial intelligence cause problems for humanity after figuring out human behavior?

Will Artificial Intelligence some day become a problem to humanity after learning human behaviors and characteristics? It can be answered in both ways, I think. Yes, they may become a problem. ...
user avatar
4 votes

What would be the best way to disable a rogue AI?

Metaphorically: make it so depressed it commits suicide. As per my answer to this AI SE question, the idea is to feed it a sequence of inputs that will cause it to become (permanently) inactive. The ...
user avatar
4 votes
Accepted

What's the term for death by dissolving in AI?

I find the concept of the a Turing machine useful. In one dimension, everything is a string. All of the parts that are "not you" are merely a substrate, a medium for the program your_mind runs on top ...
user avatar
  • 6,067
4 votes
Accepted

Why is GLIE Monte-Carlo control an on-policy control?

In this case, $\pi$ has always been an $\epsilon$-greedy policy. In every iteration, this $\pi$ is used to generate ($\epsilon$-greedily) a trajectory from which the new $Q(s, a)$ values are ...
user avatar
4 votes

Is overfitting always a bad thing?

Is overfitting always a bad thing? The answer is a resounding yes, every time. The reason being that overfitting is the name we use to refer to a situation where your model did very well on the ...
user avatar
3 votes
Accepted

Which technologies should humanity develop before developing an AI with general superhuman intelligence?

I think most answers that come to mind are going to be connected to the control problem in some way---e.g. sandboxing frameworks, AI behavior monitoring protocols, etc. However, if you aren't ...
user avatar
3 votes
Accepted

Is there a way to protect humanity against the impending singularity?

What you are talking about is known as the Control Problem. We have our own tag for this specific topic here, which you can use for this and similar questions. How to address the control problem is ...
user avatar
  • 1,644
3 votes

Would an AI with human intelligence have the same rights as a human under current legal frameworks?

Murray Shanahan, in his book The Technological Singularity, makes the case that the rights of any being are determined by its intelligence. For instance, we value the life of a dog above that of an ...
user avatar
  • 462
3 votes
Accepted

What would be the best way to disable a rogue AI?

Nuke it from orbit - it's the only way to be sure If you want to be really sure you destroy everything of the AI, you'll need to launch an EMP (electromagnetic pulse) from the orbit (there are ...
user avatar
2 votes

What AI concept is behind the Mars Exploration Rover (MER)?

The Mars Rover is a highly successful example of the 'New AI' that emerged from work by Rodney Brooks in the 1990s. In a quote from Brooks: In 1984 I joined the faculty at MIT where I have been ...
user avatar
2 votes

What would be the best way to disable a rogue AI?

If an AI is developed by humans, we surely can create another one! Develop another AI agent without all the possible bugs that can make it go rogue to tackle the rogue AI, but more technically ...
user avatar
2 votes

Would an AI with human intelligence have the same rights as a human under current legal frameworks?

No matter what rights it gets (as a company), it will still lack the right of not getting liquefied and all its properties transferred back to natural persons. This is of course if no laws are ...
user avatar
  • 244
2 votes
Accepted

Can we use a messenger that does not alter the AI to solve the control problem?

As long as the output of the AI affects the world, the way in which it communicates makes no fundamental difference to the control problem. The AI might still be able, for example, to manoeuvre ...
user avatar
2 votes

Can we use a messenger that does not alter the AI to solve the control problem?

I would say no due to the possibility of psychological manipulation of the messenger by the AI. Also, the LED communication constraints place severe limitations on the capabilities of the AI, as the ...
user avatar
2 votes

Which government agencies oversee development of new AI?

May not be quite what you’re looking for, but nonetheless helpful, I hope. The White House a year ago commissioned a report on AI that touches briefly on policy issues.
user avatar
2 votes

Is policy learning and online system identification the same?

System Identification and policy learning are two completely different aspects of a system. System Identification is basically finding out the transfer functions, the hardware parameters, the ...
user avatar
2 votes

Is overfitting always a bad thing?

Overfitting is almost always bad and hurts generalization. You say what we want from the NN is to say "take a slight left turn" or "turn right hard" if another ship comes slightly close on the ...
user avatar
  • 1,877
2 votes
Accepted

Can the AI in a box experiment be formalized?

I think that something like "strength" would be difficult to quantify in this context. I do think that formal experimentation around the "AI in a box" scenario could be interesting. I know that ...
user avatar
  • 136
2 votes
Accepted

Can future information be included in a control problem with Reinforcement Learning?

Typically in a control problem, it is OK to include data about a future event, if it can be reliably predicted at the time that a decision is required. This would include known rules such as a pricing ...
user avatar
  • 23.9k
1 vote
Accepted

Will robots rebel against their human creators?

I don't like to be a killjoy, but this question seems premature (that's why it's hd the "mythology of AI" tag added to it). The kinds of emergent artificial general intelligence depicted in the movies ...
user avatar
1 vote

Is there a way to protect humanity against the impending singularity?

Predicting what happens post-singularity is simply not possible as we cannot attempt to model let alone conceptualise a mind far more complex than ours. If that is a difficult concept to get your head ...
user avatar
  • 111
1 vote

What's the term for death by dissolving in AI?

"Ego death" Ego death is a "complete loss of subjective self-identity." -"Ego death", Wikipedia The defining aspect of an ego is an exclusive sense of self. If an ego-...
user avatar
  • 117
1 vote

What's the term for death by dissolving in AI?

How did it became diluted? if by mutations then it has "evolved" or neural plasticity then it "learned", Alzheimer then it got "sick". If by analogy, you took too much psycho-active drug and grew up ...
user avatar
  • 244

Only top scored, non community-wiki answers of a minimum length are eligible