Skip to main content
OverflowAI is here! AI power for your Stack Overflow for Teams knowledge community. Learn more
8 votes
Accepted

How will an AI comprehend the ethics of "right" and "wrong"?

Right and wrong only exist relative to some goal or purpose. To make a machine do more right than wrong, relative to human goals, one should minimize the surface area of the machine's purpose. Doing ...
Doxosophoi's user avatar
  • 1,945
5 votes

What are the reasons to belief AGI will not be dangerous?

The correct arguments against it are not so much against pessimism as against certainty. It is simply difficult to say right now, because we know very little about the concrete technical problems we ...
Daniel Paleka's user avatar
4 votes
Accepted

Is it possible to build an AI that learns humanity, morally?

I'm going to refer you to one of my favorite AI philosophers, Phillip K. Dick, who thought deeply on this subject and wrote about in some detail in Do Androids Dream of Electric Sheep. Essentially, ...
DukeZhou's user avatar
  • 6,237
3 votes

Solve the AI alignment problem using (meta-level) AI itself?

Yes, this is one the most popular approaches today. For instance a narrow AI model is built to capture the preference of the user and then this model is used to dispense reward to a reinforcement ...
Rexcirus's user avatar
  • 1,174
2 votes

What are the reasons to belief AGI will not be dangerous?

I think that a large hunk of the worry about AGI isn't because of any inherent properties or unknown unknowns of AGI per se; rather, the worry is based on inherent properties and known knowns of ...
Him's user avatar
  • 171
2 votes

Why is the Universal Declaration of Human Rights not included as statement on the AI?

If I understand what you are asking, I think the simple answer would be that AI is nowhere near having demonstrated sentience, thus they do not qualify for any type of rights. We won't have to "cross ...
DukeZhou's user avatar
  • 6,237
1 vote

How will an AI comprehend the ethics of "right" and "wrong"?

I'd like to answer this in detail, but it requires some fairly complicated theory that you don't have access to. Essentially this is related to the Abstraction Valuation Paradox. Don't bother trying ...
scientious's user avatar
1 vote

The only convergent instrumental goal for self modifying AI

Yes if we created an AI capable of modifying the reward function to "do nothing" it would almost certainly do so. But it wouldn't be very useful and we likely wouldn't make a system like ...
user10568257's user avatar
1 vote

The only convergent instrumental goal for self modifying AI

Any possible action, including changing the reward function, would be evaluated through the initial reward function. In order to avoid the scenario you described, a reward function needs to ...
guest's user avatar
  • 11
1 vote

Should we focus more on societal or technical issues with AI risk

Neither AI alignment nor AI governance are important yet. We are so far away from AGI that we don't even know what is missing. We don't set up safety instructions for interstellar travel, so why ...
Martin Thoma's user avatar
  • 1,055

Only top scored, non community-wiki answers of a minimum length are eligible