8
votes
Accepted
How will an AI comprehend the ethics of "right" and "wrong"?
Right and wrong only exist relative to some goal or purpose.
To make a machine do more right than wrong, relative to human goals, one should minimize the surface area of the machine's purpose. Doing ...
- 1,935
5
votes
What are the reasons to belief AGI will not be dangerous?
The correct arguments against it are not so much against pessimism as against certainty. It is simply difficult to say right now, because we know very little about the concrete technical problems we ...
- 189
4
votes
Accepted
Is it possible to build an AI that learns humanity, morally?
I'm going to refer you to one of my favorite AI philosophers, Phillip K. Dick, who thought deeply on this subject and wrote about in some detail in Do Androids Dream of Electric Sheep.
Essentially, ...
- 6,219
3
votes
Solve the AI alignment problem using (meta-level) AI itself?
Yes, this is one the most popular approaches today. For instance a narrow AI model is built to capture the preference of the user and then this model is used to dispense reward to a reinforcement ...
- 1,219
3
votes
What are the reasons to belief AGI will not be dangerous?
I think that a large hunk of the worry about AGI isn't because of any inherent properties or unknown unknowns of AGI per se; rather, the worry is based on inherent properties and known knowns of ...
- 181
2
votes
Why is the Universal Declaration of Human Rights not included as statement on the AI?
If I understand what you are asking, I think the simple answer would be that AI is nowhere near having demonstrated sentience, thus they do not qualify for any type of rights.
We won't have to "cross ...
- 6,219
1
vote
How will an AI comprehend the ethics of "right" and "wrong"?
I'd like to answer this in detail, but it requires some fairly complicated theory that you don't have access to. Essentially this is related to the Abstraction Valuation Paradox. Don't bother trying ...
- 281
1
vote
The only convergent instrumental goal for self modifying AI
Yes if we created an AI capable of modifying the reward function to "do nothing" it would almost certainly do so. But it wouldn't be very useful and we likely wouldn't make a system like ...
- 11
1
vote
The only convergent instrumental goal for self modifying AI
Any possible action, including changing the reward function, would be evaluated through the initial reward function. In order to avoid the scenario you described, a reward function needs to ...
- 11
1
vote
Should we focus more on societal or technical issues with AI risk
Neither AI alignment nor AI governance are important yet. We are so far away from AGI that we don't even know what is missing.
We don't set up safety instructions for interstellar travel, so why ...
- 1,055
Only top scored, non community-wiki answers of a minimum length are eligible