7

Right and wrong only exist relative to some goal or purpose. To make a machine do more right than wrong, relative to human goals, one should minimize the surface area of the machine's purpose. Doing that minimizes the intrinsic behavior of the AI, which enables us to reason about the right and wrong behaviors of the AI, relative to human purposes. Horses ...


4

I'm going to refer you to one of my favorite AI philosophers, Phillip K. Dick, who thought deeply on this subject and wrote about in some detail in Do Androids Dream of Electric Sheep. Essentially, replicants (artificial humans) had a design flaw--they lacked empathy. This flaw was allowed to persist because it had a useful side-effect in that replicants ...


2

If I understand what you are asking, I think the simple answer would be that AI is nowhere near having demonstrated sentience, thus they do not qualify for any type of rights. We won't have to "cross this bridge" until an AI demonstrates self-awareness and human-level-or-beyond intelligence, but it sure is interesting to think about! (Also, the UDHR dates ...


1

Yes if we created an AI capable of modifying the reward function to "do nothing" it would almost certainly do so. But it wouldn't be very useful and we likely wouldn't make a system like that. Self preservation isn't the only possible goal, but it is a convergent instrumental goal. Self preservation is usually useful for completing other goals. AI ...


1

Any possible action, including changing the reward function, would be evaluated through the initial reward function. In order to avoid the scenario you described, a reward function needs to disincentivize changes to itself by giving those the lowest possible reward.


1

This is a very, very controversial topic. But here's my opinion on the topic: AI won't harm humanity (at least, to the level that a lot of people think it will). As Kartik mentioned in his answer, AGI (Artificial General Intelligence) is the concept of AI that's become so "generally" intelligent, that it can do everything we as humans can, and more, but do ...


1

Neither AI alignment nor AI governance are important yet. We are so far away from AGI that we don't even know what is missing. We don't set up safety instructions for interstellar travel, so why should we do it for AGI? I can also come up with a lot of dangers of that... There are real dangers of AI, though. Including societal issues: Blind trust: ...


Only top voted, non community-wiki answers of a minimum length are eligible