3
$\begingroup$

The Wikipedia page describes AI control problem in very intricated way.

Therefore I would like to better understand it based on some simple explanation, what's going on. Basically I don't want any copy & pastes from wiki, because the articles there are written in neutral point of view, in very general way where articles are evolving very slowly, so the definition from there doesn't suit me.

I believe this is what is discussed nowadays by government and it's important aspects of AI technology where it leds to. I believe this could be a big problem in the near future, so I'm expecting to hear about this from people from much better and more up-to-date point of view.

So what is exactly the AI Control Problem?

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
8
$\begingroup$

The Control Problem is, in short, the idea that AI will eventually be much better decision-makers than humans. If we don't set things up correctly beforehand, we won't get a chance to fix it afterwards, because AI will have effective control.

There are three main areas of discussion with regards to the Control Problem:

  1. Whether or not the problem is urgent. Many AI experts, cognizant of the difficulty of getting simple systems to work effectively today, think that AI able to take control is not urgent, and as detail-minded engineers, they think it will be profoundly difficult to do any useful work today. (Andrew Ng, for example, famously called these sorts of worries like worrying about overpopulation on Mars.) Given radical uncertainty among AI experts as to when this will become an issue, however, this means we can't rule out rapid AI timescales, and should do at least some work in anticipation of those timescales.

  2. Whether or not the problem is hard. Many people give short, simple, and wrong solutions to the control problem. Probably the most famous is the idea that intelligence and morality are inherently interlinked, and thus a more intelligent machine, by definition, will be more moral. The Orthogonality Thesis is the claim of the opposite, that intelligence and morality (or, more specifically, goal alignment) are unrelated things.

  3. What foundations we can lay now. There are a bunch of open problems (see, for example, MIRI's technical agenda) that deal with mathematical logic of the sort that would be useful for ensuring robust value alignment, or on how to effectively do value learning (without giving an incentive to distort values), or on how to build value functions and goals such that they are fixable if they turn out to be mistaken. Those look like problems that we can do useful work on now, even without knowing what the actual structure of a future AI will look like.

Improve this answer
$\endgroup$
1
  • $\begingroup$ My sense is that fundamental game theory validates renormalized rationality where there is a relationship and some trust (or willingness to sacrifice.) So I personally think the simple math on empathy being a natural function of evolution is more solid than the refutation. I can't see any meaningful notion of morality unrelated to economic decision making and outcomes post 20th century. But optimism is not strictly rational, and neoluddism has only ever seemed to have been validated. $\endgroup$
    – DukeZhou
    Commented Jun 18, 2021 at 1:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .