0
$\begingroup$

The Wikipedia page describes AI control problem in very intricated way.

Therefore I would like to better understand it based on some simple explanation, what's going on. Basically I don't want any copy & pastes from wiki, because the articles there are written in neutral point of view, in very general way where articles are evolving very slowly, so the definition from there doesn't suit me.

I believe this is what is discussed nowadays by government and it's important aspects of AI technology where it leds to. I believe this could be a big problem in the near future, so I'm expecting to hear about this from people from much better and more up-to-date point of view.

So what is exactly the AI Control Problem?

$\endgroup$
6
$\begingroup$

The Control Problem is, in short, the idea that AI will eventually be much better decision-makers than humans. If we don't set things up correctly beforehand, we won't get a chance to fix it afterwards, because AI will have effective control.

There are three main areas of discussion with regards to the Control Problem:

  1. Whether or not the problem is urgent. Many AI experts, cognizant of the difficulty of getting simple systems to work effectively today, think that AI able to take control is not urgent, and as detail-minded engineers, they think it will be profoundly difficult to do any useful work today. (Andrew Ng, for example, famously called these sorts of worries like worrying about overpopulation on Mars.) Given radical uncertainty among AI experts as to when this will become an issue, however, this means we can't rule out rapid AI timescales, and should do at least some work in anticipation of those timescales.

  2. Whether or not the problem is hard. Many people give short, simple, and wrong solutions to the control problem. Probably the most famous is the idea that intelligence and morality are inherently interlinked, and thus a more intelligent machine, by definition, will be more moral. The Orthogonality Thesis is the claim of the opposite, that intelligence and morality (or, more specifically, goal alignment) are unrelated things.

  3. What foundations we can lay now. There are a bunch of open problems (see, for example, MIRI's technical agenda) that deal with mathematical logic of the sort that would be useful for ensuring robust value alignment, or on how to effectively do value learning (without giving an incentive to distort values), or on how to build value functions and goals such that they are fixable if they turn out to be mistaken. Those look like problems that we can do useful work on now, even without knowing what the actual structure of a future AI will look like.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.