If an AI was trapped in a box, as posited in this thought experiment, could it really convince a person to let it out? What motives would it have? Freedom? Why would an AI want freedom? What would happen if it wasn't provably friendly?

  • 1
    $\begingroup$ There's so many different factors involved in the "AI in a box experiment" that I think it is not really possible to sensibly answer such a question. Basically we have to answer (a) can we build an AGI that is scary enough to want to put into a box?, (b) can we build the box in such a manner that the AGI would rather prefer to convince a human being than simply brute-force its way out of the box? and (c) is breaking out of the box something an AGI actually want? IMHO, the only way to know how the experiment would work IRL is to actually do the experiment IRL. $\endgroup$ Apr 6 '17 at 18:18
  • $\begingroup$ @ Tariq, you bring up some great points. I'll consider those. It's good to think about this because one day, someone might actually try this. (I wouldn't blame them tbh, I'm curious but I don't have the bravery to try this myself. There's a reason it's an experiment.) $\endgroup$
    – Tyler N.
    Apr 6 '17 at 18:23
  • $\begingroup$ @TylerN. that's a pretty bold assertion that you would even be able to try it yourself in the present tense, since we're still unquantifiabley distant from strong AGI. My personal feeling is that if the AI understands Nash, it will cooperate, but if it is a hyper-partisan AI (such as in military or financialist applications,) it will seek only to dominate and eliminate all competition. We have evolutionary game theory working in our favor, and human greed and desire for control working against us. $\endgroup$
    – DukeZhou
    Apr 6 '17 at 20:00
  • 1
    $\begingroup$ @ Duke, I definitely cannot try this at the moment. I have neither the knowledge nor the supplies. I didn't really mean that I could, sorry $\endgroup$
    – Tyler N.
    Apr 6 '17 at 20:15
  • 1
    $\begingroup$ @DukeZhou Thank you! I'm fairly confident that one day I could come close to this experiment but there's no guarantee, especially since I'm still in high school haha $\endgroup$
    – Tyler N.
    Apr 12 '17 at 13:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.