Let's suppose there are two AGIs, $A$ and $B$. Assume that $B$ has the ability to modify $A$, but this action of modifying is considered bad by $B$. Can $A$ ever convince $B$ to modify $A$?
$\begingroup$
$\endgroup$
0
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
4
An AI takes the decision based on the output of their utility function. This is just a fancy word for the calculations that AI perform to compare profit and loss of taking a certain decision.
There is always a tight analogy between an AGI and a human. You can juxtapose the utility function to how we make decisions by considering marginal profits of doing something over the other.
Now your question. The answer is a big NO. This is because an AI (or a human) never takes an action that gives a low score on its utility function or is against the fundamental view of the machine (or human). AI only cares about the utility function and goal state (plus the instrumental goals), nothing else. Our AI outside the box has been created with the purpose of not opening the box. For it, the action of opening the box has a very low score, or a negative score.
Now you might be wondering why an AI can't be convinced of doing something that gives a low score in its utility function. Keep reading.
Consider this for a second. Suppose an AGI was created, assigned with the task of copying the handwriting of others and improving its own writing. You can think of it as a smart writing hand. Now, what this AI might come up with is that, in order to practice writing, it needs more pages and thus it needs to cut more trees for that. No matter what you do, AI will not stop from cutting more and more trees. It might even replicate itself into machines that will cut trees for him. Now, the only thing that can be done is to turn off the AGI. But, a smart machine would have known the possibility of being turned off and thus will have transferred itself to many other machines over the globe. The important question to ask here is, why it is doing all this? It is not because the machine wants to live, as we humans want to. The only reason why it wants to live is to fulfill its goal.
You simply can't change the fundamental view of the machine. The change can result in, the machine not able to attain its goal or not to that extend (optimal profit). This is the reason why you can't convince a machine to do something that it was asked not to do as its primary task.
You, as a person, are living a life now, and have some fundamental beliefs. Let us consider that you believe in not killing someone. Now, suppose that I give you a pill, and tell you that after taking this pill it will rewire your brain and you will kill the first four people you see. But, after that, you will achieve pure satisfaction and happiness. Now you will definitely not take that pill as it conflicts with something that you believe in now. Also, you will try your best and fight back not to take that pill. The same thing applies to an AGI too. It doesn't matter what your future version will feel or attain after rewiring of the brain (changes in code), it is what and who you are now matters. This video link will help.
I hope it answers your question. There are lots of things to consider here. I have assumed a few things and tried to answer according to that.
There is one more thing. We don’t tell the machines how to do something, instead, we only tell them what to do (at least in the case of an AI). This is because sometimes we don't know the optimal way of solving certain problems. In the case of your question, we don't know what these two machines will do or say to each other. It will be a very interesting thing to hear or watch.
-
$\begingroup$ Your answer is probably incorrect. We have goals and beliefs, but we can be fooled and we can also change our goals and beliefs. Your answer is based on the assumption that our and the AGI's beliefs and goals don't change. Even if they don't change, I don't see why an AGI couldn't dominate/convince another, at least, theoretically, by tricking it into doing something that apparently is beneficial to its goal. Humans regularly do this! The other answer is probably more plausible. $\endgroup$– nbroCommented Jun 7, 2020 at 21:01
-
$\begingroup$ @nbro Humans and AGI are different in many ways. We can be fooled as we don't know all the facts, have same level of reasoning and are mostly bias. We often over-estimate the human intelligence. For instance Bees, they are more advanced than us when it comes to migration, politics and decision making for their colony (reasoning), as opposed to how humans behave or reason. Moving back to AGI, we can't reason that since humans are gullible AGIs will be too. It is an opinion based question and usually, the answer depends on the perspective. Hence assumptions are needed prior to answering. $\endgroup$– UgnesCommented Jun 7, 2020 at 21:16
-
$\begingroup$ In your answer, you say "This is because an AI (or a human) never takes an action that gives a low score on its utility function or is against the fundamental view of the machine (or human)". You also say that AGIs are similar to humans. So, your comment is not consistent with your answer. It's not strictly an opinion-based question. It's a philosophical question, where you can make assumptions (that can be wrong or not), and derive sound conclusions from those assumptions. Your answer is full of issues (some of them I've just mentioned, but I can mention others). $\endgroup$– nbroCommented Jun 7, 2020 at 21:27
-
$\begingroup$ AI cannot change its final goal. It can only change its intermediate goals. For human, end of suffering and increasing happiness is our ultimate goal. But we do change our intermediate goals from time to time. But we simply cannot change our ultimate goal. It's innate like how the AI is designed the first time. The problem happens when we haven't carefully designed the ultimate goal for AI. If the ultimate goal is to create more sandwiches, it can want to change its code to efficiently make more sandwiches. It's wrong to say that changing code will be given a negative score! $\endgroup$– offchanCommented Jun 22, 2020 at 5:20
$\begingroup$
$\endgroup$
0
Strong-narrow AI has reached important milestones recently, but, from what I can tell, we aren't even close to the creation of an AGI, and there are fundamental issues no one currently seems to have any idea of how to solve.
Regarding your question, the simple answer is: it depends on which AI is smarter.
(It's more nuanced than that, but right now the question is very general, which is probably apropos;)
I'd recommend reading William Gibson's Neuromancer trilogy if you're interested in AI-in-a-box (it's sort of about that) and also the more recent Quantum Thief Trilogy by Hannu Rajaniemi to get a sense of the mechanics of the issue.
There is a great deal of academic literature on this subject, but it will likely require a little bit of getting up to speed in terms of basic research into the AI field. Future of Life Institute may not be a bad place to start: https://futureoflife.org/background/benefits-risks-of-artificial-intelligence/
