# Would AI not obsessed with winning be better citizens of the world?

Some early AI research, inspired by Claude Shannon's maze learning mouse, Theseus, sought to discover resolutions to conflict. In the case of Theseus, the goal was to resolve the conflict between the simulated hunger and the walls of the maze separating Theseus from the cheese.

Researchers of early AI theorem proving software (mostly written in LISP) sought ways out of mathematical mazes. Finding the cheese, for those theorem provers, was to find a logical proof in the maze of mathematical corridors. The walls were illegal mathematical operations.

In both cases, there is no mandatory opponent, only an individual working toward an achievement. Although others may have the same objective — although some may take the objective as a race and other mice or theorem provers as opponents, that is an arbitrary conception. The only real obstacle is the difficulty imposed by the naturally occurring features of the problem.

Framing Intelligence as an Adaptive Response to Opposition

When Morgenstern and von Neumann's game theory was applied, it was decided that the games would be games of opposition rather than games collaboration, possibly a consequence of the source of funding for much of the research. The software was designed such that the only other intelligence encountered in game play was an opponent and the goal was to annihilate it.

Dialog created by Ted Chiang and Eric Heisserer in Denis Villeneuve's Arrival, 2016, starring Amy Adams, Jeremy Renner, and Forest Whitaker, exposes the folly of approaching new minds with the assumption of adversity.

LOUISE
Following suit. Suits. Suits, honor, flowers. Colonel, those are all tile sets in mah-jongg. God, are they... Are the Chinese using a game to converse with their heptapods?

WEBER
Maybe. Why?

LOUISE
Well, let's say that I taught them chess instead of English. Every conversation would be a game. Every idea expressed through opposition, victory, defeat. You see the problem? If all I ever gave you was a hammer ...

WEBER
Everything's a nail.

In the third act, the viewer discovers that the objective of game play was more like Shannon's maze and that projected adversarialism was the only wall in the maze.

This leads to some questions underlying the main question.

• Do we want AI developed in a laboratory setting to be developmentally equivalent to a human child growing up in a war zone?
• Is elimination of the enemy a proven successful strategy in human geopolitical conflict?
• What is the trend of blow-back from annihilation demonstrated throughout history?
• Wouldn't an AI system that seeks to discover win-win scenarios where players work together to overcome shared obstacles be better?
• Should AI research return to its Claude Shannon roots, where the conflict is between obstacles of the physical universe and objectives shared by living things?
• Do we want to imbue into intelligent robots and disembodied intelligence systems an obsession with winning or a more balanced set of objectives that includes collaborating?
• How can AI be developed to win over things like poverty, crime, disease, ignorance, addiction, and economic instability?
• As automated decision making develops, is it time to think about good AI citizenship?

With the power and complexity of AI systems increasing, when we researchers and engineers create loss, error, value, and reward functions, should we develop the discipline of always considering whether we are creating learning incentives that point the AI in the direction of becoming good contributors rather than narrow minded sociopaths?

Must adaptation in artificial mental capacities be neither adversarial (as in a chess or go player) nor codependent (as in Asimov's second law) but rather compassionate, loving, transparent, and interested in growing authentic relationships of mutual benefit?

Central Question and Specifics

Would AI systems not obsessed with winning become better citizens of the world?

What work is being done along these lines and how can best practices be developed to intentionally and responsibly steer AI development?

Although these questions of the direction of technology were topics of science fiction and philosophy in the twentieth century, in the twenty first century, they are necessities of responsible research. It is wise to consider them legal and social questions that deserve the rigor of mathematical formalization and long term risk management, just as should be done with nuclear and genetic technologies.

At some point doesn't it boil down to winning against the common enemy, shared obstacles?

Whether the assignment of enemy status to members of the same species is of value to the species is questionable. Evidence indicates human excellence to be primarily the result of collaboration, win-win scenarios, and symbiotic relationships. It is possible that further work on modelling civilization may someday reveal that framing all activity as a competition of some type may be a disease of the collective the primary consequence of which is the erosion of excellence.

• At some point doesn't it boil down to winning against the enemy (ie. shared obstacles) even if AI are collaborating with each other? Thus, solving the problem you are complaining about is essentially solving the problem you desire. If a single AI can 'beat the obstacle' then it simply becomes a matter of changing the utility functions used for evaluating the desirability of outcomes to include other AI and you now have the collaboration you are looking for. It seems that the current state of AI is mostly still at the 'let's see if we can get a single AI to solve the problem' stage. – Dunk Dec 11 '18 at 21:29
• I guess I'm missing something because from a big picture perspective, besides the obvious how to get the AI to communicate effectively with each other, I don't see a lot of difference between having a group of AI collaborate to achieve some desirable goal versus working on a single AI to achieve the same goal. Abstractly, that group of AI can be viewed as just a single AI. Even if there's no 'enemy' there still is always an 'enemy' for achieving a goal. If the goal is to raise food, you still have to contend with the weather, insects, rabbits etc... – Dunk Dec 13 '18 at 0:11
• One reason 'winner/loser' scenarios are more heavily researched in AI is because the current state of the technology is in the learning how to do AI. If there's an obvious winner/loser then you can easily tell how good or bad your algorithm works. Whereas, attempting to create AI algorithms with vague measurements of success/failure accomplishes little because there's only hints of how well an algorithm is working. Nothing conclusive. There's plenty of studies in other disciplines that cover topics you mentioned but they have little relevance because there's no way to prove correctness. – Dunk Dec 13 '18 at 0:17
• @Dunk, winner/loser is only relevant in a zero sum game. If an amoeba divides and offspring find food and divide again, there are 4 winners and no losers. If a million humans work together to reduce carbon emissions, 7 billion win. If two uni-cyclists do their tricks and neither knocks the other into the net the show can still be excellent. Great long distance runners don't often turn around or look to catch up. They run to achieve a target time. That's how world records are set. The first Mars walk may be done such that no country wins a space race, rather all the world triumphs together. – FauChristian Jan 1 at 5:57
• @FauChristian - I disagree with the assertion that winners/losers is only relevant in a zero sum game. There's degrees of winning/losing and winning/losing depends purely on definition.. But using your implication, all your examples boil down to the equivalent of being 'zero sum games'. The amoeba wins if it lives to procreate, loses if it doesn't. Humans win if they reduce carbon emissions, may or may not matter if they don't, regardless of the number of people involved. A good show is a win; a bad one a loss If runners achieve their time they win, otherwise they lose. etc... – Dunk Jan 2 at 18:15

Shannon's mouse was obsessed with the cheese, which was an obsession with winning too, just not winning by prevailing over another mouse. There was no other hungry mouse in the maze.

There is a ring of truth to the idea that, as AI develops, individual AI systems will become more like minds. Artificial minds would need a healthy growing environment that fosters responsible citizenship.

The 2003 documentary film $$The Corporaton$$ written by University of British Columbia law professor Joel Bakan examines the modern corporation as a citizen and demonstrates its sociopathic behavior. The incentives of capitalism were not designed to produce corporate citizenship when the legal framework in which they rose was created. Anti trust law and security exchange oversight was introduced as a governmental retrofit later.

The development of AI could also travel down a road that would have been better if the incentives were set up correctly in advance. Rather than see where a ship without a ruder takes us, it would be better to set them up now. We shouldn't assume someone else will do it. Lawyers, legislators, and science fiction writers do not know enough about AI to be the responsible parties in this case. We must be.

There is also a ring of truth that the importance of the reward functions and the definitions of loss and gain guide whatever artificially warped obsession or artificial mental health the AI will exhibit. Obsession is a narrowly focused reward system that blocks out any balancing factors in a person's objectives. People are healthy when they have several rewards, several penalties, and some of them encourage social responsibility within families, schools, and communities.

Shannon's mouse didn't have a sword in case another came along and rushed to take the cheese, but not having a weapon doesn't make AI mice good citizens. What makes good citizens, what makes a socially responsible AI mouse in the analogy, is if the getting to the cheese is a high priority, but sharing with another hungry mouse, if one is encountered near the cheese, is a higher priority.

For a deep network to acquire these priorities, the loss function would have to introduce a term for loss corresponding to anti-social behavior. If deep reinforcement, the value function would have to introduce an advantage to the shared fulfillment of important goals by $$n$$ people over the fulfillment of $$n$$ important goals by one person.

A concave surface over the dimensions of beings would provide additional incentives for social achievement over individual achievement. This simple relation, where $$n$$ is the number of people and $$v_i$$ is the value assigned for the fulfillment of important goals for the $$i^{th}$$ person, is an example.

$$v = { \Big( \sum_{i=1}^n \large{\sqrt[x]{(v_i)}} \Big) }^{x} \\ x = 1.25$$

When $$x > 1.0$$ social behavior would be trained into the AI system.

Just as cars must have parking breaks, elevators must have safety mechanisms, and boats must have life preservers, there may come a time when AI developers need to build safety into technology and product development. It is definitely a good time to begin thinking along these lines.

The answer to the moral problem in AI is to construct systems who are good but not evil. Let me give an example. An autonomous car can be programmed to behave aggressive or friendly. It's only a simple switch which is adjusted in the car's software and the vehicle will switch into the ghost driver mode. That is equal to wrong behavior. If the switch is put into the friendly mode, the car is driving normal. That means, it stays on his lane and let grandma pass the street.

How can we avoid, that the switch in the car is put into the right position? At first, it's important to know something about programming to be aware what the difference is. If it's clear what AI planning is, it's easier to understand what an autonomous system is doing. Secondly, it's important to raise the abstraction level from a technical point of view into a legal perspective. That means, any driverless car belongs to an owner. He is responsible for the car. The owner is not allowed to do evil things and hurt humans.

• It is impossible to know exactly what any autonomous system will do in every possible situation. At best, all that can be determined is knowing what the system will do in every situation that somebody thought to test for. There's a reason that applications that are far, far, far less complex than autonomous systems are still littered with bugs, which is most likely that very few people are actually good at thinking like a machine. It will be interesting to find out if in the future non-techie grandma is going to be held legally responsible for a bug in her self-driving car. – Dunk Jan 2 at 18:03
• We don't hold people responsible for the havoc their carelessness with their home computers cause, why would you think we'd hold them responsible for their self-driving car where they certainly will know far less about than their home computer. – Dunk Jan 2 at 18:04
• @Dunk The comparison with the owner of a homecomputer make sense. Suppose, somebody doesn't care about Windows security updates, and his PC gets infiltrated by a computer virus. Now, the PC is sending spam messages into the world and the owner doesn't know and doesn't feel responsible. In case of an autonomous car, the car gets maybe a malfunction too and is cruising alone during the night while the owner sleeps. This results into many problems. It's desirable, to discuss the dangers and to think about how to solve it. – Manuel Rodriguez Jan 2 at 18:49