Some early AI research, inspired by Claude Shannon's maze learning mouse, Theseus, sought to discover resolutions to conflict. In the case of Theseus, the goal was to resolve the conflict between the simulated hunger and the walls of the maze separating Theseus from the cheese.
Researchers of early AI theorem proving software (mostly written in LISP) sought ways out of mathematical mazes. Finding the cheese, for those theorem provers, was to find a logical proof in the maze of mathematical corridors. The walls were illegal mathematical operations.
In both cases, there is no mandatory opponent, only an individual working toward an achievement. Although others may have the same objective — although some may take the objective as a race and other mice or theorem provers as opponents, that is an arbitrary conception. The only real obstacle is the difficulty imposed by the naturally occurring features of the problem.
Framing Intelligence as an Adaptive Response to Opposition
When Morgenstern and von Neumann's game theory was applied, it was decided that the games would be games of opposition rather than games collaboration, possibly a consequence of the source of funding for much of the research. The software was designed such that the only other intelligence encountered in game play was an opponent and the goal was to annihilate it.
Dialog created by Ted Chiang and Eric Heisserer in Denis Villeneuve's Arrival, 2016, starring Amy Adams, Jeremy Renner, and Forest Whitaker, exposes the folly of approaching new minds with the assumption of adversity.
Following suit. Suits. Suits, honor, flowers. Colonel, those are all tile sets in mah-jongg. God, are they... Are the Chinese using a game to converse with their heptapods?
Well, let's say that I taught them chess instead of English. Every conversation would be a game. Every idea expressed through opposition, victory, defeat. You see the problem? If all I ever gave you was a hammer ...
Everything's a nail.
In the third act, the viewer discovers that the objective of game play was more like Shannon's maze and that projected adversarialism was the only wall in the maze.
Consciously Directing Technological Advancement
This leads to some questions underlying the main question.
- Do we want AI developed in a laboratory setting to be developmentally equivalent to a human child growing up in a war zone?
- Is elimination of the enemy a proven successful strategy in human geopolitical conflict?
- What is the trend of blow-back from annihilation demonstrated throughout history?
- Wouldn't an AI system that seeks to discover win-win scenarios where players work together to overcome shared obstacles be better?
- Should AI research return to its Claude Shannon roots, where the conflict is between obstacles of the physical universe and objectives shared by living things?
- Do we want to imbue into intelligent robots and disembodied intelligence systems an obsession with winning or a more balanced set of objectives that includes collaborating?
- How can AI be developed to win over things like poverty, crime, disease, ignorance, addiction, and economic instability?
- As automated decision making develops, is it time to think about good AI citizenship?
With the power and complexity of AI systems increasing, when we researchers and engineers create loss, error, value, and reward functions, should we develop the discipline of always considering whether we are creating learning incentives that point the AI in the direction of becoming good contributors rather than narrow minded sociopaths?
Must adaptation in artificial mental capacities be neither adversarial (as in a chess or go player) nor codependent (as in Asimov's second law) but rather compassionate, loving, transparent, and interested in growing authentic relationships of mutual benefit?
Central Question and Specifics
Would AI systems not obsessed with winning become better citizens of the world?
What work is being done along these lines and how can best practices be developed to intentionally and responsibly steer AI development?
Although these questions of the direction of technology were topics of science fiction and philosophy in the twentieth century, in the twenty first century, they are necessities of responsible research. It is wise to consider them legal and social questions that deserve the rigor of mathematical formalization and long term risk management, just as should be done with nuclear and genetic technologies.
Addendum Response to Comments
At some point doesn't it boil down to winning against the common enemy, shared obstacles?
Whether the assignment of enemy status to members of the same species is of value to the species is questionable. Evidence indicates human excellence to be primarily the result of collaboration, win-win scenarios, and symbiotic relationships. It is possible that further work on modelling civilization may someday reveal that framing all activity as a competition of some type may be a disease of the collective the primary consequence of which is the erosion of excellence.