What is wrong with the idea that the AI will be capable of omniscience?

Question

In the context of artificial intelligence, the singularity refers to the advent of an artificial general intelligence capable of recursive self-improvement, leading to the rapid emergence of artificial superintelligence (ASI), the limits of which are unknown, shortly after technological singularity is achieved. Therefore, these superintelligences would be able to solve problems that we possibly are unable to solve.

According to a poll reported in Future progress in artificial intelligence: A survey of expert opinion (2014)

The median estimate of respondents was for a one in two chance that highlevel machine intelligence will be developed around 2040-2050

which isn't very far away.

What is wrong with the idea that the AI will be capable of omniscience, given that it could benefit us by solving many problems?

Answer 1

I quite like your outlook, and without getting into the details of how a "singularity" may be effected which is covered in numerous other questions, or how consciousness and "omniscience" come into play because consciousness and omniscience are not requirements, I will instead direct you to two key philosophers:

• Phillip K. Dick, for whom the central theme in his famous 1968 book on AI is empathy. (If you haven't read it, I'm not posting a spoiler, but will only say the plot is driven by the concept of Evolutionary Game Theory which was formalized just 5 years later.)

• John Nash, and in particular, the concept of the Nash Equilibrium. (Nash could be said to have mathematically demonstrated that being a "douchebag" is not an optimal strategy. His proof can be used to explain why nuclear détente actually worked, which was counter to the expectation of Von Neumann.)

So when people go nuts, focusing on the "Skynet" mythos under which machines rise up to destroy us, I have to wonder if they're simply not as smart as Nash or as profound as Dick, which might explain their lack of emphasis on what can be called the "Electric Sheep" paradigm.

Answer 2

The question is a good one and on many people's minds. There are a few misconceptions in the line of thought to consider.

• The supremacy of intelligent beings other than humans threaten civilization — Is the imagining of that threat logical? Is that a rational conclusion when human intelligence is the most threatening biological phenomenon in the biosphere today. The intelligence of insects may be more sustainable. The civilizations of ants and termites are certainly more collaborative. World wars and genocide are one of the primary features of human history.
• Artificial intelligence becomes more intelligent than humans — Artificial intelligence is already more intelligent than humans in some respects, which is why we use calculators, automated switching of communications signals instead of operators, and automated mail sorters. In other ways, AI has to cross astronomical distances to begin to approximate human intelligence. We have nothing that even shows a hint of being able to simulate or duplicate in the future the human capacities of reasoning, inventiveness, or compassion.
• The singularity is predicted 2040s — There are over a dozen unrealistic assumptions in those predictions. Look with a critical eye at any one of the arguments behind them and you will fine holes you could fly a 757 through blindfolded.
• Exponential growth in knowledge — There is an exponential growth in information, but the proportion of that information that is legitimately peer reviewed decreases as misinformation, wild conjecture, and fake news increase. My belief is that the amount of information in the world can be approximated by log(n), where n is the population. If I am on track with that relation, the informational value of the average individual is log(n)/n, which decreases as population grows.
• AI will be capable of omniscience — Omniscience is more than answering any question. Omniscience would require that the answering be 100% reliable and 100% accurate, which may require more silicon than exists in the universe. Read Gleick's Chaos if you wish to understand why. One might also argue that an omniscient being would not answer questions on command.

If you want a world that is better than one controlled by human intelligence, then one direction to take is to seek the development of a new species with a more advanced conception of peaceful civilization that can tame and domesticate us like we have done with dogs and cats.

The easier route is just for all of us to die. The biosphere may have been more civilized before we arrived and started killing everything and each other. But that's not my hope.

My recommendation is that we study NI (non-intelligence) and discover how to rid what is stupid from human behavior and geopolitical interaction. That would improve the world far more than distracting and substantially irrelevant machine learning gadgets and apps.

Answer 3

Here are some problems that my ape mind came up with.

1.Smart != all knowing

The AI self improvement explosion smakes it smarter and smarter. Being smarter doesn't mean knowing more facts. I suppose this is quite a picky argument, but I think worth thinking about.

A very smart doctor who doesn't know your history may still make a worse choice than a less intelligent one with better data.

2. Is it all for humans? Is it for all humans?

ASI which reaches a higher level could not be interested in our wellbeing.

A controlled ASI could still work for the benefit of the few only, if these few decide wrong goals we could go backwards.

3 Harsh ASI

A scientific mind in not necessarily full of sympathy or empathy.

4. Being smart and not being clever

Great minds still make mistakes: * in setting their goals, and * in executing the plan to achieve it.

Geat intellect doesn' guarantee lack of shorosightedness or lack of blind spots.

5. Limits

If there are bounds of existence (speed of light type limits) then the AI will be bound by these as well. This may mean that there are things that even ASI won't 'get'. Also, as our mind may have limits based on its structure the next AI may have limits as well - and even if it improves uppn may hit limits that it cannot find solutions to because it's 'too stupid'.

6. We won't get it

ASI's understanding of certain aspects of the world may not be communicative to most humans. We just won't get it (even if we're capable of understanding everything, it doesn't mean we will understand it).

7. How to use it?

We may destroy ourselves, and the AI with the tech it helps us build. It doesn't need to be bombs. It can be geoengineering or wonder drugs.

This is especially intense when the ASI is already powerful but not strong enough to forsee negative consequences (or we'll just ignore it anyway).