25

If I'm not mistaken you're looking for Roko's Basilisk, in which an otherwise benevolent future AI system tortures simulations of those who did not work to bring the system into existence


23

The technological singularity is a theoretical point in time at which a self-improving artificial general intelligence becomes able to understand and manipulate concepts outside of the human brain's range, that is, the moment when it can understand things humans, by biological design, can't. The fuzziness about the singularity comes from the fact that, from ...


10

I can say that among AI researchers I interact with, it far more common to view it as wild speculation than as settled fact. This is borne out by surveys of AI researchers, with 80% thinking strong forms of AI will emerge in "more than 50 years" or "never", and just a few percent thinking that such forms of AI are "near". Software Developers are not the ...


10

I quite like your outlook, and without getting into the details of how a "singularity" may be effected which is covered in numerous other questions, or how consciousness and "omniscience" come into play because consciousness and omniscience are not requirements, I will instead direct you to two key philosophers: Phillip K. Dick, for whom the central theme ...


10

I believe the term you are looking for is "(technological) singularity". https://en.wikipedia.org/wiki/Technological_singularity


6

The likely expression you are looking for is AI takeover, which is a common topic in science fiction movies, such as 2001: A Space Odyssey and The Matrix, and popular culture. Although the AI takeover is an unlikely scenario in the next years, certain scientists, such as Stephen Hawking, have expressed concerns about it and some philosophers, especially Nick ...


5

It's not just Hawking, you hear variations on this refrain from a lot of people. And given that they're mostly very smart, well educated, well informed people (Elon Musk is another, for example), it probably shouldn't be dismissed out of hand. Anyway, the basic idea seems to be this: If we create "real" artificial intelligence, at some point, it will be ...


4

That would be a no for speed or strength, if you have a super strong entity but it cannot research new materials, it will be quickly limited, same thing for speed, Basically, you need something out of their field to improve them, which makes a runaway improvement impossible. Though, we already have super strong and super fast machines, those are cars, ...


4

Because he did not yet know how far away current AI is... Working in an media AI lab, I get this question a lot. But really... we are still a long way from this. The robots still do everything we detailledly describe them to do. Instead of seeing the robot as intelligent, I would look to the human programmer for where the creativity really happens.


4

As Andrew Ng said, worrying about such threat from AI is like worrying about of overpopulation on Mars. It is science fiction. That being said, given the rise of (much weaker) robots and other (semi-)autonomous agents, the fields of the law and ethics are increasingly incorporating them, e.g. see Roboethics.


4

Given all your assumptions about AI turn out to be true, we would have some kind of utopia, where no one has to work, and there is plenty of everything. Fair enough. Your other assumptions is about human nature, and that is where I'd challenge your conclusion: Just because there are computers better than humans at some task, that does not automatically take ...


3

There is at least one very important and serious AI scientist that apparently believes in the creation of true artificial general intelligence and possibly superintelligence: Jürgen Schmidhuber, who is the co-author of the LSTM, among many other important contributions. In fact, he recently founded NNAISENSE for this ultimate purpose, that is, to build a ...


3

We are biological beings. We will continue to like whatever activates opioid receptors and we will continue to want whatever activates dopamine receptors in the nucleus accumbens. Food, drinks, sex, social dominance, altruistic acts, novelty, drugs of abuse, physical mastery, procreation, socializing, nice sunny weather, sleep when tired, etc, will continue ...


3

The singularity, in the context of AI, is a theoretical event whereby an intelligent system with the following criteria is deployed. Capable of improving the range of its own intelligence or deploying another system with such improved range Willing or compelled to do so Able to do so in the absence of human supervision The improved version sustains criteria ...


3

To put it simply in layman terms, what are the possible threats from AI? Currently, there are no threat. The threat comes if humans create a so-called ultraintelligent machine, a machine that can surpass all intellectual activities by any human. This would be the last invention man would need to do, since this machine is better in inventing machines than ...


3

The concept of "the singularity" is when machines outsmart the humans. Although Stephen Hawking opinion is that this situation is inevitable, but I think it'll be very difficult to reach that point, because every A.I. algorithm needs to be programmed by humans, therefore it would be always more limited than its creator. We would probably know when that ...


3

There are a number of long resources to answer this sort of question: consider Stuart Armstrong's book Smarter Than Us, Nick Bostrom's book Superintelligence, which grew out of this edge.org answer, Tim Urban's explanation, or Michael Cohen's explanation. But here's my (somewhat shorter) answer: intelligence is all about decision-making, and we don't have ...


2

He says this because it can happen. If something becomes smarter than us, why would it continue to serve us? The worst case scenario is that it takes over all manufacturing processes and consumes all matter to convert it into material capable of computation, extending outward infinitely until all matter is consumed. We know that AI is dangerous but it doesn'...


2

Impending or Past? Niel Postman, in his book, Technopoly, argues the preemption of human centered culture to technically driven culture has already occurred. Jaques Ellul, in his book, Technological Society, heaped evidence behind the proposal that technology became autonomous centuries ago. Their arguments are convincing. Some think other criteria must ...


2

What you are talking about is known as the Control Problem. We have our own tag for this specific topic here, which you can use for this and similar questions. How to address the control problem is heavily discussed and still considered unsolved. Two of the important approaches are motivation control and capability control. Motivation control aims at ...


2

May not be quite what you’re looking for, but nonetheless helpful, I hope. The White House a year ago commissioned a report on AI that touches briefly on policy issues.


2

Starting from Arne's point on photography: I'd like to point out how photography changed painting. You can notice that classical painting that tried to be photorealistic stopped to be relevant when photography started, and that several modern movements started, like more abstract paintings, surrealism, or cubism. One can see art as a medium to brag "i'm ...


2

Assuming super-intelligence is possible, the answer is probably yes and no. Yes in Kurzweil-like scenarios, where super-intelligence is an extension of human beings by technology (we are already in to some extent). Then control follows, as super-intelligence depends on us. It would extend our capabilities, such as speed of processing, extent of processing, ...


2

If the technological singularity always leads to the extinction of all intelligent life, then yes. If the technological singularity always leads to intelligent life migrating into higher planes of existence that aren't accessible to us right now, then also yes. Otherwise it is exactly the assumption of unbridled technological progress that makes the Fermi ...


2

Note that the statement says nothing directly about the limit of intelligence, nor even about the limit of computational intelligence - but about the limit of computing power. Perhaps the sentence "the speed of light puts a strict upper bound on how much computing can be done" needs a better explanation: The authors are probably referring to Bremermann's ...


2

The definition of "technological singularity" answers the question: The technological singularity (also, simply, the singularity) is the hypothesis that the invention of artificial superintelligence will abruptly trigger runaway technological growth, resulting in unfathomable changes to human civilization. (wiki) note the order of facts is "...


2

I found someone that has done this thing! You can hear a good explanation in Marcus Hutter's answer to this question about rewards given to AIXI. He describes a work that seems to be referring to this paper: Universal Knowledge-Seeking Agents for Stochastic Environments I'll edit this answer later with a full explanation of the approach, but essentially ...


2

Not to my knowledge. The problem is that this is such an enormous task, it cannot really be tackled at once. So the obvious solution is to reduce the scope. In early AI people were using toy domains, whereas nowadays AI systems work more generally (but still perform better if the domain is restricted). So while (slow) progress is being made putting the ...


2

Would OpenCog fit the bill? I have had tremendous amounts of trouble building up the demos, which include some non-AGI stuff, but if I’ve read the manual correctly, I think there’s something here — https://opencog.org/


1

Here is one idea. I'll start with a more specific "mathematical singularity", defined as an algorithm that can do the following in N hours or less (for all $N >= 1$): State equivalent versions (up to notional differences) of all mathematical theorems/conjectures that humans will read and understand in N*20 years after 2018 that can be stated formally in ...


Only top voted, non community-wiki answers of a minimum length are eligible