Hot answers tagged

142

This classic problem exhibits a basic misunderstanding of what an artificial general intelligence would likely entail. First, consider this programmer's joke: The programmer's wife couldn't take it anymore. Every discussion with her husband turned into an argument over semantics, picking over every piece of trivial detail. One day she sent him to the ...


52

This popular meme originated in the era of 'Good Old Fashioned AI' (GOFAI), when the belief was that intelligence could usefully be defined entirely in terms of logic. The meme seems to rely on the AI parsing commands using a theorem prover, the idea presumably being that it's driven into some kind of infinite loop by trying to prove an unprovable or ...


26

I see several good answers, but most are assuming that inferential infinite loop is a thing of the past, only related to logical AI (the famous GOFAI). But it's not. An infinite loop can happen in any program, whether it's adaptive or not. And as @SQLServerSteve pointed out, humans can also get stuck in obsessions and paradoxes. Modern approaches are ...


19

The halting problem says that it's not possible to determine whether any given algorithm will halt. Therefore, while a machine could conceivably recognize some "traps", it couldn't test arbitrary execution plans and return EWOULDHANG for non-halting ones. The easiest solution to avoid hanging would be a timeout. For example, the AI controller process could ...


15

Another similar question might be: "What vulnerabilities does an AI have?" "Kill" may not make as much sense with respect to an AI. What we really want to know is, relative to some goal, in what ways can that goal be subverted? Can a paradox subvert an agent's logic? What is a paradox, other than some expression that subverts some kind of expected behavior?...


10

Nope in the same way a circular reference on a spreadsheet cannot kill a computer. All loops cyclic dependencies, can be detected (you can always check if a finite Turing machine enters the same state twice). Even stronger assumption, if the machine is based on machine learning (where it is trained to recognize patterns), any sentence is just a pattern to ...


10

No. This is easily prevented by a number of safety mechanisms that are sure to be present in a well-designed AI system. For example, a timeout could be used. If the AI system is not able to handle a statement or a command after a certain amount of time, the AI could ignore the statement and move on. If a paradox ever does cause an AI to freeze, it's more ...


8

AIs used in computer games already encounter similar problems, and if well designed, they can avoid it easily. The simplest method to avoid freezing in case of an unsolvable problem is to have a timer interrupt the calculation if it runs too long. Usually encountered in strategy games, and more specifically in turn based tactics, if a specific move the ...


7

The authors do actually give an English definition in terms of the well-known agent formulation of AI: We intend this usage to be intuitive: death means that one sees no more percepts, and takes no more actions. It would seem that this becomes possible for a reinforcement learning agent such as AIXI in a formulation that uses semi-measures of ...


5

It seems to me this is just a probabilistic equation like any other. I'm sure Google handles paradoxical solution sets Billions of times a day, and I can't say my spam filter has ever caused a (ahem) stack overflow. Perhaps one day our programming model will break in a way we can't understand and then all bets are off. But I do take exception to the ...


5

Well, the issue of anthropomorphizing the AI aside, the answer is "yes, sort of." Depending on how the AI is implemented, it's reasonable to say it could get "stuck" trying to resolve a paradox, or decide an undecidable problem. And that's the core issue - decidability. A computer can chew on an undecidable program forever (in principle) without ...


4

Killing AI by 'thinking' about a paradox would be called a bug in implementation of that AI, so it's possible (depending how it's being done), but less likely. Most of AI implementation operate in non-linear code, therefore there is no such thing as an infinite loop which can "freeze" the computer's 'consciousness', unless code managing such AI consist ...


3

Death as we know it for natural life is terminal. That is once dead, natural life cannot come back (at least in the current understanding and with current technologies---some people believe otherwise). Death for AI is trickier. There may be only one scenario: Global destruction: Extreme scenario where everything supporting the existence of an AI disappears. ...


2

If AI arises from a replicable manufacturing process (e.g. as with modern computers), then it will presumably be possible to take a snapshot of the state of an AI and replicate it without error on some other mechanism. For such a construct, 'death' doesn't mean the same as it currently does for us fleshy organics: multiple clones of an AI could presumably ...


1

The AI agent can be designed in such a way that it could consist of two major components: The free-will component expands the experience of the AI agent and produce outputs based on artificially generated thought input. The hard-wired component that the agent cannot modify by itself. This could include a set of secured code to action sequence mapping. One ...


1

Following on from your own software verification-based answer to this question, it seems clear that ordinary (i.e. physical), notions of death or imprisonment are not strong enough constraints on an AI (since it's always possible that a state snapshot has been or can be made). What is therefore needed is some means of moving the AI into a 'mentally ...


1

"Death" exists as a single concept because the underlying reality that it's describing is closely clumped together, and our definition has changed with our ability to change that reality. It seems more reasonable that the various sorts of things that could be considered 'death' will be split apart, and a different word will be used to refer to a system with ...


Only top voted, non community-wiki answers of a minimum length are eligible