Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
Expressed in my own words:
Suppose we create something that passes all of our tests and is indistinguishable from another human. How can you know if this is truly a conscious being as a human is, or simply a simulation of conscience?
What is the name for this? Where was it first written about?
If something is indistinguishable from a human it is as intelligent as a human. There is no such thing as simulated intelligence.
Consciousness of course is a different matter and I suspect that's what you really have in mind (no pun intended).
The question how to distinguish somebody who is truly conscious from somebody who just acts outwardly like a conscious being is often discussed under the term philosophical zombie.
Your problem closely resembles John Searle's "Chinese Room" argument, which claimed that one (or more) abstract "intelligence tests" lack the discriminitive ability to distinguish between a trivial simulation of intelligence and The Real Thing.
Thus the success of an AI at one (or more) synthetic cognitive test(s) (like the Turing Test, or the games of chess or go) should be seen as insufficient as a surrogate threshold target equal to human-level intelligence. The implication is that a sufficient AI test must require deeper understanding of a problem space from the AI than would be needed to "simply" manipulate (Chinese character) symbols, or play around cleverly but meaninglessly with English words in a convincing manner, as bots have done for years when competing in Turing Test competitions.
