My understanding is that there is no singular "The Turing Test Ruleset" and competitions don't all do it the exact same way. Still, I'm wondering about some commonly accepted rules and their nuances. My Googling is not producing any specifics about this.
I think most people agree that the purpose of the humans who talk to the judges is to just act normal. In the so-called "passed Turing test" instances where the humans tried to fool judges into thinking they were AIs, I would say the tests should be thrown out and I've seen critics in the field agree with that.
But this question is more about how the judge should act.
Let's say we are doing a competition where the threshold is for 40% of judges to call an AI a human after 5 minutes of chat.
During the chat, is the judge supposed to try to trick the potential AI into revealing itself, or is the judge supposed to attempt to act as unbiased and natural as possible, as if it is accepted the potential AI is human, and judge based on a completely natural conversation?
For example, asking "What is the value of PI out 10 digits?" or "What is 123456 times 654321?" or "If you saw a bunny and a dime stuck on the road with a truck about to hit them, which one would you pick up?" would be trying to trick AIs into revealing themselves because you are relying on exploiting the fact that the AI might tell you the correct answer or the inhuman answer.
This is as opposed to simply carrying on a natural and normal conversation, with no biases or expectations. If you came upon someone on the street you would not spend 5 minutes trying to hurriedly ask ridiculous interrogation questions in an effort to prove the other person was an AI.
So is the point of a Turing test generally assumed to be an attempt to flush out AIs or an attempt to judge their natural human conversation without interrogation-like prejudice?
