I've asked a question and given a couple answers that propose the OpenAI ChatGPT chatbot has humans in the loop (HITL), and that explains the chatbot's extraordinary abilities.
I've been repeatedly told this is absurd. However, I haven't been given a clear reason why this is absurd, nor how the critic knows HITL is absurd.
Here are my reasons in a nutshell.
ChatGPT's capabilities seem to violate what a neural network can do, and give explicit indication of being human driven. See this answer for a running catalogue of examples I've published.
OpenAI's own documentation states that HITL is best practice, and should be done whenever possible, especially in high stakes domain. OpenAI has a $10B deal on the table with Microsoft, so the ChatGPT chatbot seems like a high stakes domain.
Wherever possible, we recommend having a human review outputs before they are used in practice. This is especially critical in high-stakes domains, and for code generation.
The main criticism of my position is ChatGPT's response speed. However, a fast response is doable if the AI is responding most of the time, while humans monitor, and occasionally intervening, such as with a suffix prompt to guide the AI response. Plus, I've had a number of experiences where ChatGPT does not respond quickly, and it seems like a human is typing.
The other main criticism is that a company like OpenAI would never do something so fraudulent. However, technically OpenAI does strongly imply in their documentation that they use HITL, and OpenAI has never said they don't use HITL. It is only the popular media that claims ChatGPT is pure AI. Additionally, use of HITL under the guise of AI is actually common practice in industry. So, if ChatGPT does have HITL, this would not be fraud on OpenAI's part, and would be in line with AI industry standards.
Now, what I would like in an answer is a clear articulation of the following points:
A) How you know ChatGPT does not have HITL. This needs to be a clear statement in the negative from an official source, or some other evidence based analysis. So far people have only pointed to the ChatGPT main page which states you are interacting with ChatGPT, but this does not say you are not interacting with a HITL, so does not count.
B) Bonus: explain how ChatGPT can repeat long random numbers and recognize its own comments, the two examples I've documented in this answer that seem to defy what a neural network is capable of doing. Please give a technically detailed response, ideally referencing GPT's architecture. Handwaving about magical abilities of transformers and self attention does not count. I need a testable break down of how such a capability would be specifically instantiated using transformers and self attention.
UPDATE: Philosophical answers are not acceptable. Philosophical answers are things like:
- the burden of proof is mine
- AGI is inevitable so we should expect ChatGPT to be getting there
- we can't know for sure, so let's just assume it's AI
Only technical answers are acceptable. Such as:
- OpenAI explicitly says there is no HITL involved in this linked doc
- GPT's transformer architecture can repeat random numbers in this precise manner
- Self recognition can be encoded using embeddings in this precise way, here is a working example