0
$\begingroup$

I've asked a question and given a couple answers that propose the OpenAI ChatGPT chatbot has humans in the loop (HITL), and that explains the chatbot's extraordinary abilities.

I've been repeatedly told this is absurd. However, I haven't been given a clear reason why this is absurd, nor how the critic knows HITL is absurd.

Here are my reasons in a nutshell.

  1. ChatGPT's capabilities seem to violate what a neural network can do, and give explicit indication of being human driven. See this answer for a running catalogue of examples I've published.

  2. OpenAI's own documentation states that HITL is best practice, and should be done whenever possible, especially in high stakes domain. OpenAI has a $10B deal on the table with Microsoft, so the ChatGPT chatbot seems like a high stakes domain.

Wherever possible, we recommend having a human review outputs before they are used in practice. This is especially critical in high-stakes domains, and for code generation.

  1. The main criticism of my position is ChatGPT's response speed. However, a fast response is doable if the AI is responding most of the time, while humans monitor, and occasionally intervening, such as with a suffix prompt to guide the AI response. Plus, I've had a number of experiences where ChatGPT does not respond quickly, and it seems like a human is typing.

  2. The other main criticism is that a company like OpenAI would never do something so fraudulent. However, technically OpenAI does strongly imply in their documentation that they use HITL, and OpenAI has never said they don't use HITL. It is only the popular media that claims ChatGPT is pure AI. Additionally, use of HITL under the guise of AI is actually common practice in industry. So, if ChatGPT does have HITL, this would not be fraud on OpenAI's part, and would be in line with AI industry standards.

Now, what I would like in an answer is a clear articulation of the following points:

A) How you know ChatGPT does not have HITL. This needs to be a clear statement in the negative from an official source, or some other evidence based analysis. So far people have only pointed to the ChatGPT main page which states you are interacting with ChatGPT, but this does not say you are not interacting with a HITL, so does not count.

B) Bonus: explain how ChatGPT can repeat long random numbers and recognize its own comments, the two examples I've documented in this answer that seem to defy what a neural network is capable of doing. Please give a technically detailed response, ideally referencing GPT's architecture. Handwaving about magical abilities of transformers and self attention does not count. I need a testable break down of how such a capability would be specifically instantiated using transformers and self attention.

UPDATE: Philosophical answers are not acceptable. Philosophical answers are things like:

  • the burden of proof is mine
  • AGI is inevitable so we should expect ChatGPT to be getting there
  • we can't know for sure, so let's just assume it's AI

Only technical answers are acceptable. Such as:

  • OpenAI explicitly says there is no HITL involved in this linked doc
  • GPT's transformer architecture can repeat random numbers in this precise manner
  • Self recognition can be encoded using embeddings in this precise way, here is a working example
Improve this question
$\endgroup$
10
  • 4
    $\begingroup$ I've played a little bit with ChatGPT. It's sometimes remarkable and it's possible that there's some human that reviews the answers, but I doubt there's a human that reviews every answer. It's possible that some humans review some answers in order to improve the model itself and some answers may be rewritten if they are detected to contain toxic content (OpenAI has apparently an API that detects this). We cannot know what OpenAI is really doing unless they make it open-source, which could be a problem, if people misuse it, and could also be a problem for them (fewer millions in the bank). $\endgroup$
    – nbro
    Jan 22 at 23:43
  • 4
    $\begingroup$ What I did was just an educated guess. We cannot know sure. I don't know why you think that what ChatGPT is doing is beyond what a neural network could do. ChatGPT sometimes fails or gives incorrect answers, and cuts answers, which suggests that there's a model that is doing the job, which does not imply that the model's answers are not reviewed or rewritten occasionally. $\endgroup$
    – nbro
    Jan 22 at 23:48
  • $\begingroup$ @nbro like you noticed, sometimes ChatGPT seems somewhat suspect. Note in my question I propose that AI does most of the answering, with some human intervention. And like you mention, not every question needs to be monitored. They probably have some alerting system that flags a request or response for human review. $\endgroup$
    – yters
    Jan 23 at 0:47
  • 1
    $\begingroup$ On your point 3. There could also be human selection of pre-generated answers according to labels prompted to the human, like "sure", "likely", "possible", "impossible". The question could be summarized to the unknown part, for speed. Note a common starter in sentences is "it is likely that..." "it is possible that", besides yes/no. This would help get factual accuracy without a human reading the full answer nor typing it. The human contribution would still be tiny (2 bits per answer) and fast to provide (one click) respect to the total information output. Wouldn't this match the RLHF idea? $\endgroup$
    – Jaume Oliver Lafont
    Jan 23 at 11:36
  • $\begingroup$ @JaumeOliverLafont yes, exactly, and also OpenAI's API has a suffix prompt option, so the human could contribute a few keywords, and GPT surrounds the keywords with a big bunch of autogenerated text. $\endgroup$
    – yters
    Jan 23 at 15:32

1 Answer 1

Reset to default
8
$\begingroup$

Until you can prove that OpenAI has HITL in ChatGPT, it is just an idea with no basis. It's not up to us to disprove it, it's up to you to prove it.

Let me address your points:

  1. You seem to be basing your reasoning on an assumption of what neural networks can or can not do. What are those capabilities, and what is your source? Then we can talk about what ChatGPT seems to violate.
  2. OpenAI states that HITL is the best practice for its end users like me and you (specifically for API users), you have provided no evidence that OpenAI themselves use a HITL in ChatGPT.
  3. "I've had a number of experiences where ChatGPT does not respond quickly, and it seems like a human is typing" That is because of plain old traffic. ChatGPT is a free service and is also extremely popular, making the news several times. Source.
  4. What one company does is not a reliable source of what another company is doing. ChatGPT is top of the line at the moment in terms of AI capabilities. One of your sources is from 2016, and none of them are about OpenAI. OpenAI has whitepapers to back up the claims it makes, and there is a clear capability progression in GPT 1, 2 & 3. Nothing ChatGPT does seems impossible to people who have been keeping up with OpenAI and deeply understand how neural networks work.

For your 2 points at the end:

a) "How you know ChatGPT does not have HITL. This needs to be a clear statement in the negative from an official source, or some other evidence based analysis. So far people have only pointed to the ChatGPT main page which states you are interacting with ChatGPT, but this does not say you are not interacting with a HITL, so does not count."

From ChatGPT:

The statement above violates the principle of burden of proof, which states that the person making a claim has the responsibility to provide evidence to support it. In this case, the person is asking for clear evidence that ChatGPT does not have a human-in-the-loop (HITL), and is suggesting that the statement on the main page is not sufficient evidence.

As I said before: OpenAI states that HITL is the best practice for its end users, you have provided no evidence that OpenAI themselves use a HITL in ChatGPT.

It's up to you to prove otherwise, not the general public.

Edit: This is not a helpful response to the question point b). Ignore this.

b) "Bonus: explain how ChatGPT can repeat long random numbers and recognize its own comments, the two examples I've documented in this answer that seem to defy what a neural network is capable of doing. Please give a technically detailed response. Handwaving about magical abilities of transformers and self attention does not count. I need a testable break down of how such a capability would be specifically instantiated using transformers and self attention."

By passing a representation of the dialog you have had along with your prompt. I don't know exactly how it is done, but it doesn't require a conspiracy. This answer does a decent job tackling the question. No magic necessary.

As a bonus:

Prompt What is the burden of proof principle in arguments?

ChatGPT The burden of proof principle in arguments refers to the idea that the person making a claim or assertion has the responsibility to provide evidence or proof to support it. This principle is important in a logical and fair argument because it ensures that both parties are held accountable for providing evidence to support their claims. Without the burden of proof principle, one party could make an unfounded claim without having to provide any evidence to support it. This principle applies to both formal and informal arguments, and is often used in legal proceedings and scientific research.

Improve this answer
$\endgroup$
9
  • $\begingroup$ Yes, I was expecting a "I don't have a burden of proof, you have a burden of proof" sort of response. Which of course doesn't answer the question :) So, my takeaway from all this is that no one has a good positive reason to believe there is no HITL. I, on the other hand, have offered a number of positive arguments for my position. So, if we start with the principle of maximum entropy, i.e. weighting each hypothesis equally, and then updating based on evidence offered, my hypothesis is currently in the lead. $\endgroup$
    – yters
    Jan 23 at 0:49
  • $\begingroup$ Also, per your answer to B it doesn't address either the random number or self recognition problems. There is no single token for an ad hoc random number. Instead, it must be assembled from many individual number tokens on a probabilistic basis. So, the probability of repeating a long random number exactly is near zero. Yet ChatGPT always repeats the long random numbers exactly. Clearly, that's not being done by a neural network. There is no possible explanation given GPT's published architecture. $\endgroup$
    – yters
    Jan 23 at 1:05
  • $\begingroup$ Self recognition is a different problem, it requires a representation of a representation. The ChatGPT style would be a combination of certain embeddings as well as attention weights. So, to merely represent the style, let alone recognized it, requires a second order neural network that can embed embeddings and attention weights. Another thing clearly impossible with the published architecture. $\endgroup$
    – yters
    Jan 23 at 1:06
  • 3
    $\begingroup$ "So, if we start with the principle of maximum entropy, i.e. weighting each hypothesis equally, and then updating based on evidence offered, my hypothesis is currently in the lead." No, I don't accept that. Whether you like it or not, the burden of proof is on you. So, your hypothesis in not in the lead. Do you understand why the burden of proof principle is important? If you don't then I don't see myself continuing this conversation. $\endgroup$
    – GalacticRaph
    Jan 23 at 1:21
  • 1
    $\begingroup$ "Self recognition is a different problem, it requires a representation of a representation. The ChatGPT style would be a combination of certain embeddings as well as attention weights. So, to merely represent the style, let alone recognized it, requires a second order neural network that can embed embeddings and attention weights. Another thing clearly impossible with the published architecture" This is a great point! However, OpenAI has not, to my knowledge, released a whitepaper on ChatGPT. So who knows? $\endgroup$
    – GalacticRaph
    Jan 23 at 1:22

Not the answer you're looking for? Browse other questions tagged .