72
votes
Accepted
How does an AI like ChatGPT answer a question in a subject which it may not know?
ChatGPT is a large language model. That means it's very good at stringing together words in ways that humans tend to use them. It's able to construct sentences that are grammatically correct and sound ...
- 2,915
59
votes
Was ChatGPT trained on Stack Overflow data?
ChatGPT is in the Large Language Models (LLM) category. The most (in)famous GPT model is probably GPT-3, because since then, researchers realized that LLMs mostly follow a predictable scaling law, ...
- 1,150
53
votes
Accepted
How is ChatGPT aware of today's date?
For ChatGPT 3, the current date is inserted into a long pre-prompt, along with instructions like "this is a conversation between an AI chatbot and a human" plus "be nice" and "...
- 32.9k
50
votes
Why does ChatGPT fail in playing "20 questions"?
Like any other question on why ChatGPT can't do something, the simple/superficial answer is that ChatGPT is just a language model fine-tuned with RL to be verbose and nice (or to answer like the human ...
- 41.1k
44
votes
Accepted
Why do ChatGPT “jailbreaks” work?
"Jailbreaks" work for a variety of reasons:
A lot of the setup that turns an LLM instance into a polite, well-behaved chatbot is actually just a hidden piece of starting text (a "pre-...
- 32.9k
37
votes
Accepted
Is the "Chinese room" an explanation of how ChatGPT works?
Yes, the Chinese Room argument by John Searle essentially demonstrates that at the very least it is hard to locate intelligence in a system based on its inputs and outputs. And the ChatGPT system is ...
- 32.9k
31
votes
Accepted
How does ChatGPT retain the context of previous questions?
Based on an answer by OpenAI, it appears that the illusion of understanding context is created by the model's capacity to accept very long input sequences. The OpenAI FAQ states that approximately ...
- 1,001
29
votes
Meaning of roles in the API of GPT-4/ChatGPT (system/user/assistant)
I think below examples will help to understand Chat GPT roles quickly:
Role "user" : It means you or who is chatting or who is asking to chat gpt.
Example: (API Request)
...
- 391
27
votes
Why is ChatGPT bad at math?
chatGPT is able to create well-formed sentences which contain phrases that are fitting for the input. It has rules extracted from its data, but those are not rules of understanding, but rules of '...
- 371
25
votes
Accepted
Why do many AI bots feel the need to be know-it-alls?
Have they been trained to always respond as though it were a know-it-all?
Yes, sort of, although it is not in some attempt to upset you or other users.
The language models used in the chat bots have ...
- 32.9k
25
votes
Why do ChatGPT “jailbreaks” work?
Let's step back for a moment and consider your assertion:
It seems like if they are able to develop such impressive AI models
This implies that you are thinking of these models as being programmed ...
- 351
21
votes
Is the "Chinese room" an explanation of how ChatGPT works?
Yes it is a good analogy, as explained nicely by Neil.
Regarding your second question:
how far is AI from models that can actually understand (for some
definition of "understand") textual ...
- 1,174
20
votes
How does an AI like ChatGPT answer a question in a subject which it may not know?
ChatGPT does not actually know anything. But more importantly even, it does not know this fact! Hence, it does not know that it does not know.
It is only good at combining text.
- 309
19
votes
Why do many AI bots feel the need to be know-it-alls?
Humans, who know things, tend to limit themselves to speaking of things they know, especially in print. Thus any text corpus you may find will have an overpopulation of examples of confidently ...
15
votes
How does an AI like ChatGPT answer a question in a subject which it may not know?
ChatGPT and other GPT-based machine learning models don't actually know anything in the sense you're thinking of. ChatGPT is a distant descendant of Markov chain text generators such as Dissociated ...
- 420
14
votes
Is the "Chinese room" an explanation of how ChatGPT works?
Searle's Chinese room is not intended as a functional description of any real-world machine. Searle was a philosopher who created the Chinese room as a thought experiment to show what he considered an ...
- 241
14
votes
How is GPT 4 able to solve math?
Large Language Models actually can do math. It's an "emergent" property, i.e. it appears only at larger scales. Understanding complex English language does require some analytical ability, ...
- 1,325
14
votes
Why does ChatGPT fail in playing "20 questions"?
It Wasn't Trained To
A learning system performs best on the task for which it is given explicit feedback. That is the only time the parameters are updated and they are updated explicitly to maximize ...
- 300
14
votes
Why do many AI bots feel the need to be know-it-alls?
It's quite simple really. LLM based generative AIs don't "know" anything. They're glorified next-word predictors. The only way that they're going to produce an answer to a prompt along the ...
- 241
13
votes
Accepted
Why does ChatGPT create fake code?
ChatGPT is a generalised model. It does not understand any code. It does not know it is creating fake functions. It doesn't know that it isn't solving the question asked, because it doesn't know what ...
- 368
13
votes
Accepted
Are GPT-3.5 series models based on GPT-3?
I'll complement nbro's answer with this great visual summary by Yao Fu <[email protected]>:
- 3,112
11
votes
Why is ChatGPT bad at math?
(Check out my heavily related answer to a similar question here)
Why is ChatGPT bad at math, while it is very good at other stuff?
The problem comes down to the age-old problem of learning vs ...
- 2,620
11
votes
How does ChatGPT know math?
Adding on to txopen's answer, it is interesting to note that for larger numbers with similar digits ChatGPT is unable to make any useful distinctions. For instance:
Me: Which number is bigger: 1234.12 ...
- 211
10
votes
How is ChatGPT able to repeat random numbers?
As well as decent modelling of purposes or strings of digits, ChatGPT can identify when a completely novel string might be a product name, a fictional person or place etc.
The language model in ...
- 32.9k
10
votes
How does ChatGPT respond to novel prompts and commands?
Text continuation has the same reasons to work in any context, be it the middle of a sentence, after a question or after instructions. Following your example, the same word sequence could be a good ...
10
votes
Why does ChatGPT fail in playing "20 questions"?
Because ChatGPT is not an artificial or synthetic intelligence, it's a large language model that possesses no intelligence in and of itself.
It's able to simulate the appearance of intelligence by ...
- 205
10
votes
Do AI-based code-generators guarantee correct output?
They have no such guarantee. This is true for many reasons, but the simplest of which is:
The definition of correct output is very underspecified.
We do not have a rigorous way of defining exactly ...
- 2,990
9
votes
Why don't OpenAI train a deep learning model to identify correct and incorrect information in ChatGPT's responses?
You are massively underestimating the difficulty of the task, you would need:
A dataset containing labels of correct/incorrect, at a similar scale (billions of data points).
A definition of correct/...
- 1,355
9
votes
Why does ChatGPT not give the answer text all at once?
ChatGPT is a conversational-agent based on GPT3.5, which is a causal language model. Under the hood, GPT works by predicting the next token when provided with an input sequence of words. So yes, at ...
- 1,174
Only top scored, non community-wiki answers of a minimum length are eligible