-3
$\begingroup$

My question to ChatGPT, after reading about a vole (as a German I had to look it up) and immediately associating "mole":

"Here are pairs of animal names which differ only in the first letter: monkey-donkey, mouse-louse, and even bat-rat-cat. ("gnat" wouldn't fit, since two letters differ, and gnat-goat is halfway OK, since the second letter differs.) Do you know more examples?"

And the answer was:

Yes, there are several more examples of animal pairs that differ only in the first letter. Here are a few:

    Cow-now
    Hen-pen
    Hare-dare
    Ant-pant
    Duck-tuck
    Hog-dog
    Seal-real
    Snake-rake
    Yak-lak
    Ox-fox

These are just a few examples, but there might be more out there!

2/10. I halfway expected that ChatGPT would fail epically, since the question needs some creativity. Still, even a computer could grep its animal list, regex match over it and give only the hits. I don't see either how ChatGPT can misinterpret the question by lifting the requirement "animal" for the rhyme pair.

Do you have an explanation? ("Hallucination" is a bit unspecific.)

$\endgroup$
3
  • 1
    $\begingroup$ This is a duplicate of any question about why ChatGPT fails. Essentially, it fails because it's just a statistical model that just produces the most likely output given the training data. In the training data, there were not enough examples to produce an output that you expect. ChatGPT does not understand the world as we do. It just produces words, which are not grounded in any world concept. It's indeed a statistical parrot and people hope it will just be clever, but it won't in many cases. For example, I often ask it to return answers without quotes and it still does. $\endgroup$
    – nbro
    Commented Jan 14 at 3:04
  • 1
    $\begingroup$ So, in your case, it probably does not even understand what an animal is. You probably can get a better output if you first explain or ask it to explain what an animal is, so that it's more context (i.e. you condition it), and provide some few-shot examples, but still ChatGPT is not good for understanding or facts, it's good just for creativity (if we can say that) and summarization (sometimes). $\endgroup$
    – nbro
    Commented Jan 14 at 3:06
  • $\begingroup$ @nbro: Could you make an answer elucidating exactly what ChatGPT can and what it can't? As I understand you, it doesn't even have a fact base (CAT is-instance-of ANIMAL) but only an association?! $\endgroup$ Commented Jan 14 at 8:40

1 Answer 1

3
$\begingroup$

Be careful attributing too much to the model behind ChatGPT. It's a language model, it's job is to predict the next word in the sentence, that's it. Any perceived creativity is most likely an artifact of it being in the training set or a pattern found in the training set, it should not be expected or depended upon. Prompting can sometimes activate different neurons, so maybe a slightly different prompt would help. Here is my log with GPT-3.5:

Are there two animals whose name differ by only the first letter?

ChatGPT

Yes, there are many pairs of animals whose names differ by only the first letter. Here are a few examples:

Cat and Bat

Dog and Hog

Rat and Bat

Cow and Owl

Fox and Ox

There is still an error but it's closer than the one it gave you.

As for the second question about why it might drop the animal part of your prompt, the GPT models have an attention mechanism. It's roughly an encoding of what was important during training in predicting the next word. This is often the same thing that's important to the user but it doesn't have to be. If the user's prompting is significantly different than data found in it's training set, the attention may not perform well. This is roughly what someone might call misalignment and is an issue with all AI models, not just language models.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged .