2
$\begingroup$

I just gave GPT-3 the following prompt (in the playground, using text-davinci-001 with default settings):

What's the German word for "can" in the sentence "The man removes the can."?

The word "can" in this sentence is obviously a noun and not a verb. "the can" is the metal container used to sell coke or beer, not the common verb "can" which means "be able to".

GPT-3 obviously knows this, otherwise it could not translate the sentence correctly in many languages (which it indeed can).

However, to my prompt, it answered:

The word for "can" in the sentence "The man removes the can." is "kann."

("kann" is German for the verb "can" in the sense "be able to")

Did I ask it in a wrong way or is GPT-3 unable to answer questions like this one?

$\endgroup$

1 Answer 1

2
$\begingroup$

GPT-3 obviously knows this

GPT-3 doesn't "know" things in the sense that it has learned specific translations, maths or science etc.

GPT-3 is not a knowledge database or question answering system, but it can behave like one if prompted carefully. Sometimes it is hard to get specific responses. This appears to be one of those cases.

Did I ask it in a wrong way or is GPT-3 unable to answer questions like this one?

GPT-3 doesn't really "answer questions", it completes text, and probablisitically fits that text to match the prompts. The fact that it gets a lot of Q&A, maths or translation-based prompting correct is an interesting side effect of the strong language model. It is not clear what the gaps are in it capabilities to perform secondary tasks like this, but very reasonable to expect some.

In general, you should not expect GPT-3 to automatically pick up on questions with answers, and assume it is answering the question as written. Some reasonable questions might look like the start of a novel or newspaper article to it, or in your case it might be acting as if sentences are only loosely related, like a list of different translations. To reduce the flexibility in possible completions, it is normal to set some context in the first paragraph - a kind of meta-explanation of what the rest of the text is doing.

I managed to get it to work like this:

The following is a translation, with individual words explained.

English: The man removes the can.

German for "can" is "Dose"

My prompts are in bold. I am sure plenty of variations of what I wrote will work for this specific case. I don't think you will find a 100% reliable prompt for arbitrary words from any sentences.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .