0
$\begingroup$

(The example here isn't meant to offend or to take a political position, I simply needed two recent consecutive former presidents.)

I realize that ChatGPT can make mistakes and can lie. But I thought that it would at least be able to follow simple logic.

Yet in the following, it makes a very significant error in logic:

User

Logically, these two sentences mean exactly the same thing: "Trump was the worst president since Obama." and "Trump was a better president than Obama.". Can an AI tell that their intended meanings are quite different?

ChatGPT

Yes, an AI can understand that the intended meanings of the two sentences are quite different. While the literal meaning of the two sentences may appear similar, the context in which they are used changes the meaning entirely.

The first sentence implies that Trump was a worse president than all the presidents who came after Obama. It also suggests that Obama was a better president than Trump.

On the other hand, the second sentence implies that Trump was a better president than Obama. This statement could be true or false, depending on one's political perspective and beliefs.

An AI model like GPT-3 is capable of analyzing the context and inferring the intended meaning of a sentence based on the words used, grammar, and surrounding text. Therefore, an AI can understand the nuances between these two sentences and interpret them differently based on their context.

The claim that "It also suggests that Obama was a better president than Trump." is the exact opposite of what the first sentence actually suggests.

Given that it is impressively good at so many other things, how does it get such simple, two statement logic wrong?

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
1
$\begingroup$

I would think that a complete answer to this question has two parts:

A) ChatGPT is not strong at logic (although chain-of-thought prompting and similar tricks can help it a lot, especially if we are talking about GPT-4). Many questions don't require deep or very precise logical inference, so that does not stop it from doing fairly well on a broad range of questions.

B) I don't think the two sentences are logically equivalent. The first one claims that Trump has been the worst president among all those presidents who came after Obama. I do not see that this really implies that Obama was worse; the assumption that Obama was worse than Trump is consistent with the claim, but not entailed by it. In the other direction, the claim that Trump was a better president than Obama does clearly not imply that he was the worst since Obama, unless the statement is made at a time when there has been only one president since Obama.

Questioning GPT-4 about this, it seems to me that it understands all of (B) and I think it would probably agree with (A). In any case, it is not hard to find examples of (A).

Improve this answer
$\endgroup$
1
  • $\begingroup$ Also worth noting that when it comes to common misunderstandings and logical inconsistencies, that GPT will learn to "go with the crowd" unless something in the text so far puts it into a different enough state. It's a text prediction engine, and it will predict what a person will write (depending on how often such statements appear in the training data), and only pay attention to what is "correct" if that helps it make such a prediction. $\endgroup$
    – Neil Slater
    Commented Jun 7, 2023 at 6:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .