2
$\begingroup$

Poor reasoning, and ignorance in general, is the source of a lot of suffering and evil. Covertly erroneous logic is often used in manipulation. And much of this broken thought is being used directly in the training of AI.

There has been talk of, and development in, fact-checking, such as for language transformers. But what about reasoning?

The function in mind is specifically being able to process a potentially large text body, analysing all logic and implied relations for fallacy and other misleading reasoning. Perhaps shades of colour could indicate level of error. A bonus would be output listing and explaining the mistakes, maybe like compiler errors -- "fallacy x between premise y and conclusion z".

Are any AI systems available, or in development, for finding and analysing fallacious inference in natural language text?

$\endgroup$
1
  • $\begingroup$ It is very complicated to even recognise inferences; deciding whether they are fallacious would require understanding of how the world works, and AI is a long way away from that. So the answer, I'm afraid, is no. $\endgroup$ Mar 21, 2022 at 9:13

2 Answers 2

2
$\begingroup$

There's some impressive work going on at IBM under the name of "Project Debater", which already produced some impressive results, that you can check in this video.

The project is about creating a system capable of debating with human on random topics by scraping the internet to get ground knowledge (mainly scientific papers, not random gibberish of course). The system is pretty massive, it include speech synthesis and lot of other components, what's interesting for you are the parts I highlighted in red, especially those on the left that contribute to generate the counter arguments to the human opponent (rebuttal construction).

I won't go into the details cause all those three topics constitute entire branches of Natural Language Processing. But you can easily see how combined together they try to accomplish a rough version of deductive reasoning:

  • claim detection: identifying the main argument of a sentence/document
  • evidence detection: identifying what facts are presented along with the argument
  • stance detection: identifying the positive/neutral/negative stance of each evidence (and possibly opinion based facts presented) toward the claim has been made.

Assuming that we managed to extract all this information from a text (big time, unfortunately not easy at all) then we can build reasoning out of it. For example:

  • Public schools are worse than private ones for children education -> detected negative claim
  • Researches show that there's an equal amount of graduated students coming from public and private schools. -> detected evidence
  • The previous evidence has a positive stance toward the initial claim -> stance detection

Putting everything together, since we have a negative claim followed by a positive evidence toward the same fact, we just found out that the document contains a contradiction.

enter image description here

$\endgroup$
1
$\begingroup$

Yes, current large language models since GPT-4 are able to find errors in reasoning, probably zero-shot, not much prompt engineering needed.

Giving a precise probability for something being wrong is much harder, even if the the model knows it, it is hard to query. Rough estimates should be easy.


Some examples using GPT-4:

Prompt:

Alice had blue shoes. Therefore, Bob had to eat the cake.

Analyze the reasoning.

Result:

The reasoning presented in the statement is illogical and does not follow a coherent thought process. There is no apparent connection or correlation between Alice having blue shoes and Bob having to eat the cake. The conclusion does not follow from the premise, and the argument lacks a proper cause-and-effect relationship.

Starting a new chat:

Prompt:

Alice had birthday cake. Therefore, Alice had to eat the cake.

Result:

It is not necessarily true that Alice had to eat the cake just because she had a birthday cake. While it is common for people to eat cake on their birthdays, Alice could have chosen not to eat it, shared it with others, or saved it for later.

Starting a new chat:

Prompt:

Alice had blue shoes. Therefore, Alice had to shoe the blues.

Result:

This statement is a play on words. "Alice had blue shoes" means that Alice owned a pair of blue-colored shoes. "Shoe the blues" is a pun on the phrase "sing the blues," which means to express one's troubles or sadness, often through music. In this case, the pun implies that because Alice had blue shoes, she had to express her sadness or troubles, possibly through wearing her blue shoes.

These examples are not cherry picked.

$\endgroup$
13
  • $\begingroup$ With that in mind, can any current models convert real-world natural argumentation into something more formal, like a list of premises and conclusions? An example would be taking an Online argument thread, or political debate, then summarising its arguments cleanly. $\endgroup$
    – Michael
    Mar 26 at 18:46
  • $\begingroup$ Do you get a similar result if you substitute other words for Alice/blue/shoes/Bob/eat/cake, maintaining the same form while getting closer to a real argument? What if you ask "Alice had birthday cake. Therefore, Alice had to eat the cake." or "Alice had blue shoes. Therefore, Alice had to shoe the blues."? $\endgroup$
    – user253751
    Mar 28 at 13:00
  • $\begingroup$ @user253751 I added your examples to the answer, thanks for asking. It shows that I did not cherry pick the question. I did not cherry pick the answers either. $\endgroup$ Mar 28 at 16:43
  • $\begingroup$ @Michael Yes, summarizing is one of the most important tasks it is used for, $\endgroup$ Mar 28 at 16:44
  • $\begingroup$ okay, that went better than I expected. Very impressive. It's too bad I can't also think of one that is actually true and not a fallacy. The careful reader might notice the last one is wrong, but that's hardly surprising when it was a nonsense phrase to begin with. $\endgroup$
    – user253751
    Mar 28 at 16:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .