Are any AI systems available, or in development, for finding and analysing fallacious inference in natural language text?

Question

Poor reasoning, and ignorance in general, is the source of a lot of suffering and evil. Covertly erroneous logic is often used in manipulation. And much of this broken thought is being used directly in the training of AI.

There has been talk of, and development in, fact-checking, such as for language transformers. But what about reasoning?

The function in mind is specifically being able to process a potentially large text body, analysing all logic and implied relations for fallacy and other misleading reasoning. Perhaps shades of colour could indicate level of error. A bonus would be output listing and explaining the mistakes, maybe like compiler errors -- "fallacy x between premise y and conclusion z".

Are any AI systems available, or in development, for finding and analysing fallacious inference in natural language text?

Answer 1

There's some impressive work going on at IBM under the name of "Project Debater", which already produced some impressive results, that you can check in this video.

The project is about creating a system capable of debating with human on random topics by scraping the internet to get ground knowledge (mainly scientific papers, not random gibberish of course). The system is pretty massive, it include speech synthesis and lot of other components, what's interesting for you are the parts I highlighted in red, especially those on the left that contribute to generate the counter arguments to the human opponent (rebuttal construction).

I won't go into the details cause all those three topics constitute entire branches of Natural Language Processing. But you can easily see how combined together they try to accomplish a rough version of deductive reasoning:

• claim detection: identifying the main argument of a sentence/document
• evidence detection: identifying what facts are presented along with the argument
• stance detection: identifying the positive/neutral/negative stance of each evidence (and possibly opinion based facts presented) toward the claim has been made.

Assuming that we managed to extract all this information from a text (big time, unfortunately not easy at all) then we can build reasoning out of it. For example:

• Public schools are worse than private ones for children education -> detected negative claim
• Researches show that there's an equal amount of graduated students coming from public and private schools. -> detected evidence
• The previous evidence has a positive stance toward the initial claim -> stance detection

Putting everything together, since we have a negative claim followed by a positive evidence toward the same fact, we just found out that the document contains a contradiction.