(Un-original) idea:

Wouldn't it be cool if we could fact-check using an algorithm that could understand a whole bunch of documents (e.g. scientific papers) as higher-order logic?


What work has been done on this to date?

What I've got so far:

(1) I seem to recall there being prior work to create a subset of English (I think intended for use in scientific writing) that could be easily interpreted by an algorithm. This doesn't quite get us to the algorithm described above (as it's restricted to a subset of English) - but seems pertinent.

(2) Once parsed, I guess a resolution algorithm like that in Prolog could be used to check wether a fact (presumably also inputted as a logical statement) contradicts the logic of the documents?

  • $\begingroup$ Welcome to Ai , As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance $\endgroup$ – quintumnia Nov 13 '18 at 17:36
  • $\begingroup$ Thanks @quintumnia. Would it be possible to discuss in this comment section (or elsewhere) the edits I should make? I'm curious whether asking about prior art ("What work has been done on this to date?") is too vague - as hinted to in the help center guidance? $\endgroup$ – act Nov 13 '18 at 17:41

There are several problems with this, which is why people have been working on tasks like that for about 50 years without getting very far. As you rightly notice, it has been tried in restricted domains, where it works reasonably well. Reason being, there is less ambiguity.

Human language is full of vagueness and ambiguity. We generally have few problems handling that, but we have a wealth of experience available to interpret utterances, and a quite sophisticated 'natural intelligence' to exclude non-sensical interpretations. Computer programs struggle with this. Even on a syntactic level there are many way to interpret a sentence, most of which a human being wouldn't even notice. A famous example sentence in linguistics is Time flies like an arrow., which has four different interpretations to a computer (and a trained linguist), while most human beings would only see one.

And after the vagueness/ambiguity you have the problem of representing meaning. There is a whole field in AI dealing with that. You will find that the expressive power of higher order logic falls short of the expressive power of human language, so there is an issue there.

Then you have the analysis. Prolog, by the way, is a programming language, not a resolution algorithm. How do you decide whether something follows from something else? You would need to understand a whole lot of contextual information to say that It has rained, so the streets are wet. is a logical sentence. And now imagine doing this for scientific papers that are about new discoveries, so you are dealing with unknown phenomena your software hasn't encountered yet.

I cannot answer your question "what work has there been done so far", because there is too much to list. The whole field of natural language processing is concerned among others with the problem of parsing/syntactic/semantic analysis, and there are literally decades of academic work you would need to look at. Same for knowledge representation.

  • $\begingroup$ A simple search in existing papers shows, that the argumentation doesn't fit to reality. It's true, that Academia researched the topic since 50 years, but they have found out some promising results. Not mention them and start a new AI winter because language processing is too complicated, doesn't reflect current research. So i interpret your statement as a philosophical opinion like “there is no need for converting English language into higher-order logic”. $\endgroup$ – Manuel Rodriguez Nov 14 '18 at 10:42
  • 1
    $\begingroup$ @ManuelRodriguez I have worked in the field for 30 years; trust me, I can assess what progress has been made in NLP during that time. "Some promising results" is nowhere near human-level analysis of unrestricted English texts. $\endgroup$ – Oliver Mason Nov 14 '18 at 11:42
  • $\begingroup$ Human-level understanding of natural language is called “Enhanced turing test” and the idea is to evaluate if an animatronics puppet is able to do so. $\endgroup$ – Manuel Rodriguez Nov 14 '18 at 12:24
  • $\begingroup$ @ManuelRodriguez I don't think you have understood my point. But this is not the right place for a discussion. $\endgroup$ – Oliver Mason Nov 14 '18 at 13:52

Converting pure natural language into higher order logic is indeed a difficult task which goes into the direction of projects like Cyc and Watson AI. Instead of explaining how Question & Answering algorithm work, i can give some hints how to convert a simplified language into logic format. Suppose, we have a document which contains a waypoint description. The document was generated by a plan-to-text generator and contains information like: “go ahead, then stop, next crossing left, drive slowly, then stop”. This input stream is natural language, but it is grounded speech from a Journey planner too. This kind of speech can be converted into an abstract model and it is possible to identify if it's wrong or not. Because we can recognize if the way description brings us to the goal or not. That means, the text can be evaluated against the ground model. In case of freely natural language this is much more complicated. The Cyc project was mentioned before, it was an early attempt for grounded natural language. Building a universal ontology around a given language is a demanding task. The easier way is to allow only restricted speech because the model is already there. A simple tagging software which identifies the subject, verb and object in natural language text is not enough for a grounded model. Because non-sense text contains of subject and verbs too but it contradicts sense-making.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.