I know some people will try to crucify me for asking such question here, since it's too generic, but ill give it a shot.

So basically what I'm after in trying to find if there any tools exist (simply white papers are welcome too) for creating chatbots based on a plain text corpus, so that questions could be asked about it. That is, there is some textual data (like articles), but there are no question, answer pairs to train a conventional model with, like seq2seq. I understand that there are ways to achieve this by using tools to extract intents,entities from a question and match it with paragraphs, articles (based on indexes and topics), but to my understanding this creates more of a NLP search bot, rather than a chatbot, which provides answers in conversation-like manner.

Thanks in advance.

  • 1
    $\begingroup$ I would say this is currently not possible. You could in theory mine factual data from text (which is a hard problem in itself), but to package that up in a natural conversation framework without providing examples of real conversation will not work. $\endgroup$ – Oliver Mason Jun 5 '18 at 16:16

A chatbot is a dialogue system which provides interactive intelligence to a human user. Interaction is equal to what the Turing Test is asking for. If the human is not able to detect if his opponent is real or not, the dialogue system has passed the Turing Test. A possible application of a chatbot could be a job interview, that means the bot is trying to identify a human expert (recruiting task).

Usually the creation of chatbots is done via rapid prototyping which is using a collected dialogue corpus. That is a number of examples from real dialogues. This dialogue corpus is translated into the AIML format, which is a standard description for creating chatbots. The problem is: what happens if no dialogue corpus is available?

The answer is, to use existing plain text as input, parse the text and annotate it, then transfer the annotated text into a formal ontology (OWL) and extend this model with external sources (natural language grounding). Against this newly created model some example queries can be executed and the traces of these queries are the collected roleplay. I know, this description is a bit vague, the problem is, that the topic is under research and the number of published literature is low. But it would be a great project for future research.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.