I'm looking to build a conversational AI. The goal is to create an AI that is your friend (as opposed to personal assistants that are meant for carrying out commands like "send a text, play that song..." etc).

What are the underlying techniques involved ? Is there any good open source project I can build on top of. What kind of training data do I need ? I looked at Jasper - doesnt look like a good fit based on a shallow evaluation.

I'm imagining that the user text needs to be parsed out (like for eg using Google's syntaxnet). May be it queries some knowledge graph and then uses NLG. Or it uses some kind of seq-seq modelling like LSTM/RNN to generate responses.

  • $\begingroup$ It is a good but broad question. You will probably get better answers by making it more specific. Prior work spans back to Eliza (Meta-x doctor in Emacs, the doctor.el file in the standard distribution) which is more rules-based, you can go the way of Bayes/Markov models or neural networks as you suggest. $\endgroup$
    – mjul
    Nov 5 '17 at 14:59
  • $\begingroup$ Try to be precise and straight to point,inline with community guidelines.Honestly,analyse critically,the second paragraph;you need to make some corrections. $\endgroup$
    – quintumnia
    Nov 5 '17 at 19:27
  • $\begingroup$ @mjul Your fellow human needs support. $\endgroup$
    – quintumnia
    Nov 5 '17 at 19:29

Before we commence I will recommend that you refer to similar questions on the network i.e https://stackoverflow.com/questions/9706769/any-tutorials-for-developing-chatbots , https://stackoverflow.com/questions/55042/how-can-i-program-a-simple-chat-bot-ai and https://stackoverflow.com/questions/1748887/how-to-create-a-smart-chat-bot

In my answer I follow the approach laid out on two outstanding articles. Chatbots with Seq2Seq (Suriyadeepan R 2016) and Deep Learning for Chatbots Part 1 & 2 (Benny B 2016) which are available here http://suriyadeepan.github.io/2016-06-28-easy-seq2seq/ and here http://www.wildml.com/2016/04/deep-learning-for-chatbots-part-1-introduction/ respectively.

Chatbots are also known as conversational agents or dialog systems. Your goal is to build a friendly AI. Modeling personality may appear straight forward, however incorporating such knowledge into a model is still a challenging problem. This is because the models are trained on a lot of data from different sources and users. Below is a link to a paper on building such a Persona-Based Neural Conversation Model https://arxiv.org/abs/1603.06155

There are two important concepts that one needs to understand when learning about chatbots. That is the chatbot publishing platform and the chatbot development platform. A chatbot publishing platform is a medium through which the chatbot can be accessed by the users i.e. FB messenger, LINE, Telegram or WhatsApp, on the other hand a chatbot development platform is a tool that can be used to create a chatbot. Chatbot development platforms i.e. Beep Boop, Flow XO, Botsify and Chatfuel can be used to build chatbots without writing any code, this is done by simply using a drag and drop interface. Such development platforms however offer only limited functionality and customizability. (Marutitech blog)

A developer can choose to use a rule/retrieval based approach, here you simply write a pattern and a template. Such a bot replies with one of the templates when it encounters a similar pattern from the user. Rule based models tend to perform poorly when they encounter completely new sentences. (Suriyadeepan Ram 2016)

A different approach involves utilizing a generative model. Generative models construct responses word by word based on the query. Because of this the generated responses are prone to grammatical errors. However once they are trained, the generative model outperforms the rule based approach especially in handling previously unseen queries.

In probability and statistics a generative model is a model that generates all values for a phenomenon, both those that can be observed in the world and "target" variables that can only be computed from those observed.

For our example below, we will first preprocess the data and then train the generative model using sequence 2 sequence. Seq 2 Seq is a general purpose encoder decoder framework for Tensorflow which can be applied to machine translation, summarizing text ,image captioning and conversational modeling. The seq2seq network connects two RNN's to work together to transform one sequence to another. An encoder network condenses an input sequence into a vector while a decoder network unfolds that vector into a new sequence.

Before training a dataset on a generative model we need to perform a preprocessing step called padding. Here we work on converting the variable length sequences into fixed length sequences. Some of the datasets which can be used to train a chatbot using a generative model approach include: the Cornell Movie Dialog Corpus, which is a collection of dialogs from movie scripts and the Ubuntu Dialog Corpus which is based on chat logs from the Ubuntu IRC public channels.

The next step will be to implement the Continuous Bag of Words (CBOW) model, which is a form of word embedding. CBOW is a model of simplifying representation used in NLP. In this step text is represented as a multiset (Bag) of words disregarding grammer but retaining the frequency of words.

The final step in our model is implementing the Attention Mechanism. This step is very important since it prevents the information loss that could have occured while compressing all the necessary information of a source sentence into a single vector.

For source code, I recommend implementing Siraj Rawal's Seq 2 Seq Chatbot because of its clear documentation. Its available here https://github.com/llSourcell/tensorflow_chatbot


A useful intro to conversational Q&A systems like IBM's Jeopardy Watson should be available from the IBM DeepQA research team ("This is Watson"), much of which is available free: http://researcher.watson.ibm.com/researcher/view_group_pubs.php?grp=2099

In May/June 2012, the IBM Journal of R&D published a special issue entirely about the tech behind the IBM Watson Q&A software that played Jeopardy.


AFAIK, the articles are no longer freely available on-line, but similar work should form a useful basis for building the essential pieces in a conversational agent -- a knowledge base, queries & updates, understanding and generating text and speech, etc.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.