I don't know anything about neural networks, but I got the information that making a chatbot with a neural network is very good. Is this really true?

What do I need to know in order to build a chat bot using neural networks?

I have understood a little about text preprocessing (such as tokenization, stemming, stop word lists, named entity recognition), but when I want to try classification using the neural network I am confused.

  • $\begingroup$ There is too much to learn to write in an answer. Instead I suggest you take a course on neural networks that includes some language processing. This one springs to mind (although it doesn't do chatbots): coursera.org/learn/nlp-sequence-models and also sentdex builds a chatbot in Python in this series of tutorials: youtube.com/… However, you will probably want to start with something more basic otherwise you may not understand what it is you are making $\endgroup$ – Neil Slater Mar 16 '19 at 10:21
  • $\begingroup$ I believe neural networks are not a good approach to creating chatbots; conversational structure is far to complex to be learned by a NN. The best results at present (production standard) are not using NNs. It is much better to use a rule-based approach and pattern matching. I suggest to have a look at the ELIZA system, which provides a good starting point, despite being very old. $\endgroup$ – Oliver Mason Mar 16 '19 at 14:15
  • $\begingroup$ @OliverMason: My understanding is that modern chatbots in production environments for business use are using NN language models for POS tagging, or for predicting intent from user input, but not for modelling the conversation. Whilst chatbots have been built using seq2seq NN models, these are more for research and entertainment. OP doesn't say whether they want an entertaining chatbot or something practical to help run some process (like a support channel) $\endgroup$ – Neil Slater Mar 16 '19 at 17:59
  • $\begingroup$ @NeilSlater heyy can you give input on what should I learn? just the point like i must learn sequence-models or something $\endgroup$ – Faris Dewantoro Mar 17 '19 at 4:31
  • $\begingroup$ @NeilSlater is NN always a hidden layer for classification ? and what is sequence models $\endgroup$ – Faris Dewantoro Mar 17 '19 at 4:32

The basic idea behind neural networks based chatbots is to search in a textcorpus for a replay from the past and put this to the screen. The starting point is usually a textfile which contains 100 MB of chat protocols from the past. It has a structure like “user1 says sentence1, user2 says sentence2, user1 says sentence3” and so on. If the user interacts with the chatbot, he enters a sentence, for example “Hi, what is your name?”. The chatbot takes this input, generates a sql-query to the 100 MB textfile and searches for a similar sentence in the database. The correct replay is given by the line stored below the original sentence. That means the chatbot will imitate an existing dialogue.

In reality, some detail problems have to be answered. For example what to do if an input was not in the database. Or how to modify the output slightly, so that it will sound more natural. This is treated with different techniques like nlp-parsers, dialogue models and neural networks. All these methods support the query on an existing textfile. They interpolate between existing information with the aim to improve the language retrieval.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.