Skip to main content
Artificial Intelligence

Return to Question

made the title more descriptive of what the question actually seemed to be
Source Link
nbro
nbro
  • 41.4k
  • 12
  • 114
  • 205

How Which approaches could I use to traincreate a simple chatbot using a neural network?

I wanted to startedstart experimenting with neural network and as a toy problemnetworks, so I wished to train onedecided to chat, i.e. implementmake a chatting bot like cleverbot. Notchatbot (like Cleverbot, which is not that clever anyway) using them.

I looked around for some documentation and I found many tutorialtutorials on general tasks, but few on this specific topic. The one I found just exposed the results without giving insights on the implementation. The ones that did, did it pretty shallowyshallowly (the tensorflowTensorFlow documentation page on seq2seq is lacking imho, IMHO).

Now, I feel I may have understood the principle more or less, but I'm not sure and I am not even sure how to start. Thus I will explain how I would tackle the problem and I'd like a feedback on this solution, telling me where I'm mistaken, and possibly have any link to detailed explainationsexplanations and practical knowledge on the process.

  1. The dataset I will use for the task is the dump of all my facebookFacebook and whatsappWhatsApp chat history. I don't know how big it will be but possibly still not large enough. The target language is not englishEnglish, therefore I don't know where to quickly gather meaningful conversation samples.

  2. I am going to generate a thought vector out of each sentence. Still don't know how, actually; I found a nice example for word2vec on the deeplearning4j website, but none for sentences. I understood how word vectors are built and why, but I could not find an exhaustive explainationexplanation for sentence vectors.

  3. Using thought vectors as input and output I am going to train the neural network. I don't know how many layers it should have, and which ones have to be lstmLSTM layers.

  4. Then there should be another neural network that is able to transform a thought vector into a sequence of charactercharacters composing a sentence. I read that I should use padding to make up for different sentence lengths, but I miss how to encode characters (are codepoints enough?).

How to train a chatbot

I wanted to started experimenting with neural network and as a toy problem I wished to train one to chat, i.e. implement a chatting bot like cleverbot. Not that clever anyway.

I looked around for some documentation and I found many tutorial on general tasks, but few on this specific topic. The one I found just exposed the results without giving insights on the implementation. The ones that did, did it pretty shallowy (the tensorflow documentation page on seq2seq is lacking imho).

Now, I feel I may have understood the principle more or less but I'm not sure and I am not even sure how to start. Thus I will explain how I would tackle the problem and I'd like a feedback on this solution, telling me where I'm mistaken and possibly have any link to detailed explainations and practical knowledge on the process.

  1. The dataset I will use for the task is the dump of all my facebook and whatsapp chat history. I don't know how big it will be but possibly still not large enough. The target language is not english, therefore I don't know where to quickly gather meaningful conversation samples.

  2. I am going to generate a thought vector out of each sentence. Still don't know how actually; I found a nice example for word2vec on deeplearning4j website, but none for sentences. I understood how word vectors are built and why, but I could not find an exhaustive explaination for sentence vectors.

  3. Using thought vectors as input and output I am going to train the neural network. I don't know how many layers it should have, and which ones have to be lstm layers.

  4. Then there should be another neural network that is able to transform a thought vector into a sequence of character composing a sentence. I read that I should use padding to make up for different sentence lengths, but I miss how to encode characters (are codepoints enough?).

Which approaches could I use to create a simple chatbot using a neural network?

I wanted to start experimenting with neural networks, so I decided to make a chatbot (like Cleverbot, which is not that clever anyway) using them.

I looked around for some documentation and I found many tutorials on general tasks, but few on this specific topic. The one I found just exposed the results without giving insights on the implementation. The ones that did, did it pretty shallowly (the TensorFlow documentation page on seq2seq is lacking, IMHO).

Now, I feel I may have understood the principle more or less, but I'm not sure and I am not even sure how to start. Thus I will explain how I would tackle the problem and I'd like feedback on this solution, telling me where I'm mistaken, and possibly have any link to detailed explanations and practical knowledge on the process.

  1. The dataset I will use for the task is the dump of all my Facebook and WhatsApp chat history. I don't know how big it will be but possibly still not large enough. The target language is not English, therefore I don't know where to quickly gather meaningful conversation samples.

  2. I am going to generate a thought vector out of each sentence. Still don't know how, actually; I found a nice example for word2vec on the deeplearning4j website, but none for sentences. I understood how word vectors are built and why, but I could not find an exhaustive explanation for sentence vectors.

  3. Using thought vectors as input and output I am going to train the neural network. I don't know how many layers it should have, and which ones have to be LSTM layers.

  4. Then there should be another neural network that is able to transform a thought vector into a sequence of characters composing a sentence. I read that I should use padding to make up for different sentence lengths, but I miss how to encode characters (are codepoints enough?).

Source Link
Totem
Totem
  • 381
  • 2
  • 6

How to train a chatbot

I wanted to started experimenting with neural network and as a toy problem I wished to train one to chat, i.e. implement a chatting bot like cleverbot. Not that clever anyway.

I looked around for some documentation and I found many tutorial on general tasks, but few on this specific topic. The one I found just exposed the results without giving insights on the implementation. The ones that did, did it pretty shallowy (the tensorflow documentation page on seq2seq is lacking imho).

Now, I feel I may have understood the principle more or less but I'm not sure and I am not even sure how to start. Thus I will explain how I would tackle the problem and I'd like a feedback on this solution, telling me where I'm mistaken and possibly have any link to detailed explainations and practical knowledge on the process.

  1. The dataset I will use for the task is the dump of all my facebook and whatsapp chat history. I don't know how big it will be but possibly still not large enough. The target language is not english, therefore I don't know where to quickly gather meaningful conversation samples.

  2. I am going to generate a thought vector out of each sentence. Still don't know how actually; I found a nice example for word2vec on deeplearning4j website, but none for sentences. I understood how word vectors are built and why, but I could not find an exhaustive explaination for sentence vectors.

  3. Using thought vectors as input and output I am going to train the neural network. I don't know how many layers it should have, and which ones have to be lstm layers.

  4. Then there should be another neural network that is able to transform a thought vector into a sequence of character composing a sentence. I read that I should use padding to make up for different sentence lengths, but I miss how to encode characters (are codepoints enough?).