I wanted to start experimenting with neural networks, so I decided to make a chatbot (like Cleverbot, which is not that clever anyway) using them.
I looked around for some documentation and I found many tutorials on general tasks, but few on this specific topic. The one I found just exposed the results without giving insights on the implementation. The ones that did, did it pretty shallowly (the TensorFlow documentation page on seq2seq is lacking, IMHO).
Now, I feel I may have understood the principle more or less, but I'm not sure and I am not even sure how to start. Thus I will explain how I would tackle the problem and I'd like feedback on this solution, telling me where I'm mistaken, and possibly have any link to detailed explanations and practical knowledge on the process.
The dataset I will use for the task is the dump of all my Facebook and WhatsApp chat history. I don't know how big it will be but possibly still not large enough. The target language is not English, therefore I don't know where to quickly gather meaningful conversation samples.
I am going to generate a thought vector out of each sentence. Still don't know how, actually; I found a nice example for word2vec on the deeplearning4j website, but none for sentences. I understood how word vectors are built and why, but I could not find an exhaustive explanation for sentence vectors.
Using thought vectors as input and output I am going to train the neural network. I don't know how many layers it should have, and which ones have to be LSTM layers.
Then there should be another neural network that is able to transform a thought vector into a sequence of characters composing a sentence. I read that I should use padding to make up for different sentence lengths, but I miss how to encode characters (are codepoints enough?).