I wanted to started experimenting with neural network and as a toy problem I wished to train one to chat, i.e. implement a chatting bot like cleverbot. Not that clever anyway.
I looked around for some documentation and I found many tutorial on general tasks, but few on this specific topic. The one I found just exposed the results without giving insights on the implementation. The ones that did, did it pretty shallowy (the tensorflow documentation page on seq2seq is lacking imho).
Now, I feel I may have understood the principle more or less but I'm not sure and I am not even sure how to start. Thus I will explain how I would tackle the problem and I'd like a feedback on this solution, telling me where I'm mistaken and possibly have any link to detailed explainations and practical knowledge on the process.
The dataset I will use for the task is the dump of all my facebook and whatsapp chat history. I don't know how big it will be but possibly still not large enough. The target language is not english, therefore I don't know where to quickly gather meaningful conversation samples.
I am going to generate a thought vector out of each sentence. Still don't know how actually; I found a nice example for word2vec on deeplearning4j website, but none for sentences. I understood how word vectors are built and why, but I could not find an exhaustive explaination for sentence vectors.
Using thought vectors as input and output I am going to train the neural network. I don't know how many layers it should have, and which ones have to be lstm layers.
Then there should be another neural network that is able to transform a thought vector into a sequence of character composing a sentence. I read that I should use padding to make up for different sentence lengths, but I miss how to encode characters (are codepoints enough?).