I have created a chatbot by Keras based on movie dialog. I used RNN more specifically GRU . My bot can reply well. But the problem is , it can't hold the context . As an example if I say Tell me a joke, the bot will reply something , and then if I say one more , the bot simply doesn't understand that I was asking for another joke and many more similar cases, like if I used a slang against the bot , the bot will reply me with something similar , but if I just say something romantic or good immediately after using slang , the bot will reply to me with something good . I want to keep context or environment . How can I do so . Any lead would be helpful .
$\begingroup$
$\endgroup$
4
-
$\begingroup$ Could you add a little more context about your current model? I would guess it is a seq2seq model that "translates" a phrase input by user into a reply phrase. The behaviour you notice is definitely a limitation of using seq2seq like that. $\endgroup$– Neil SlaterCommented Jul 25, 2019 at 14:49
-
$\begingroup$ Yes u are right , I am using seq2seq . Is there any alternative which can solve my problem ? $\endgroup$– Mithun Sarker ShuvroCommented Jul 25, 2019 at 16:57
-
$\begingroup$ There's lots of ways to make chatbots. You could ask about comparison between a couple of methods (but please don't ask for some full guide or review, it's too much for a question here). Or your current question is OK in my opinion, if you use edit to add some details from your comment, so that people have some clue to what you've got compared to where you want to go next. I'm not a chatbot expert, but I think you are in the right place to ask such a question. I just think it would help you get the best answer if you say a little more about your current model. $\endgroup$– Neil SlaterCommented Jul 25, 2019 at 20:49
-
$\begingroup$ It may also help if you explain your longer-term goal. Is your chatbot intended for some specific use? Or is the main purpose in building it to teach yourself topics in Natural Language Processing and AI? If the former, you may want to fall back on more robust simple methods than neural networks. If the latter, it may not be too important to you if the bot has flaws provided it exhibits some interesting behaviour such as assigning pronouns correctly between different exchanges at least some of the time $\endgroup$– Neil SlaterCommented Jul 25, 2019 at 20:56
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
This is an idea that I used for my model - Try using two RNN (GRU) Networks, one of them to manage current output state and the other to maintain context
Say we are at timestamp $t$ and the two GRUCells are represented as $GRU_c$ and $GRU_s$ for GRU context network and state network. (Your output coming from the state network)
At time stamp $t$ , the input $GRU_s(t) = concat(input, att(all~GRU_c~from ~[0, ~t-1]))$ where $att$ is an attention mechanism to give importance to specific parts of the conversation uptil that point (This is what maintains context) and input $GRU_c(t) = learned~representation~of~GRU_s(t)$ , hence updating $GRU_c$ for that timestamp, which along with the historical information can be used for $GRU_s(t+1)$
Hope this helped!
