Training an RNN to answer simple quesitons

Question

I would like to train an RNN to follow the sentences:

"Would you like some cheese"? with "Yes, I would like some cheese."

So whenever the template "Would you like some ____?" appears then RNN produces the sequence above. And it should even work on sentences which are new like "Would you like some blumf?"

I have thought of various ways of doing this. Such as, as well as having 26 outputs for letters of the alphabet have about 20 more for "repeat the character that is 14 characters to the left" and so on.

Has this been done before or is there a better way?

Answer 1

I have thought of various ways of doing this. Such as, as well as having 26 outputs for letters of the alphabet have about 20 more for "repeat the character that is 14 characters to the left" and so on

Creating a system of rules like your example above is the very opposite of training an RNN to perform this task.

If you would like to train an RNN to answer simple questions you would not need to come up with ingenious rules, but with enough training data of the form

question -> answer

Then you could use one of many popular sequence-to-sequence NLP tools to try and learn this behaviour, effectively treating the problem as if it were machine translation between languages.

---

More broadly, yes, question answering has been done before, it is very popular in fact. It is an active subfield of NLP research and many methods have been developed, some of them involving RNN networks.