4
$\begingroup$

I would like to train an RNN to follow the sentences:

"Would you like some cheese"? with "Yes, I would like some cheese."

So whenever the template "Would you like some ____?" appears then RNN produces the sequence above. And it should even work on sentences which are new like "Would you like some blumf?"

I have thought of various ways of doing this. Such as, as well as having 26 outputs for letters of the alphabet have about 20 more for "repeat the character that is 14 characters to the left" and so on.

Has this been done before or is there a better way?

Improve this question
$\endgroup$
3
  • $\begingroup$ One of the important things to realize about the modern approach to AI/ML is that you do not provide it rules, you use the training data to get the network to work out rules for itself. $\endgroup$ – David Hoelzer Jan 28 '20 at 23:16
  • $\begingroup$ Except an RNN can't learn the rule "repeat last character." It can only learn 26 separate rules for each character. Which is kind of useless to learn template type things. $\endgroup$ – zooby Jan 29 '20 at 0:46
  • $\begingroup$ Why limit yourself to letters? You could certainly train it to look for and predict word sequences. $\endgroup$ – David Hoelzer Jan 29 '20 at 0:57
1
$\begingroup$

I have thought of various ways of doing this. Such as, as well as having 26 outputs for letters of the alphabet have about 20 more for "repeat the character that is 14 characters to the left" and so on

Creating a system of rules like your example above is the very opposite of training an RNN to perform this task.

If you would like to train an RNN to answer simple questions you would not need to come up with ingenious rules, but with enough training data of the form

question -> answer

Then you could use one of many popular sequence-to-sequence NLP tools to try and learn this behaviour, effectively treating the problem as if it were machine translation between languages.

More broadly, yes, question answering has been done before, it is very popular in fact. It is an active subfield of NLP research and many methods have been developed, some of them involving RNN networks.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.