I want to develop a system to generate grammatically correct sentences. The input would be some words. The output would be a grammatically correct human-like sentence.


Input: capital, Paris, France

output : Paris is the capital of France

Input: cute, cat

output: cats are cute

The system adds the missing words such as 'is', 'as', 'are', 'the', 'of', etc.

How can I build a system like this? My gut feeling is it can be done through Reinforcement Learning by training on a huge corpus like Wikipedia.

So the states being the individual input words. The reward would be 1 when the sentence is correct and 0 when not. The actions available are taking an individual word available from the input and attaching it to the connecting word (is,of,the..). Then in the second step take the generated word and pick another word from input and connect it and so on. Stop when all the input words have been used. Its win when the final sentence is grammatically correct,else fail.

Ultimately, what I imagine is there will be a knowledge graph. The user asks some question. Navigating through the knowledge graph, the system will generate some keywords. Then the RL system will take those keywords and construct a human-like sentence.

I'm totally new to RL. I just finished watching David Silver's 10 part course on RL in YouTube. Any guidance on this topic is much appreciated.

  • $\begingroup$ DeepCoder (openreview.net/pdf?id=ByldLrqlx) uses RL to generate computer code. Its different from natural language, I agree, but it should give you a headstart. $\endgroup$ – Souradeep Nanda Apr 25 '17 at 14:04
  • 1
    $\begingroup$ @SouradeepNanda: Where does DeepCoder user RL? I can see it uses a deep NN and SMT solver . . . but nothing about any RL technique. The term q(a | E), appears, but I don't think that is an action value? $\endgroup$ – Neil Slater Jul 30 '17 at 16:19
  • $\begingroup$ Sorry, I confused with this one. My bad. arxiv.org/abs/1706.05125 $\endgroup$ – Souradeep Nanda Aug 1 '17 at 15:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.