Given a robot in a situation such as in a library reading a book.
Now I want to create a neural network that suggests an appropriate action in this situation. And, generally, ignore actions such as "get up and dance" and so on.
Since there are limitless actions a robot could do, I need to narrow it down to the ones in this situation. Using its vision system, the word "book" and book neurons should already be activated as well as "reading".
One idea I had was to create an adversarial network which generates words (sequences of letters) based on the situation such as "turn page", "read next line" and so on. And then have another neural network which translates these words into actions. (It would them simulate whether this was a good idea. If not it would somehow suppress the first word and try to generate a new word.)
Another example is the robot is in a maze and gets to a crossroads. The network would generate the word "turn left" and "turn right".
Another idea would be to have the actions be composed of a body part e.g. "eyes" and a movement such as "move left" and it would combine these to suggest actions.
Either way, it seems like I need a way to encode actions so that the robot doesn't consider every possible action in the universe.
Is there any research in this area or ideas on how to achieve this?
(I think this may be somewhat related to the task of "try to name as many animals as you can.")
