Given a robot in a situation such as in a library reading a book.

Now I want to create a neural network that suggests an appropriate action in this situation. And, generally, ignore actions such as "get up and dance" and so on.

Since there are limitless actions a robot could do, I need to narrow it down to the ones in this situation. Using its vision system, the word "book" and book neurons should already be activated as well as "reading".

One idea I had was to create an adversarial network which generates words (sequences of letters) based on the situation such as "turn page", "read next line" and so on. And then have another neural network which translates these words into actions. (It would them simulate whether this was a good idea. If not it would somehow suppress the first word and try to generate a new word.)

Another example is the robot is in a maze and gets to a crossroads. The network would generate the word "turn left" and "turn right".

Another idea would be to have the actions be composed of a body part e.g. "eyes" and a movement such as "move left" and it would combine these to suggest actions.

Either way, it seems like I need a way to encode actions so that the robot doesn't consider every possible action in the universe.

Is there any research in this area or ideas on how to achieve this?

(I think this may be somewhat related to the task of "try to name as many animals as you can.")


One idea I had was to create an adversarial network which generates words

It may be better to use for e.g. numbers instead of words, then map these numbers to the language of your choice like English in your case. This mapping can be changed to other languages and you will have a robot that overcomes language barriers.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.