1
$\begingroup$

I was reading about the grounding problem after seeing it mentioned in another answer today. The article states that, in order to avoid the "infinite regress" of defining all words with other words, we must ground the meaning of some words in the "sensorimotor."

To be grounded, the symbol system would have to be augmented with nonsymbolic, sensorimotor capacities—the capacity to interact autonomously with that world of objects, events, actions, properties and states that its symbols are systematically interpretable (by us) as referring to.

Obviously, this made me think of Reinforcement Learning. But I'm not exactly sure what counts as "interaction." Would this necessarily imply an MDP-like formulation with rewards, state transitions, etc? Or could some form of grounding be accomplished with supervised learning?

This seems like a pretty fundamental problem of AI. Does anyone know of research being done on grounding words/symbols within an RL agent?

$\endgroup$
  • 1
    $\begingroup$ I think you need to reduce the number of questions at the end. In fact I would remove the "On a related note" paragraph, because it although it is related, answering it could double the length of any answer here. By stating the question, you are setting expectation that someone should attempt to answer it. If you want an answer to that, and it is not already covered by answers to your first question, you can always ask a second, separate question here. That would use the site better, IMO $\endgroup$ – Neil Slater Mar 17 at 10:08
  • $\begingroup$ Got it. Thanks for the feedback $\endgroup$ – Philip Raeisghasem Mar 17 at 10:11
  • $\begingroup$ Have a look at github.com/adityathakker/awesome-rl-nlp. $\endgroup$ – nbro Apr 14 at 15:21
-1
$\begingroup$

Language grounding in reinforcement learning can be realized with the PDDL syntax. A domain is described with motion primitives like grasp, stand-up, walkto and the primitives are used to abstract from lowlevel servo actions. For the untrained eye, the concept of language grounding seems to be a promising method to handle complex domains with reinforcement learning. The state-space gets smaller and the agent's behavior is transparent because the program is using the same words like a human will do.

But, it's possible to describe grounding as a dead end. The real benefit of a PDDL solver is not, that English verbs are used to describe the actions, but the dominant feature is that a PDDL file can predict future outcomes of a system. For example, the action “walkto” is executed, and the robot is in a new position on the map. It's possible to change the action name to a random string, which means that the domain is no longer grounded. But, the post-condition in the pddl file remains the same. And the solver is able to predict what will happen after the un-named function gets executed. So the advantage isn't the name of actions, but it's the prediction engine which knows what the result of an action is.

Model based language processing A typical example for grounded language is a textadventure. The user gets feedback from the system only with textual words and he has to enter commands in English. As default, a textadventure provides a forward model. After entering a command sequence the game will provide the future state. The only problem is, that for most domains no textadventure is available but only a textcorpus. It make sense to convert an existing textcorpus into a game, which means to take the textual sequence as input and generate a playable forward simulation as output. In such a simulation it's possible to take trial actions and answer what-if problems. The system uses the database to retrieve sequence of steps to reach a predefined goal.

$\endgroup$
  • $\begingroup$ From what I read, this doesn't seem to be related to NLP or grounding. $\endgroup$ – Philip Raeisghasem Mar 17 at 16:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.