I've been reading the attached paper - which aims to model entities in the world as objects, including the learning agent itself!

To say the least, the goal is to navigate through what seems like a maze (path-planning problem) - and drop off passengers in desired destinations, while avoiding walls in the map of the world (5x5 grid for now). The objects involved are, a taxi, passengers, walls and a destination.

Now, a particular paragraph says the following:

"Whereas in the classical MDP model, the effect of encountering walls is felt as a property of specific locations in the grid, the OO-MDP view is that wall interactions are the same regardless of their location. As such, agents’ experience can transfer gracefully throughout the state space."

What does this mean? How are the classical MDP and the object-oriented MDP views different?

I can't make sense of the above extract, at all. Any help would be appreciated!

P.S. I did not consider posting parts of the extract as separate questions since my problem has more to do with understanding the extract as a whole which inevitably relies on understanding the parts.

  • $\begingroup$ Maybe it doesn't define the state in terms of the location but instead the local properties. Thus the mdp only considers the agents past local properties. This kind of makes sense. $\endgroup$ – FourierFlux Apr 28 '20 at 19:08
  • $\begingroup$ it should be also noted this approach will always result in a finite horizon solution, you can't do global planning with it since it's always operating locally, $\endgroup$ – FourierFlux Apr 28 '20 at 20:40
  • $\begingroup$ Could you elaborate a bit on what you mentioned in the comments, @FourierFlux? Also, what is meant by "local properties"? How does it help in making sense of the line - "wall interactions are same regardless of location" and "agents' experience can transfer gracefully throughout the state space"? $\endgroup$ – epsilon-emperor Apr 29 '20 at 3:31
  • $\begingroup$ The state space im assuming only consists of what the agent sees and directly interacts with. $\endgroup$ – FourierFlux Apr 29 '20 at 5:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.