Before a robot can act with meaning, some planning is needed. The idea is, that the decision making process is independent from action. The task of figuring out what the best decision is, was introduced under the term Cybernetics and is focused on intelligence in animals and artificial systems. In a robot, the act of thinking is realized with computer programs.
If we are researching how exactly a computer program determines what the next action for a robot is, the term reinforcement learning is often referenced. The idea is, to store all situations in a q-table and if the robot is in a certain situation he can look into the q-matrix and sees in the row what action is the right one. This is called a policy because it answers the question what action is needed in a certain state.
Some researchers have recognized that Q-learning is not the best way for dealing with complex problems. A policy which is similar to a lookup table will provide only simple formed answers to difficult challenges. The more elaborated form of storing knowledge is called an indirect policy. The idea is here not to store state-action values in a table but use a prediction model. The first thing what the robot software is answering are potential actions and for each action an outcome is calculated. The predicted result of the actions is stored in graph and the best node is identified with a solver.
In a concrete situation a direct policy knows what the correct action is, but the indirect policy don't. The indirect policy has to figure out first what the prediction of future states will be. My question is: is an indirect policy which is using a prediction model superior to a direct policy known as a q-table?