If a policy maps states to actions in reinforcement learning, then for a path planning with obstacles, can't we simply use Artificial Potential Field fields for path planning and model policy mathematically as a field where the obstacles form repulsive field and goal form attractive field?

So, technically, is a policy simply a field?


Since most policies depend solely on actions and states/observations, then if you model the space of this field as the Cartesian Product of your state and action spaces, then the policy learns a surface over this combined space, similar to the way a field is parameterized.

The policy an agent learns could exhibit the same behavior as the field you describe above (obstacles form an repulsive field, and goal(s) form an attractive field). However, unlike the field described above, it is not guaranteed that the learned policy will capture this behavior - the policy learned depends on:

  1. The learning algorithm used
  2. The approximators (e.g. neural networks) used for learning, and their respective hyperparameters
  3. The formulation of the reward function
  4. The number of episodes/total steps the policy/agent is trained over.

To sum this answer up, I believe you could train the policy in such a way (using the mechanisms above) such that it resembles the field you describe.

  • $\begingroup$ The field has much closer analogy with a value function than with a policy function. The policy is analagous to a solution found by using the field. $\endgroup$ Jul 29 '20 at 9:18
  • $\begingroup$ So isnt technically a policy a field surface, with the exception of the fact that field is more or less deterministic while policy is field + probability assigned to it every action in the field? $\endgroup$
    – gfdsal
    Jul 29 '20 at 14:10
  • $\begingroup$ @NeilSlater, if we merely assign probability to the field at each state, will then it be close analogy to the policy as policies are probabilistic transition from states to actions? $\endgroup$
    – gfdsal
    Jul 29 '20 at 14:12
  • 1
    $\begingroup$ @gfdsal: A potential field cannot directly represent probabilities. You might derive them from it, according to the local gradients of the field. Again that is a closer analogy to a value function used as a heuristic, not a policy function. A policy can be derived from a state value function in a similar way as a plan can be derived from an APF. $\endgroup$ Jul 29 '20 at 14:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.