2
$\begingroup$

If a policy maps states to actions in reinforcement learning, then for a path planning with obstacles, can't we simply use Artificial Potential Field fields for path planning and model policy mathematically as a field where the obstacles form repulsive field and goal form attractive field?

So, technically, is a policy simply a field?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Since most policies depend solely on actions and states/observations, then if you model the space of this field as the Cartesian Product of your state and action spaces, then the policy learns a surface over this combined space, similar to the way a field is parameterized.

The policy an agent learns could exhibit the same behavior as the field you describe above (obstacles form an repulsive field, and goal(s) form an attractive field). However, unlike the field described above, it is not guaranteed that the learned policy will capture this behavior - the policy learned depends on:

  1. The learning algorithm used
  2. The approximators (e.g. neural networks) used for learning, and their respective hyperparameters
  3. The formulation of the reward function
  4. The number of episodes/total steps the policy/agent is trained over.

To sum this answer up, I believe you could train the policy in such a way (using the mechanisms above) such that it resembles the field you describe.

Improve this answer
$\endgroup$
4
  • $\begingroup$ The field has much closer analogy with a value function than with a policy function. The policy is analagous to a solution found by using the field. $\endgroup$
    – Neil Slater
    Jul 29, 2020 at 9:18
  • $\begingroup$ So isnt technically a policy a field surface, with the exception of the fact that field is more or less deterministic while policy is field + probability assigned to it every action in the field? $\endgroup$
    – gfdsal
    Jul 29, 2020 at 14:10
  • $\begingroup$ @NeilSlater, if we merely assign probability to the field at each state, will then it be close analogy to the policy as policies are probabilistic transition from states to actions? $\endgroup$
    – gfdsal
    Jul 29, 2020 at 14:12
  • 1
    $\begingroup$ @gfdsal: A potential field cannot directly represent probabilities. You might derive them from it, according to the local gradients of the field. Again that is a closer analogy to a value function used as a heuristic, not a policy function. A policy can be derived from a state value function in a similar way as a plan can be derived from an APF. $\endgroup$
    – Neil Slater
    Jul 29, 2020 at 14:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .