3
$\begingroup$

In the apprenticeship learning algorithm described by Ng et al. in Apprenticeship Learning via Inverse Reinforcement Learning, they mention that expert trajectories come in the form of $\{s_0^i, s_1^i\, ...\}_{i=1}^m$. However, they also mentioned that $s_0 $ is drawn from distribution D. Do all expert trajectories then have to have the same starting state? Why is it not possible to compute the feature expectation based on a single trajectory?

$\endgroup$
3
$\begingroup$

All right, I figured it out. trajectories need not have the same starting state because the distribution of $s_0$ is drawn from a distribution D (mentioned in the paper). Had been confused because many of the code implementations on github focus on trajectories starting from the same state.

Hope this helps everyone !

| improve this answer | |
$\endgroup$
  • $\begingroup$ Maybe you should say where/how you discovered that $s_0$ is drawn from $D$. Does the paper you mention in your question state this? $\endgroup$ – nbro Apr 3 at 13:40
  • 1
    $\begingroup$ Yes the paper mentioned above states this. The confusion came when I was reading a blog post about IRL. I might have misunderstood when the blog said that "The Starting position(state) of the bot is fixed, as according to the IRL algorithm it is necessary that the starting state is same for all the iterations.", from jangirrishabh.github.io/2016/07/09/virtual-car-IRL leading to some doubts that I had $\endgroup$ – calveeen Apr 3 at 13:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.