In the apprenticeship learning algorithm described by Ng et al. in Apprenticeship Learning via Inverse Reinforcement Learning, they mention that expert trajectories come in the form of $\{s_0^i, s_1^i\, ...\}_{i=1}^m$. However, they also mentioned that $s_0 $ is drawn from distribution D. Do all expert trajectories then have to have the same starting state? Why is it not possible to compute the feature expectation based on a single trajectory?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
All right, I figured it out. trajectories need not have the same starting state because the distribution of $s_0$ is drawn from a distribution D (mentioned in the paper). Had been confused because many of the code implementations on github focus on trajectories starting from the same state.
Hope this helps everyone !
-
$\begingroup$ Maybe you should say where/how you discovered that $s_0$ is drawn from $D$. Does the paper you mention in your question state this? $\endgroup$– nbroApr 3, 2020 at 13:40
-
1$\begingroup$ Yes the paper mentioned above states this. The confusion came when I was reading a blog post about IRL. I might have misunderstood when the blog said that "The Starting position(state) of the bot is fixed, as according to the IRL algorithm it is necessary that the starting state is same for all the iterations.", from jangirrishabh.github.io/2016/07/09/virtual-car-IRL leading to some doubts that I had $\endgroup$– calveeenApr 3, 2020 at 13:54