1
$\begingroup$

In the maximum entropy inverse reinforcement learning paper, Ziebart et al. show that the state visitation frequency $\rho(s)$ of a state $s$ can be computed as $$ \rho_{\pi}(s) = \sum_{t}^{T} P(s_t=s|\pi), $$ which is the sum of the probability that the state being visited at each time step.

I just don't understand why is it the sum? From my perspective, a frequency should be the less than one, so that it should be the average value $$ \rho_{\pi}(s) = \frac{1}{T}\sum_{t}^{T} P(s_t=s|\pi). $$

$\endgroup$
1
  • $\begingroup$ My feeling is that they define the first equation and will then normalise it to make it a state distribution. $\endgroup$ Apr 26 at 8:34
0
$\begingroup$

It should be the average and this is rarely mentioned by people except for a IRL summer camp at UCB. You can check this GithubIssue for details.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.