1
$\begingroup$

In the maximum entropy inverse reinforcement learning paper, Ziebart et al. show that the state visitation frequency $\rho(s)$ of a state $s$ can be computed as $$ \rho_{\pi}(s) = \sum_{t}^{T} P(s_t=s|\pi), $$ which is the sum of the probability that the state being visited at each time step.

I just don't understand why is it the sum? From my perspective, a frequency should be the less than one, so that it should be the average value $$ \rho_{\pi}(s) = \frac{1}{T}\sum_{t}^{T} P(s_t=s|\pi). $$

$\endgroup$
1
  • $\begingroup$ My feeling is that they define the first equation and will then normalise it to make it a state distribution. $\endgroup$ Apr 26, 2021 at 8:34

1 Answer 1

0
$\begingroup$

It should be the average and this is rarely mentioned by people except for a IRL summer camp at UCB. You can check this GithubIssue for details.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .