# Why is it that the state visitation frequency equals the sum of state visitation frequency from initial time step to the horizon?

In the maximum entropy inverse reinforcement learning paper, Ziebart et al. show that the state visitation frequency $$\rho(s)$$ of a state $$s$$ can be computed as $$\rho_{\pi}(s) = \sum_{t}^{T} P(s_t=s|\pi),$$ which is the sum of the probability that the state being visited at each time step.

I just don't understand why is it the sum? From my perspective, a frequency should be the less than one, so that it should be the average value $$\rho_{\pi}(s) = \frac{1}{T}\sum_{t}^{T} P(s_t=s|\pi).$$

• My feeling is that they define the first equation and will then normalise it to make it a state distribution. Apr 26, 2021 at 8:34