# How is the state-visitation frequency computed in "Maximum Entropy Inverse Reinforcement Learning"?

I am trying to understand the formulation of the maximum entropy Inverse RL method by Brian Ziebart. Particularly, I am stuck on how to understand the computation of state - visitation frequencies.

In order to do so, they utilize a dynamic programming approach to compute the visitation frequency, in which the next state frequency is calculated based upon the state visitation frequency at the previous time step.

This is the algorithm below where, $$D_{s_i,t}$$ is the probability of state $$s_i$$ being visited at time step $$t$$. What is the difference between this way of computing state visitation frequency compared to the naive method of summing the total number of times state $$s_i$$ appears in the trajectory divided by the trajectory length?

If $$P(a_{i,j}|s_i)$$ is equal to the policy that is used for generating the demonstrated trajectories, then it could be the same. However, in inverse RL you don't know $$P(a_{i,j}|s_i)$$ and you iteratively approximate it.
Furthermore, your technique would work if the MDP is deterministic or you would have an infinite amount of trajectories (central limit theorem). Otherwise, your summation neglects the probability of $$s'$$ given $$s,a$$ pairs, which is why you have $$P(s_k|a_{i,j},s_i)$$ in the equation.