Why do we have to solve MDP in each iteration of Maximum Entropy Inverse Reinforcement Learning?

Gradient in Maximum Entropy IRL requires to find the probability of expert trajectories given the reward function weights. This is done in the paper by calculating state visitation probabilities but I do not understand why we can’t just calculate the probability of a trajectory by summing up all the rewards that are collected following that trajectory? The paper defines the probability of a trajectory as exp(R(traj.)/Z. I do not understand why we have to solve MDP for calculating that.