I've read that for MDPs the state transition function $P_a(s, s')$ is a probability. This seems strange to me for modeling because most environments (like video games) are deterministic.
Now, I'd like to assert that most systems we work with are deterministic given enough information in the state (i.e. in a video game, if you had the random number seed, you could predict 'rolls', and then everything else follows game logic).
So, my guess for why would MDP state transitions are probabilities is because the state given to the MDP is typically a subset (i.e. from feature engineering) of total information available. That, and of course to model non-deterministic systems.
Is my understanding correct?