I was wondering how can we deal with movement inertia in an environment that is constantly changing?
Imagine that you make a step on an environment that moves a ball. When you make the step, you make the ball move and at one time, it returns an observation and a reward. Then you give the observation and the reward to your RL algorithm and you find the next best policy.
While calculating the best policy, the ball is still moving in the environment because it has an inertia made by the action we took previously. How do we deal with that? The policy that we just calculated is already based on an old value right?
If the RL algorithm and the communication between the RL and the environment is fast I don't think it is much of an issue, but what if your algorithm is very slow ?