IPS estimator, which is used for off-policy evaluation in a contextual bandit problem, is well explained here: Doubly Robust Policy Evaluation andOptimization https://arxiv.org/pdf/1503.02834.pdf

The old policy $\mu$, or the behavior policy, is okay to be non-stationary in the IPS estimator even if the new policy $\nu$, or the target policy, should be stationary.

I wonder if that is true for IS estimator, which seems to be a variant of IPS, for off-policy evaluation in a reinforcement learning problem. IS estimator is explained here: Doubly Robust Off-policy Value Evaluation for Reinforcement Learning https://arxiv.org/abs/1511.03722. The target policy should be stationary, but can the old policy be non-stationary in the IS estimator?