# What is the difference between a stationary and a non-stationary policy?

In reinforcement learning, there are deterministic and non-deterministic (or stochastic) policies, but there are also stationary and non-stationary policies.

What is the difference between a stationary and a non-stationary policy? How do you formalize both? Which problems (or environments) require a stationary policy as opposed to a non-stationary one (and vice-versa)?

A stationary policy, $$\pi_t$$, is a policy that does not change over time, that is, $$\pi_t = \pi, \forall t \geq 0$$, where $$\pi$$ can either be a function, $$\pi: S \rightarrow A$$ (a deterministic policy), or a conditional density, $$\pi(A \mid S)$$ (a stochastic policy). A non-stationary policy is a policy that is not stationary. More precisely, $$\pi_i$$ may not be equal to $$\pi_j$$, for $$i \neq j \geq 0$$, where $$i$$ and $$j$$ are thus two different time steps.