The definition of non-stationarity in multi-agent learning is indeed correct.

The 'moving target problem' you are referring to is investigated here: [Transient Non-stationarity and Generalisation in Deep Reinforcement Learning][1]. Both of them together have been discussed in section 5.10.1 in this a book: [Multi-Agent Reinforcement Learning: Foundations and Modern Approaches][2]


  [1]: https://openreview.net/forum?id=Qun8fv4qSby
  [2]: https://www.marl-book.com/