The definition of non-stationarity in multi-agent learning is indeed correct.

The 'moving target problem' you are referring to is investigated here: Transient Non-stationarity and Generalisation in Deep Reinforcement Learning. Both of them together have been discussed in section 5.10.1 in this a book: Multi-Agent Reinforcement Learning: Foundations and Modern Approaches

The definition of non-stationarity in multi-agent learning is indeed correct.

The 'moving target problem' you are referring to is investigated here: Transient Non-stationarity and Generalisation in Deep Reinforcement Learning. Both of them together have been discussed in section 5.10.1 in this book: Multi-Agent Reinforcement Learning: Foundations and Modern Approaches