# Why does fictitious self-play use the data collected by the average strategy for reinforcement learning?

I'm reading paper "Fictitious Self-Play in Extensive-Form Games", which introduces fictitious self-play(FPS).

In extensive-form games, let $$\beta$$ be the best response strategy, $$\pi$$ be the average strategy over a history of the best response strategies. We have the update rule for the average strategy as $$\sigma=(1-\eta)\pi+\eta\beta$$, where $$\eta$$ is the step size. In that paper, FPS uses two sources of data for reinforcement learning.

1. data sampled by the average strategy profile, i.e., $$(\sigma^i,\sigma^{-i})$$
2. data sampled by the best response strategy against opponent's average strategy profile, i.e., $$(\beta^i,\sigma^{-i})$$

My question is why FSP uses the data sampled by the average strategy profile $$(\sigma^i,\sigma^{-i})$$ for reinforcement learning? Why not use the data sampled by $$(\beta^i,\sigma^{-i})$$ only, which to my best knowledge is more suitable for training an RL algorithm due to the on-policy nature(even for an off-policy reinforcement learning method like Q-learning)?