# What is meant by “generate the data” in describing the difference between on-policy and off-policy?

From the book:

Sutton, Richard S.,Barto, Andrew G.. Reinforcement Learning (Adaptive Computation and Machine Learning series) (p. 100). The MIT Press. Kindle Edition. "

following is stated:

"On-policy methods attempt to evaluate or improve the policy that is used to make decisions, whereas off-policy methods evaluate or improve a policy different from that used to generate the data."

Looking at off policy:

and on-policy:

What is meant by "generate the data"? I'm confused as to what 'data' means in this context.

Does "generate the data" translate to the actions generated by the policy ? or Does "generate the data" translate to the Q data state action mappings?

You can see from the pseudocode that both algorithms describe this behaviour policy as "policy derived from Q (e.g. $$\epsilon$$-greedy)".
• In Q learning, the $$\text{max}_a Q(S',a)$$ from the Bellman optimality equation removes the need to use A' in the update. Effectively it is a local search for actions to find one that is potentially better than A', and runs the update as if that one was the one that was taken. This revision of A' is what makes the off-learning evaluate a different target policy to the behaviour policy that generated the rest of the data used in the update.