I am struggling to understand what makes a scheme on-policy or off-policy. From what I have read, we can say that deep Q-learning is off-policy because we use a different policy like $\epsilon$-greedy technique for exploration while training our DNN, so this makes the behavior policy ($\epsilon$-greedy) and the target policy (from DNN) to be different.
However, in the case of soft actor-critic (SAC) we use the same policy DNN to generate the data stored in the memory buffer. Later we use this data to update the policy DNN. There is only one DNN that is being used for generating the data for exploration, while this data is used for updating the policy network (same DNN). When the DNN is the same, how can we say that the behavior policy and target policy are different?