I do not understand the link of importance sampling to Monte Carlo off-policy learning.
We estimate a value using sampling on whole episodes, and we take these values to construct the target policy.
So, it is possible that in the target policy, we could have state values (or state action values) coming from different trajectories.
If the above is true, and if the values depend on the subsequent actions (the behavior policy), there is something wrong there, or else, better, something I do not understand.
Linking this question with importance sampling, do we use this ro value to correct this inconsistency?
Any clarification is welcome.