# Derive Importance Sampling as Expected Value Notation

I'm new to RL. Recently, I took a course on Coursera. In the Off-policy MC method, I learned the concept of Importance Sampling as follows: where the importance sampling ratio is the ratio of the target policy over the behavior policy.

But in Suton book the expectation under the target policy is estimated like this: Given that both sources used the same importance sampling ratio. However, I ended up getting $$E[G_{t}|s] = \sum{G_{t} b \frac{\pi}{\pi}} = \sum{G_{t} \frac{1}{\rho}\pi} = E[\frac{G_{t}}{\rho_{t:T-1}}|s]$$ instead

Did I do something wrong?

The importance sampling ratio is changing the measure of the expectation. We have the behaviour policy that generates trajectories in the environment, so we can calculate $$\mathbb E_b[G_t|S_t = s_t] = v_b(s_t)$$, but that's not what we want to calculate, we want to calculate $$\mathbb E_\pi[G_t|S_t = s_t] = v_\pi(s_t)$$. So, we need to find a scaling factor, $$\rho$$, that we can use in the first expectation to get to the second:
$$\mathbb E_b[\rho G_t|S_t = s_t] = \mathbb E_\pi[G_t|S_t = s_t] = v_\pi(s_t)$$
That scaling factor is the importance sampling ratio. In your example, the first equation, $$E[G_t|s]$$, is something we aren't interested in because we don't want to improve the behaviour policy.