I have seeing a variation in importance sampling (IS) in Prioritized Experience Replay (PER) in some implementations regarding the original paper approach stated as (in section 3.4):

$$ w_{i}=\left(\frac{1}{N} \cdot \frac{1}{P(i)}\right)^{\beta} $$

For something like this:

$$ w_{i}=\left(\frac{\min (P(i))}{P(i)}\right)^{\beta} $$

Does anyone know where it comes from? A reference that explains the reason for that new formula and improvements obtained?

My intuition guides me to some conclusions, not necessarily correct, using this new formula:

  • In the beginning, supposing that the PER stills have empty positions, $\min(P(i)) \sim 0$, not giving too much weight for samples. But it grows substantially once the capacity is achieved as well as when the error becomes low (plus the incrementing Beta)

A code on github that applies this: link



You must log in to answer this question.