In my related question, I asked about the one step actor critic from The Reinforcement Learning Book by Richard Sutton et al, section 13.5:
The learning is becoming less significant as the episode progresses, by means of making $I$ smaller by the discounting factor, as the episode progresses.
How would this discounting generalize to experience replay?
Meaning, if we want to update $\theta$ and $w$ by some experience e=$(s, a, r, s')$, for which we don't know by how much to discount, how would we accomplish a correct update?
Should we remember the discount amount $I$ in the experience?
Please note the critic here is different from the critic here, because it estimates the state-value function $V(s)$, rather than the action-state-value function $Q(s,a)$