New answers tagged return
2
The discussion uses poor notation, there should be a time index. You obtain a list of tuples $(s_t, a_t, r_t, s_{t+1})$ and then, for every visit MC, you update
$$Q(s_t, a_t) = Q(s_t, a_t) + \alpha (G_t - Q(s_t, a_t))\;;$$
where $G_t = \sum_{k=0}^\infty \gamma^k r_{t+k}$, for each $t$ in the episode. You can see that the returns for each time step are ...
Top 50 recent answers are included
Related Tags
reinforcement-learning × 19return × 19
rewards × 6
value-functions × 5
q-learning × 3
markov-decision-process × 3
expectation × 3
sutton-barto × 3
terminology × 2
comparison × 2
discount-factor × 2
importance-sampling × 2
monte-carlo-methods × 2
temporal-difference-methods × 2
definitions × 1
hyper-parameters × 1
reinforce × 1
eligibility-traces × 1
bellman-equations × 1
td-lambda × 1
off-policy-methods × 1
policies × 1
time-step × 1