(2.1c) on p.1130 of the original paper on regret matching states that $$ D_t^i(j,k) = \frac{1}{t} \sum_{\tau=1}^t \big[ u^i(k,s_\tau^{-j}) - u^i(s_\tau) \big] \;. $$

At the end of page 5 of this tutorial from CMU seems to mean that $$ D_t^i(j,k) = \frac{1}{t} \sum_{\tau=1}^t \big[ u^i(k,s_\tau^{-j}) - \sum_{n=1}^N x_\tau^n u^i(n,s_\tau^{-n}) \big] \;, $$ where $X_\tau = (x_\tau^1,\cdots,x_\tau^N)$ is the mixed strategy, i.e., a PMF at time $\tau$. The notation is modified to fit the original paper.

Or, in the notation of the tutorial, the paper has $$ r^t = \sum_{\tau=1}^t \big( l^\tau - u(s_\tau) \vec{1} \big)\;, $$ where $u(s_\tau) $ is the reward received at time $\tau$, while the tutorial has $$ r^t = \sum_{\tau=1}^t \big( l^\tau - <l^\tau,x^\tau> \vec{1} \big) \;. $$

Where did I get wrong?

New contributor
Chp is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

Your Answer

Chp is a new contributor. Be nice, and check out our Code of Conduct.

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.