Everything I know about Monte Carlo counterfactual regret minimization (CFR) comes from the paper Monte Carlo Sampling for Regret Minimization in Extensive Games by Marc Lanctot et al. So, I will use the same denotations as are used there.

If $V$ are the number of nodes traversed over in vanilla CFR and $M$ are the number of nodes traversed over in external sampling, then, if we divide the root block $Q$ containing all terminal histories to $\{Q_1,..., Q_n\}$, then I assume we are in external sampling traversing over $V/n$ nodes, since there are $n$ blocks and we are not traversing all of $Q$ as in vanilla CFR.

Is this the reason for external sampling, meaning we are going over fewer nodes for each iteration compared to vanilla CFR? We are only gaining $1/n$, as much information, so what is the point of that?

Are we during every iteration calculating $\tilde{r}(I,a)=(1-\sigma(a|I)\sum_{z\in Q∩Z_I}u_i(z)\pi_i^\sigma(z[I]a, z)$. Also is $Z_I⊆Q_i$? I am not complety sure what $z[I]$ in $\pi_i^\sigma(z[I]a, z)$ refer to.

Does $\pi_i^\sigma(z[I]a, z)$ mean the probability of player I to get from information set I and action a to terminal history $z\in Q∩Z_I$?

Also, how is I determined during each iteration?

  • $\begingroup$ Please, as I asked in the other post, ask one question per post. Split this post into multiple ones because your questions seem to be distinct enough, although I'm not really familiar with the paper you're reading, so maybe I'm wrong. $\endgroup$
    – nbro
    Feb 19 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.