I am building an agent for a board game and would like to use Rapid Action Value Estimation (RAVE), which is an alternative selection approach to UCT. There are different variations of the UCT like UCT2 in here, where the action values are stored in the children rather than in the parent. The way I see it, this makes it more memory efficient (because a single value is stored at each node in the tree rather than the values for each valid actions per node) and it has also more samples since the values in the children can be updated through other paths that do not go through the parent.

I was thinking if we can adapt the same idea for the AMAF values in RAVE, so we can benefit from it by storing the AMAF values one-by-one at the corresponding child nodes.

The other question relates to the ordering of AMAF values. At the moment, I think it would not be efficient to sort them (as opposed to UCT) as there are multiple actions to update for each node during the backpropagation phase which is $\mathcal{O}(N \log N)$ per node. The best I can do is to simply update the values and check each valid child during the selection phase to find the best action. Am I correct?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.