In the original DDQN article (https://arxiv.org/pdf/1509.06461.pdf,) the phrase "number of actions" is used twice;

First, in the following context: enter image description here

Secondly in Theorem 1.

enter image description here

I have a hard time understanding the way the phrase is being used or if it is being used in the same way, could anyone care to explain how it is meant to be understood?

  • 1
    $\begingroup$ nbro's answer is correct; I don't think I've ever seen anyone in RL literature referring to the number of actions that have already been taken as just the "number of actions" (they'd have to explicitly add something like "that have already been taken" to make that clear). Normally the latter would just be referred to as the time step $t$ instead. $\endgroup$
    – Dennis Soemers
    Jan 7 at 19:40

The expression "number of actions" is being used in the same way in both cases. In fact, the letter $m$ is used in both cases. The number of actions (in the state $s$) is the number of possible actions that you can take in the state $s$. So, here, $m$ does not refer to the dimensionality of an action, but to the size of the action set for a state. So, if $A(s)$ is the set of actions that you can take in $s$, then $m = |A(s)|$. The paper that they refer to in the first excerpt actually uses (on page 3, in the lemma) $n$ to refer to $m$ and states that $n$ is the number of actions. Here there's a related question, which you may want to read.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.