0
$\begingroup$

In the original DDQN article (https://arxiv.org/pdf/1509.06461.pdf,) the phrase "number of actions" is used twice;

First, in the following context: enter image description here

Secondly in Theorem 1.

enter image description here

I have a hard time understanding the way the phrase is being used or if it is being used in the same way, could anyone care to explain how it is meant to be understood?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ nbro's answer is correct; I don't think I've ever seen anyone in RL literature referring to the number of actions that have already been taken as just the "number of actions" (they'd have to explicitly add something like "that have already been taken" to make that clear). Normally the latter would just be referred to as the time step $t$ instead. $\endgroup$
    – Dennis Soemers ♦
    Commented Jan 7, 2022 at 19:40

1 Answer 1

Reset to default
1
$\begingroup$

The expression "number of actions" is being used in the same way in both cases. In fact, the letter $m$ is used in both cases. The number of actions (in the state $s$) is the number of possible actions that you can take in the state $s$. So, here, $m$ does not refer to the dimensionality of an action, but to the size of the action set for a state. So, if $A(s)$ is the set of actions that you can take in $s$, then $m = |A(s)|$. The paper that they refer to in the first excerpt actually uses (on page 3, in the lemma) $n$ to refer to $m$ and states that $n$ is the number of actions. Here there's a related question, which you may want to read.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .