# Does "number of actions" refer to the number of actions taken or size of the action space?

In the original DDQN article (https://arxiv.org/pdf/1509.06461.pdf,) the phrase "number of actions" is used twice;

First, in the following context:

Secondly in Theorem 1.

I have a hard time understanding the way the phrase is being used or if it is being used in the same way, could anyone care to explain how it is meant to be understood?

• nbro's answer is correct; I don't think I've ever seen anyone in RL literature referring to the number of actions that have already been taken as just the "number of actions" (they'd have to explicitly add something like "that have already been taken" to make that clear). Normally the latter would just be referred to as the time step $t$ instead. Jan 7 at 19:40

## 1 Answer

The expression "number of actions" is being used in the same way in both cases. In fact, the letter $$m$$ is used in both cases. The number of actions (in the state $$s$$) is the number of possible actions that you can take in the state $$s$$. So, here, $$m$$ does not refer to the dimensionality of an action, but to the size of the action set for a state. So, if $$A(s)$$ is the set of actions that you can take in $$s$$, then $$m = |A(s)|$$. The paper that they refer to in the first excerpt actually uses (on page 3, in the lemma) $$n$$ to refer to $$m$$ and states that $$n$$ is the number of actions. Here there's a related question, which you may want to read.