1
$\begingroup$

In the context of Reinforcement Learning, what does it mean to have a multi-dimensional continuous action space?

I came across the following in the COBRA Paper

A method for learning a distribution over a multi-dimensional continuous action space. This learned distribution can be sampled efficiently.

and

During the initial exploration phase it explores its environment, in which it can move objects freely with a continuous action space but is not rewarded for its actions.

So, what do the multi-dimensionality and the continuity of the action space refer to? It'd be great if someone could provide an explanation with examples!

$\endgroup$
3
  • 2
    $\begingroup$ It means $a \in R^n$, where $a$ is action $\endgroup$
    – Brale
    May 1, 2020 at 13:59
  • $\begingroup$ Ohh, okay! I don't see why researchers resort to complicated terms in papers. $\endgroup$ May 1, 2020 at 14:00
  • 2
    $\begingroup$ I wouldn't know how to write it simpler. It is just the translation of $a \in \mathbb{R}^n$ into text. Continuous values means (most likely) $\mathbb{R}$ and multidimensional $^n$. If you understand the mathematical expression you are most likely familiar with math and if that is the case then you most likely also know about these two terms. Maybe action space is a term that is quite unfamiliar for people who are learning about reinforcement learning? That just means the set of all possible actions. $\endgroup$
    – alfa
    May 1, 2020 at 14:33

2 Answers 2

2
$\begingroup$

Let me rephrase it a little - it's a multidimensional continuous space of actions. So, you assign each action some vector from $R^{n}$. For intuition -- imagine you have a robot arm with four joints. For every joint you could applied a rotation force from [-1, 1] and thus you get a 4-D vector with float numbers for each possible action.

$\endgroup$
0
1
$\begingroup$

The question has already been answered by Kirill, but I thought I'll add a good example of a multi-dimensional continuous action space too, namely the one I just encountered in the COBRA paper itself.

In all of our experiments we use a 2-dimensional virtual "touch-screen" environment that contains objects with configurable shape, position, and color. The agent can move any visible object by clicking on the object and clicking in a direction for the object to move. Hence the action space is continuous and 4-dimensional, namely a pair of clicks.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .