0
$\begingroup$

In the context of Reinforcement Learning, what does it mean to have a multi-dimensional continuous action space?

I came across the following in the COBRA Paper -

"A method for learning a distribution over a multi-dimensional continuous action space. This learned distribution can be sampled efficiently."

and

"During the initial exploration phase it explores its environment, in which it can move objects freely with a continuous action space but is not rewarded for its actions."

So, what do the multi-dimensionality and the continuity of the action space refer to? It'd be great if someone could provide an explanation with examples!

Thanks a lot! :)

$\endgroup$
  • 1
    $\begingroup$ It means $a \in R^n$, where $a$ is action $\endgroup$ – Brale May 1 at 13:59
  • $\begingroup$ Ohh, okay! I don't see why researchers resort to complicated terms in papers. $\endgroup$ – strawberry-sunshine May 1 at 14:00
  • 1
    $\begingroup$ I wouldn't know how to write it simpler. It is just the translation of $a \in \mathbb{R}^n$ into text. Continuous values means (most likely) $\mathbb{R}$ and multidimensional $^n$. If you understand the mathematical expression you are most likely familiar with math and if that is the case then you most likely also know about these two terms. Maybe action space is a term that is quite unfamiliar for people who are learning about reinforcement learning? That just means the set of all possible actions. $\endgroup$ – alfa May 1 at 14:33
1
$\begingroup$

Let me rephrase it a little - it's a multidimensional continuous space of actions. So, you assign each action some vector from $R^{n}$. For intuition -- imagine you have a robot arm with four joints. For every joint you could applied a rotation force from [-1, 1] and thus you get a 4-D vector with float numbers for each possible action.

| improve this answer | |
$\endgroup$
0
$\begingroup$

The question has already been answered by Kirill, but I thought I'll add a good example of a multi-dimensional continuous action space too, namely the one I just encountered in the COBRA paper itself.

"In all of our experiments we use a 2-dimensional virtual “touch-screen” environment that contains objects with configurable shape, position, and color. The agent can move any visible object by clicking on the object and clicking in a direction for the object to move. Hence the action space is continuous and 4-dimensional, namely a pair of clicks."

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.