Questions tagged [continuous-action-spaces]
For questions about continuous action spaces in the context of reinforcement learning (or other artificial intelligence sub-fields). There is also the tag for discrete action spaces.
18 questions with no upvoted or accepted answers
4
votes
0
answers
419
views
What is the simplest policy gradient method to implement for a problem continuous action space?
I have a problem I would like to tackle with RL, but I am not sure if it is even doable.
My agent has to figure out how to fill a very large vector (let's say from 600 to 4000 in the most complex ...
- 145
3
votes
0
answers
39
views
Algorithms for average reward reinforcement learning in continuous/general state-action space
I see that discounted reward reinforcement learning has been extensively studied in the literature. However, the average reward metric receives less attention, and it looks like algorithms for this ...
- 155
3
votes
0
answers
267
views
How to deal with variable action ranges in RL for continuous action spaces
I am reading this paper on battery management using RL. The action consist in the charging/discharging power of the battery at timestep $t$. For instance, in the case of the charging power, the ...
- 69
1
vote
1
answer
106
views
RL - Can RL be applied to problems where the next state is not the next observation?
I'm quite new on the study of reinforcement learning, and Im working on a communication problem with continuous large actions range for my final graduation work. I'm trying to use Gaussian Policy and ...
1
vote
0
answers
97
views
How can I get an integer as output for continuous action space PPO reinforcement learning?
I have a huge discrete action space, the learning stability is not good. I'd like to move to continuous action space but the only output for my task can be a positive integer (let's say in the range 0 ...
- 111
1
vote
0
answers
51
views
What Kind of Reinforcement Learning Algorithms Can Be Used when the Action Space is Unfeasibly Large?
I know Deep Q network as a $S\times A$ DNN which maps the $S$ dimensional statespace to q-values of $A$ distinct actions.
In my problem, the action space is still discrete, and finite, but depending ...
- 121
1
vote
0
answers
28
views
Is there a gentle introduction to reinforcement learning applied to MDPs with continuous state spaces?
I am looking for a gentle introduction (videos, lecture notes, tutorials, books) on reinforcement learning (MDPs) involving continuous states (or very large cardinality of state space). In particular, ...
- 185
1
vote
0
answers
1k
views
PPO in continuous control not working
I have PPO agent for discrete action space for LunarLander-v2 env in gym and it works well. However, when i am trying to solve continuous version of the same env - <...
1
vote
0
answers
110
views
Which RL algorithm would be suitable for this multi-dimensional and continuous action space?
Is there an RL approach/algorithm that would be suited for the following kind of problem?
There is a continuous action space with an action value $A_{a,t}$ for each action dimension $a$.
The ...
- 111
1
vote
0
answers
385
views
How to make SAC (Soft-Actor-Critic) learn a policy?
I cannot make SAC learn a task in a certain environment. The point is that it actually sometimes finds a very good policy, but it never learns the policy in the end. I am using the SAC implementation ...
- 11
1
vote
0
answers
593
views
Are actions deterministic during testing in continuous action space PPO?
In a continuous action space (for instance, in PPO, TRPO, REINFORCE, etc.), during training, an action is sampled from the random distribution with $\mu$ and $\sigma$. This results in an inherent ...
- 361
1
vote
0
answers
500
views
What would be the AlphaGo's performance in continuous action space?
During my research for Google DeepMind's Go-playing program Alpha Go and its successor Alpha Go Zero, I discovered that the system uses a clever pipeline and an interplay of blocks of both policy and ...
- 51
1
vote
0
answers
448
views
What adapts an algorithm to continuous or to discrete action spaces?
Some RL algorithms can only be used for environments with continuous action spaces (e.g TD3, SAC), while others only for discrete action spaces (DQN), and some for both
REINFORCE and other policy ...
- 541
1
vote
0
answers
109
views
DDPG: how to implement continuous action space bounded in the interval [-2, 2]?
I am a newbie in reinforcement learning and trying to understand how to implement continuous actions bounded by $[-2, 2]$. My research shows that doing nothing is a possible solution (i.e. action of 4....
- 73
1
vote
0
answers
124
views
How to generalize finite MDP to general MDP?
Suppose, for simplicity sake, to be in a discrete time domain with the action set being the same for all states $S \in \mathcal{S}$. Thus, in a finite Markov Decision Process, the sets $\mathcal{A}$, $...
- 706
0
votes
0
answers
55
views
How to apply DRL to solve a problem that involves mixed discrete-continuous action spaces where the action's size changes over time?
I have a reinforcement learning problem where a possible action is a probability vector $[p_1\ldots,p_n]$ of size $n\in\{1,\ldots,N\}$, where each element $p_i$ of the vector is between $0$ and $1$ ...
- 309
0
votes
0
answers
116
views
Training a RL agent using different data at each episode
I am training a RL agent whose state is composed of two numbers, ranging between 4 ~ 16 and 0 ~ 360. The action is continuous and between 0~90. In real life, the states can be any I am training a TD3 ...
- 69
0
votes
0
answers
653
views
RLlib's Multi-agent PPO continuous actions turn into nan
After some amount of training on a custom Multi-agent sparse-reward environment using RLlib's (1.4.0) PPO network, I found that my continuous actions turn into nan (explodes?) which is probably caused ...
- 243