Questions tagged [continuous-state-spaces]
For questions about continuous state spaces, in the context of reinforcement learning or other AI sub-fields.
10
questions
0
votes
1
answer
58
views
RL - Can RL be applied to problems where the next state is not the next observation?
I'm quite new on the study of reinforcement learning, and Im working on a communication problem with continuous large actions range for my final graduation work. I'm trying to use Gaussian Policy and ...
1
vote
1
answer
104
views
What would be the Bellman optimality equation for $q_∗(s, a)$ for an MDP with continuous states and actions?
I'm currently studying Reinforcement Learning and I'd like to know what would be the Bellman optimality equation for action values $q_∗(s, a)$ for a MDP with continuous states and actions, written out ...
- 91
1
vote
1
answer
92
views
Reinforcement learning algorithms for large problems that are not based on a neural network
I have a large control problem with multidimensional continuous inputs (13) and outputs (3). I tried several Reinforcement learning algorithms like Deep-Q-Networks (DQN), Proximal Policy Optimization (...
- 212
0
votes
0
answers
40
views
Can you apply TRPO to a problem involving a continuous state space and justify it theoretically?
I am currently reading and trying to understand the theory behind TRPO, i.e. sections 2 and 3 from the paper here.
Ultimately, I want to apply PPO to do a (single) stock trading task using the FinRL ...
- 101
0
votes
0
answers
70
views
Training a RL agent using different data at each episode
I am training a RL agent whose state is composed of two numbers, ranging between 4 ~ 16 and 0 ~ 360. The action is continuous and between 0~90. In real life, the states can be any I am training a TD3 ...
- 59
0
votes
0
answers
112
views
What do we actually 'approximate' when dealing with large state spaces in Q-learning?
I realized that my state space is very large in size. I had planned to use tabular Q-learning (Bellman equation to update the $Q(s, a)$ after each action taken). But this 'large space' realization has ...
2
votes
1
answer
1k
views
Can neural networks have continuous inputs and outputs, or do they have to be discrete?
In general, can ANNs have continuous inputs and outputs, or do they have to be discrete?
So, basically, I would like to have a mapping of continuous inputs to continuous outputs. Is this possible? ...
- 212
1
vote
0
answers
25
views
Is there a gentle introduction to reinforcement learning applied to MDPs with continuous state spaces?
I am looking for a gentle introduction (videos, lecture notes, tutorials, books) on reinforcement learning (MDPs) involving continuous states (or very large cardinality of state space). In particular, ...
- 175
18
votes
2
answers
14k
views
Can Q-learning be used for continuous (state or action) spaces?
Many examples work with a table-based method for Q-learning. This may be suitable for a discrete state (observation) or action space, like a robot in a grid world, but is there a way to use Q-learning ...
- 431
1
vote
0
answers
103
views
How to generalize finite MDP to general MDP?
Suppose, for simplicity sake, to be in a discrete time domain with the action set being the same for all states $S \in \mathcal{S}$. Thus, in a finite Markov Decision Process, the sets $\mathcal{A}$, $...
- 666