Questions tagged [continuous-state-spaces]
For questions about continuous state spaces, in the context of reinforcement learning or other AI sub-fields.
10
questions
2
votes
1
answer
61
views
Model-based RL algorithms for continuous state space and finite action space
At the beginning, if I have a complete model $p(s' \mid s, a)$ (an assumed true model that describes the environment well enough) and the reward function $r(s,a,s')$. How can I exploit the model and ...
- 35
0
votes
1
answer
60
views
RL - Can RL be applied to problems where the next state is not the next observation?
I'm quite new on the study of reinforcement learning, and Im working on a communication problem with continuous large actions range for my final graduation work. I'm trying to use Gaussian Policy and ...
1
vote
1
answer
112
views
What would be the Bellman optimality equation for $q_∗(s, a)$ for an MDP with continuous states and actions?
I'm currently studying Reinforcement Learning and I'd like to know what would be the Bellman optimality equation for action values $q_∗(s, a)$ for a MDP with continuous states and actions, written out ...
- 95
1
vote
1
answer
101
views
Reinforcement learning algorithms for large problems that are not based on a neural network
I have a large control problem with multidimensional continuous inputs (13) and outputs (3). I tried several Reinforcement learning algorithms like Deep-Q-Networks (DQN), Proximal Policy Optimization (...
- 212
0
votes
0
answers
76
views
Training a RL agent using different data at each episode
I am training a RL agent whose state is composed of two numbers, ranging between 4 ~ 16 and 0 ~ 360. The action is continuous and between 0~90. In real life, the states can be any I am training a TD3 ...
- 69
0
votes
0
answers
112
views
What do we actually 'approximate' when dealing with large state spaces in Q-learning?
I realized that my state space is very large in size. I had planned to use tabular Q-learning (Bellman equation to update the $Q(s, a)$ after each action taken). But this 'large space' realization has ...
2
votes
1
answer
2k
views
Can neural networks have continuous inputs and outputs, or do they have to be discrete?
In general, can ANNs have continuous inputs and outputs, or do they have to be discrete?
So, basically, I would like to have a mapping of continuous inputs to continuous outputs. Is this possible? ...
- 212
1
vote
0
answers
26
views
Is there a gentle introduction to reinforcement learning applied to MDPs with continuous state spaces?
I am looking for a gentle introduction (videos, lecture notes, tutorials, books) on reinforcement learning (MDPs) involving continuous states (or very large cardinality of state space). In particular, ...
- 175
18
votes
2
answers
15k
views
Can Q-learning be used for continuous (state or action) spaces?
Many examples work with a table-based method for Q-learning. This may be suitable for a discrete state (observation) or action space, like a robot in a grid world, but is there a way to use Q-learning ...
- 431
1
vote
0
answers
108
views
How to generalize finite MDP to general MDP?
Suppose, for simplicity sake, to be in a discrete time domain with the action set being the same for all states $S \in \mathcal{S}$. Thus, in a finite Markov Decision Process, the sets $\mathcal{A}$, $...
- 686