Questions tagged [continuous-state-spaces]
For questions about continuous state spaces, in the context of reinforcement learning or other AI sub-fields.
15
questions
19
votes
2
answers
21k
views
Can Q-learning be used for continuous (state or action) spaces?
Many examples work with a table-based method for Q-learning. This may be suitable for a discrete state (observation) or action space, like a robot in a grid world, but is there a way to use Q-learning ...
- 441
3
votes
1
answer
3k
views
Can neural networks have continuous inputs and outputs, or do they have to be discrete?
In general, can ANNs have continuous inputs and outputs, or do they have to be discrete?
So, basically, I would like to have a mapping of continuous inputs to continuous outputs. Is this possible? ...
- 256
3
votes
0
answers
37
views
Algorithms for average reward reinforcement learning in continuous/general state-action space
I see that discounted reward reinforcement learning has been extensively studied in the literature. However, the average reward metric receives less attention, and it looks like algorithms for this ...
- 55
2
votes
1
answer
396
views
Variable observation space at each episode
I have an enviroment with continuous actions and state variables. Every time I reset my env, between 2 and 5 balls spawn randomly in a box of 100x100 size. One of those balls (the red one) will ...
1
vote
1
answer
162
views
Model-based learning in continuous state and action spaces
I am interested in learning how transition probabilities/mdps are constructed in continuous state and action space model-based learning setting. There is some literature available on this matter, but ...
- 11
1
vote
1
answer
428
views
Model-based RL algorithms for continuous state space and finite action space
At the beginning, if I have a complete model $p(s' \mid s, a)$ (an assumed true model that describes the environment well enough) and the reward function $r(s,a,s')$. How can I exploit the model and ...
- 55
1
vote
1
answer
150
views
What would be the Bellman optimality equation for $q_∗(s, a)$ for an MDP with continuous states and actions?
I'm currently studying Reinforcement Learning and I'd like to know what would be the Bellman optimality equation for action values $q_∗(s, a)$ for a MDP with continuous states and actions, written out ...
- 145
1
vote
1
answer
136
views
Reinforcement learning algorithms for large problems that are not based on a neural network
I have a large control problem with multidimensional continuous inputs (13) and outputs (3). I tried several Reinforcement learning algorithms like Deep-Q-Networks (DQN), Proximal Policy Optimization (...
- 256
1
vote
0
answers
37
views
How do I make an artificial intelligence for a game played on a continuous board?
There is a lot of introductory literature about artificial intelligence for time and space discrete games, like chess. A few fundamental methods are offered:
Minimax to depth $n$ with an evaluation ...
- 111
1
vote
1
answer
99
views
RL - Can RL be applied to problems where the next state is not the next observation?
I'm quite new on the study of reinforcement learning, and Im working on a communication problem with continuous large actions range for my final graduation work. I'm trying to use Gaussian Policy and ...
1
vote
0
answers
27
views
Is there a gentle introduction to reinforcement learning applied to MDPs with continuous state spaces?
I am looking for a gentle introduction (videos, lecture notes, tutorials, books) on reinforcement learning (MDPs) involving continuous states (or very large cardinality of state space). In particular, ...
- 185
1
vote
0
answers
122
views
How to generalize finite MDP to general MDP?
Suppose, for simplicity sake, to be in a discrete time domain with the action set being the same for all states $S \in \mathcal{S}$. Thus, in a finite Markov Decision Process, the sets $\mathcal{A}$, $...
- 706
0
votes
0
answers
11
views
Predicting next 2D location from sparse 2D inputs which are received sequentially
Problem: You toss a coin on a 2D table with known dimension. There are certain regions on the table where the probability of get heads is high. At the maximum you can toss N=20 times at an arbitrary ...
- 13
0
votes
0
answers
112
views
Training a RL agent using different data at each episode
I am training a RL agent whose state is composed of two numbers, ranging between 4 ~ 16 and 0 ~ 360. The action is continuous and between 0~90. In real life, the states can be any I am training a TD3 ...
- 69
0
votes
0
answers
162
views
What do we actually 'approximate' when dealing with large state spaces in Q-learning?
I realized that my state space is very large in size. I had planned to use tabular Q-learning (Bellman equation to update the $Q(s, a)$ after each action taken). But this 'large space' realization has ...
