Questions tagged [continuous-state-spaces]

For questions about continuous state spaces, in the context of reinforcement learning or other AI sub-fields.

Filter by
Sorted by
Tagged with
3 votes
0 answers
33 views

Algorithms for average reward reinforcement learning in continuous/general state-action space

I see that discounted reward reinforcement learning has been extensively studied in the literature. However, the average reward metric receives less attention, and it looks like algorithms for this ...
k2pctdn's user avatar
  • 55
0 votes
0 answers
10 views

Predicting next 2D location from sparse 2D inputs which are received sequentially

Problem: You toss a coin on a 2D table with known dimension. There are certain regions on the table where the probability of get heads is high. At the maximum you can toss N=20 times at an arbitrary ...
goldfinch's user avatar
1 vote
1 answer
79 views

Model-based learning in continuous state and action spaces

I am interested in learning how transition probabilities/mdps are constructed in continuous state and action space model-based learning setting. There is some literature available on this matter, but ...
hogger's user avatar
  • 11
1 vote
1 answer
268 views

Variable observation space at each episode

I have an enviroment with continuous actions and state variables. Every time I reset my env, between 2 and 5 balls spawn randomly in a box of 100x100 size. One of those balls (the red one) will ...
Optical_flow_lover's user avatar
1 vote
1 answer
278 views

Model-based RL algorithms for continuous state space and finite action space

At the beginning, if I have a complete model $p(s' \mid s, a)$ (an assumed true model that describes the environment well enough) and the reward function $r(s,a,s')$. How can I exploit the model and ...
k2pctdn's user avatar
  • 55
1 vote
1 answer
93 views

RL - Can RL be applied to problems where the next state is not the next observation?

I'm quite new on the study of reinforcement learning, and Im working on a communication problem with continuous large actions range for my final graduation work. I'm trying to use Gaussian Policy and ...
MaarcosNascimen's user avatar
1 vote
1 answer
138 views

What would be the Bellman optimality equation for $q_∗(s, a)$ for an MDP with continuous states and actions?

I'm currently studying Reinforcement Learning and I'd like to know what would be the Bellman optimality equation for action values $q_∗(s, a)$ for a MDP with continuous states and actions, written out ...
user's user avatar
  • 145
1 vote
1 answer
131 views

Reinforcement learning algorithms for large problems that are not based on a neural network

I have a large control problem with multidimensional continuous inputs (13) and outputs (3). I tried several Reinforcement learning algorithms like Deep-Q-Networks (DQN), Proximal Policy Optimization (...
PeterBe's user avatar
  • 246
0 votes
0 answers
104 views

Training a RL agent using different data at each episode

I am training a RL agent whose state is composed of two numbers, ranging between 4 ~ 16 and 0 ~ 360. The action is continuous and between 0~90. In real life, the states can be any I am training a TD3 ...
Leibniz's user avatar
  • 69
0 votes
0 answers
140 views

What do we actually 'approximate' when dealing with large state spaces in Q-learning?

I realized that my state space is very large in size. I had planned to use tabular Q-learning (Bellman equation to update the $Q(s, a)$ after each action taken). But this 'large space' realization has ...
knowledge_seeker's user avatar
3 votes
1 answer
3k views

Can neural networks have continuous inputs and outputs, or do they have to be discrete?

In general, can ANNs have continuous inputs and outputs, or do they have to be discrete? So, basically, I would like to have a mapping of continuous inputs to continuous outputs. Is this possible? ...
PeterBe's user avatar
  • 246
1 vote
0 answers
27 views

Is there a gentle introduction to reinforcement learning applied to MDPs with continuous state spaces?

I am looking for a gentle introduction (videos, lecture notes, tutorials, books) on reinforcement learning (MDPs) involving continuous states (or very large cardinality of state space). In particular, ...
cgo's user avatar
  • 175
19 votes
2 answers
19k views

Can Q-learning be used for continuous (state or action) spaces?

Many examples work with a table-based method for Q-learning. This may be suitable for a discrete state (observation) or action space, like a robot in a grid world, but is there a way to use Q-learning ...
Bryan McGill's user avatar
1 vote
0 answers
114 views

How to generalize finite MDP to general MDP?

Suppose, for simplicity sake, to be in a discrete time domain with the action set being the same for all states $S \in \mathcal{S}$. Thus, in a finite Markov Decision Process, the sets $\mathcal{A}$, $...
gvgramazio's user avatar