12 votes

Can Q-learning be used for continuous (state or action) spaces?

Q-learning for continuous state spaces Yes, this is possible, provided you use some mechanism of approximation. One approach is to discretise the state space, and that doesn't have to reduce the ...
Neil Slater's user avatar
  • 32.1k
6 votes
Accepted

Can neural networks have continuous inputs and outputs, or do they have to be discrete?

Neural networks normally work in continuous spaces. A typical neural network function could be written as $f(\mathbf{x}, \mathbf{\theta}): \mathbb{R}^N \rightarrow \mathbb{R}^M$. That is, a function ...
Neil Slater's user avatar
  • 32.1k
2 votes

Model-based learning in continuous state and action spaces

You can use function approximation like neural networks to learn the whole environment, i.e. both the transition function, $p(s'\mid s, a)$, and the reward model, $r(s,a,s')$: $$p(s',r\mid s,a)$$ In ...
Luca Anzalone's user avatar
1 vote

Can Q-learning be used for continuous (state or action) spaces?

Q-Learning for continuous state space Reinforcement learning algorithms (e.g Q-Learning) can be applied to both discrete and continuous spaces. If you understand how it works in discrete mode, then ...
HLeb's user avatar
  • 579
1 vote

Variable observation space at each episode

Actually, in most of these algorithms, that state is just used as input for some functions (e.g. some value or policy functions). Given the correct class of functions (e.g. recurrent neural networks), ...
Broele's user avatar
  • 561
1 vote
Accepted

Model-based RL algorithms for continuous state space and finite action space

In optimal control field to minimize certain well-defined costs especially in process industries, continuous state space model-based planning methods such as model predictive control (MPC) is a common ...
cinch's user avatar
  • 2,307
1 vote
Accepted

What would be the Bellman optimality equation for $q_∗(s, a)$ for an MDP with continuous states and actions?

I think your equations are alright. Anyway, this is just a question of mathematical notation. In measure theory, a discrete random variable $X$ is said to have a counting measure over it's support $\...
easyliving's user avatar
1 vote
Accepted

Reinforcement learning algorithms for large problems that are not based on a neural network

There are many state-of-the-art reinforcement learning algorithms for large problems with multidimensional continuous state spaces and actions. All of them rely on some sort of function approximator. ...
chessprogrammer's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible