# Tag Info

### Can Q-learning be used for continuous (state or action) spaces?

Q-learning for continuous state spaces Yes, this is possible, provided you use some mechanism of approximation. One approach is to discretise the state space, and that doesn't have to reduce the ...
• 23.1k
Accepted

### Can neural networks have continuous inputs and outputs, or do they have to be discrete?

Neural networks normally work in continuous spaces. A typical neural network function could be written as $f(\mathbf{x}, \mathbf{\theta}): \mathbb{R}^N \rightarrow \mathbb{R}^M$. That is, a function ...
• 23.1k
Accepted

### What techniques are used to make MDP discrete state space manageable?

tl:dr Read chapter 9 of an Introduction of Reinforcement Learning There is definitely a problem (a curse if you will) when the dimensionality of a task (MDP) grows. For fun, lets extend your problem ...
• 3,557
Accepted

### Can a large discrete action space be represented using Gaussian distributions?

The answer is "it depends". Once you have arranged the actions into order, a key trait is whether the action value function has a simple enough shape that sampling from a Gaussian policy ...
• 23.1k

### It is possible to solve a problem with continuous action spaces and no states with reinforcement learning?

A stateless RL problem can be reduced to a Multiarmed Bandit (MAB) problem. In such a scenario, taking an action will not change the state of the agent. So, this is the setting of a conventional MAB ...
Accepted

Sounds like you have several problems with the way your policy is parametrized. You don't have to use the multivariate normal distribution. It can work, and probably others have done it already (if ...
• 746

• 4,016
1 vote

### What is meant by a multi-dimensional continuous action space?

The question has already been answered by Kirill, but I thought I'll add a good example of a multi-dimensional continuous action space too, namely the one I just encountered in the COBRA paper itself. ...
1 vote

### If the average rewards start high and then decrease, could that indicate that the PPO is stuck at a local maximum?

I had the same problem where the reward kept decreasing and started to search for answers in the forum. I let the model trained while I search. As the model trained, the reward started to increase. ...
• 11

Only top scored, non community-wiki answers of a minimum length are eligible