8
votes
Accepted
Is reinforcement learning only about determining the value function?
There are many algorithms that are not based on finding a value function. The most common ones are policy gradients. These methods attempt to map states to actions through a neural network. They learn ...
- 590
3
votes
Accepted
What is the advantage of using MCTS with value based methods over value based methods only?
Assuming a continuous/uncountable state space, we can only estimate our value function using function approximation, so our estimates will never be true for all states simultaneously (because, loosely ...
- 5,000
2
votes
Accepted
Is it possible for value-based methods to learn stochastic policies?
Is it possible for value-based methods to learn stochastic policies?
Yes, but only in a limited sense, due to the ways it is possible to generate stochastic policies from a value function. For ...
- 32.9k
1
vote
Can Q-learning rewards and next states be non-deterministic?
There's a lot being asked here and I don't know that I'm tracking well enough to comment on this formulation. But I will try to clarify some RL theory and answer the title question.
If the transition ...
- 1,437
1
vote
Why are policy gradient methods more effective in high-dimensional action spaces?
Above softmax in action preferences is used for policy gradient methods with (large) spaces with discrete actions, while for continuous spaces with infinite number of actions Gaussian distribution is ...
- 3,212
1
vote
Accepted
Why do we need to have two heads in D3QN to obtain value and advantage separately, if V is the average of Q values?
Regarding your first question, $$V^{\pi}(s) = \sum_{a \in A}\pi(a|s)Q^{\pi}(s,a)$$ so recovering the value function from Q really depends on what policy $\pi$ you are using. Hence, you can't really ...
- 1,291
Only top scored, non community-wiki answers of a minimum length are eligible