Skip to main content
Share Your Experience: Take the 2024 Developer Survey
8 votes
Accepted

Is reinforcement learning only about determining the value function?

There are many algorithms that are not based on finding a value function. The most common ones are policy gradients. These methods attempt to map states to actions through a neural network. They learn ...
S2673's user avatar
  • 590
2 votes
Accepted

Is it possible for value-based methods to learn stochastic policies?

Is it possible for value-based methods to learn stochastic policies? Yes, but only in a limited sense, due to the ways it is possible to generate stochastic policies from a value function. For ...
Neil Slater's user avatar
  • 32.5k
2 votes
Accepted

What is the advantage of using MCTS with value based methods over value based methods only?

Assuming a continuous/uncountable state space, we can only estimate our value function using function approximation, so our estimates will never be true for all states simultaneously (because, loosely ...
David's user avatar
  • 4,920
1 vote

Can Q-learning rewards and next states be non-deterministic?

There's a lot being asked here and I don't know that I'm tracking well enough to comment on this formulation. But I will try to clarify some RL theory and answer the title question. If the transition ...
foreverska's user avatar
  • 1,288
1 vote

Why are policy gradient methods more effective in high-dimensional action spaces?

Above softmax in action preferences is used for policy gradient methods with (large) spaces with discrete actions, while for continuous spaces with infinite number of actions Gaussian distribution is ...
cinch's user avatar
  • 2,277
1 vote
Accepted

Why do we need to have two heads in D3QN to obtain value and advantage separately, if V is the average of Q values?

Regarding your first question, $$V^{\pi}(s) = \sum_{a \in A}\pi(a|s)Q^{\pi}(s,a)$$ so recovering the value function from Q really depends on what policy $\pi$ you are using. Hence, you can't really ...
calveeen's user avatar
  • 1,271

Only top scored, non community-wiki answers of a minimum length are eligible