Questions tagged [value-based-methods]

For questions about value-based reinforcement learning (RL) methods (or algorithms), which first learn a value function and then derive the policy from it. An example of a value-based RL algorithm is Q-learning.

Filter by
Sorted by
Tagged with
0 votes
0 answers
6 views

As someone starting out in RL, could you help me understand the differences between actor-only, critic-only, and actor-critic methods?

I have been reading some medium articles and these three methods pop up a lot. I am wondering what the differences between these are, what are the advantages of one over the other, etc. Also from my ...
user avatar
1 vote
0 answers
19 views

Is it possible to combine two policy-based RL agents?

I am developing an RL agent for a game environment. I have found out that there are two strategies to do well in the game. So I have trained two RL agents using neural networks with distinct reward ...
user avatar
3 votes
1 answer
105 views

What is the advantage of using MCTS with value based methods over value based methods only?

I have been trying to understand why MCTS is very important to the performance of RL agents, and the best description I found was from the paper Bootstrapping from Game Tree Search stating: ...
user avatar
  • 33
1 vote
1 answer
96 views

Question on identifiability in the “Dueling Network Architectures for Deep Reinforcement Learning” paper

I have two questions on the Dueling DQN paper. First, I have an issue on understanding the identifiability that Dueling DQN paper mentions: Here is my question: If we have given Q-values $Q(s, a; \...
user avatar
1 vote
0 answers
87 views

What kind of reinforcement learning method does AlphaGo Deepmind use to beat the best human Go player?

In reinforcement learning, there are model-based versus model-free methods. Within model-based ones, there are policy-based and value-based methods. AlphaGo Deepmind RL model has beaten the best Go ...
user avatar
5 votes
1 answer
209 views

Is reinforcement learning only about determining the value function?

I started reading some reinforcement learning literature, and it seems to me that all approaches to solving reinforcement learning problems are about finding the value function (state-value function ...
user avatar
  • 287
1 vote
0 answers
65 views

What are the disadvantages of actor-only methods with respect to value-based ones?

While the advantages of actor-only algorithms, the ones that search directly the policy without the use of the value function, are clear (possibility of having a continuous action space, a stochastic ...
user avatar
  • 281
1 vote
0 answers
33 views

Are policy-based methods better than value-based methods only for large action spaces?

In different books on reinforcement learning, policy-based methods are motivated by their ability to handle large (continuous) action spaces. Is this the only motivation for the policy-based methods? ...
user avatar
  • 362
3 votes
1 answer
225 views

Is it possible for value-based methods to learn stochastic policies?

Is it possible for value-based methods to learn stochastic policies? I'm trying to get a clear picture of the different categories for RL algorithms, and while doing so I started to think about ...
user avatar
  • 211