All Questions
Tagged with comparison q-learning
15 questions
2
votes
1
answer
541
views
What are the similarities between Q-learning and Value Iteration?
This is the explanation of value iteration in our notes where you keep applying bellman optimality equation till it stops changing and then acting greedily wrt the value function gives the optimal ...
- 23
3
votes
2
answers
4k
views
What is the difference between A2C and Q-Learning, and when to use one over the other?
I'm trying to get an accurate answer about the difference between A2C and Q-Learning. And when can we use each of them?
- 33
1
vote
1
answer
385
views
What is meant by "two action selections" in SARSA?
I have some difficulties understanding the difference between Q-learning and SARSA. Here (What are the differences between SARSA and Q-learning?) the following updating formulas are given:
Q-Learning
$...
- 266
6
votes
2
answers
8k
views
When to use Value Iteration vs. Policy Iteration
Both value iteration and policy iteration are General Policy Iteration (GPI) algorithms. However, they differ in the mechanics of their updates. Policy Iteration seeks to first find a completed ...
- 541
17
votes
3
answers
13k
views
What is the difference between Q-learning, Deep Q-learning and Deep Q-network?
Q-learning uses a table to store all state-action pairs. Q-learning is a model-free RL algorithm, so how could there be the one called Deep Q-learning, as deep means using DNN; or maybe the state-...
- 1,318
5
votes
1
answer
511
views
Why does off-policy learning outperform on-policy learning?
I am self-studying about Reinforcement Learning using different online resources. I now have a basic understanding of how RL works.
I saw this in a book:
Q-learning is an off-policy learner. An off-...
- 373
3
votes
1
answer
4k
views
What are the differences between Q-Learning and A*?
Q-learning seems to be related to A*. I am wondering if there are (and what are) the differences between them.
- 191
12
votes
1
answer
6k
views
What exactly is the advantage of double DQN over DQN?
I started looking into the double DQN (DDQN). Apparently, the difference between DDQN and DQN is that in DDQN we use the main value network for action selection and the target network for outputting ...
- 369
2
votes
1
answer
825
views
What are the differences between SARSA and Q-learning? [closed]
From Sutton and Barto's book Reinforcement Learning (Adaptive Computation and Machine Learning series), are the following definitions:
To aid my learning of RL and gain an intuition, I'm focusing on ...
- 335
4
votes
1
answer
358
views
What is the difference between on-policy and off-policy for continuous environments?
I'm trying to understand RL applied to time series (so with infinite horizon) which have a continous state space and a discrete action space.
First, some preliminary questions: in this case, what is ...
- 331
11
votes
2
answers
2k
views
Are Q-learning and SARSA the same when action selection is greedy?
I'm currently studying reinforcement learning and I'm having difficulties with question 6.12 in Sutton and Barto's book.
Suppose action selection is greedy. Is Q-learning then exactly the same ...
- 131
2
votes
1
answer
91
views
Do we need an explicit policy to sample $A'$ in order to compute the target in SARSA or Q-learning?
I would much appreciate if you could point me in the right direction regarding this question about targets for SARSA and Q-learning (notation: $S$ is the current state, $A$ is the current action, $R$ ...
- 123
2
votes
0
answers
92
views
What is the complexity of policy gradient algorithms compared to discrete action space algorithms?
I am using a policy gradient algorithm (actor-critic) for wireless networks. The policy gradient-based algorithm helps because it considers continuous action space.
But how much does a policy ...
- 21
5
votes
2
answers
628
views
What is the difference between return and expected return?
At a time step $t$, for a state $S_{t}$, the return is defined as the discounted cumulative reward from that time step $t$.
If an agent is following a policy (which in itself is a probability ...
- 51
47
votes
2
answers
26k
views
What is the relation between Q-learning and policy gradients methods?
As far as I understand, Q-learning and policy gradients (PG) are the two major approaches used to solve RL problems. While Q-learning aims to predict the reward of a certain action taken in a certain ...
- 595