Share Your Experience: Take the 2024 Developer Survey

# Tag Info

Accepted

### What is the relation between Q-learning and policy gradients methods?

However, both approaches appear identical to me i.e. predicting the maximum reward for an action (Q-learning) is equivalent to predicting the probability of taking the action directly (PG). Both ...
• 32.4k
Accepted

### What are the differences between Q-Learning and A*?

Q-learning and A* can both be viewed as search algorithms, but, apart from that, they are not very similar. Q-learning is a reinforcement learning algorithm, i.e. an algorithm that attempts to find a ...
• 40.8k

### What is the relation between Q-learning and policy gradients methods?

This Tutorial by OpenAI offers a great comparison of different RL methods. I'll try to summarize the differences between Q-Learning and Policy Gradient methods: Objective Function In Q-Learning we ...
• 451

• 40.8k
Accepted

### What are some online courses for deep reinforcement learning?

Let me first say that deep RL is just the combination of RL with deep learning. So, if you study RL and deep learning, then studying deep RL should be straightforward. For this reason, this answer ...
• 40.8k
Accepted

### Is there any good reference for double deep Q-learning?

If you're interested in the theory behind Double Q-learning (not deep!), the reference paper would be Double Q-learning by Hado van Hasselt (2010). As for Double deep Q-learning (also called DDQN, ...
• 722

### Why do we need target network in deep Q learning?

In DQN that was presented in the original paper the update target for the Q-Network is $\left(r_t + \max_aQ(s_{t+1},a;\theta^-) - Q(s_t,a_t; \theta)\right)^2$ were $\theta^-$ is some old version of ...
• 4,910
Accepted

### What is the difference between Q-learning, Deep Q-learning and Deep Q-network?

In Q-learning (and in general value based reinforcement learning) we are typically interested in learning a Q-function, $Q(s, a)$. This is defined as Q(s, a) = \mathbb{E}_\pi\left[ G_t | S_t = s, ...
• 4,910

### Does increasing the number of Q functions in Q-Learning scale?

Probably adding more Q estimators that are trained on separate data would not improve performance, and may even degrade it. At least there is no theoretical justification. Double Q learning addresses ...
• 32.4k
Accepted

### Does increasing the number of Q functions in Q-Learning scale?

Yes, there are variations of Q-learning which use $n$ Q-functions named "enseble Q-learning" or "ensemble Q-functions". You can have a look at REDQ algorithm. The main benefit of ...
• 2,948
Accepted

### Why is the target $r + \gamma \max_{a'} Q(s', a'; \theta_i^-)$ in the loss function of the DQN architecture?

This is the problem that reinforcement learning (RL) is trying to solve: What is the best way to behave when we don’t know what the right action is and only have a scalar (the reward (r) is a scalar) ...
• 3,817