# Tag Info

Accepted

### Why are there two different q-learning formulas?

The first equation (the one from the video) looks very wrong, for a few different reasons: It doesn't involve state-action values $Q(s, a)$, but only something that looks like "state values"...
• 10.4k
Accepted

### Unclear line in prioritized sweeping algorithm

$R + \gamma {max}_aQ(S^{\prime},a)$ is the target Q value for $(S,A)$. The present estimate $Q(S,A)$ is subtracted from it to arrive at the error. The absolute value is taken to arrive at the ...
• 1,298
1 vote

### Proof that Temporal-Difference TD(1) is Equivalent to Widrow-Hoff

The derivation is a result of manipulating sums. Starting with $$w=w+\sum_{t=1}^{m}\alpha\sum_{k=t}^{m}(P_{k+1}-P_{k})\nabla_{w}P_{t}$$ we focus on the double sum where $m=3$. Then, \sum_{t=1}^{3}\...
1 vote

### What is the difference between Q-learning, Deep Q-learning and Deep Q-network?

Q-learning Q-learning is a basic reinforcement learning algorithm. It uses a Q-table to store and update the value of each state-action pair. The algorithm updates the Q-values using the Bellman ...
1 vote
Accepted

### What is the right DRL algorithm to use when the goal in an environment is not fixed?

There is a possibility that the agent can generalize to land in an arbitrary goal region if the goal region is varied at the start of each training episode. If the locations of the goal region and ...
• 1,424
1 vote

### Which niche textbook(s) should I read to master the math of AI?

It seems that you're looking for a book that covers the mathematics that would allow you to study and understand all AI and ML topics. First of all, if you have a CS and math background, you should ...
• 40.9k
1 vote
Accepted

### What are disadvantages/limitations of Monte Carlo Tree Search in RL?

The hard to work around traits of Monte Carlo Tree Search (MCTS) are: It is model-based, and due to using a long look-ahead distance it can be sensitive to errors in the model (for learned models). ...
• 32.7k
1 vote

### Should the experience replay memory only contain unique experiences?

This question is related to some unresolved issues in reinforcement learning with approximation. The following is a partial answer, but is the advice you should initially follow, unless you have a ...
• 32.7k
1 vote
Accepted

### Are there problems where the optimal policy is stochastic?

Sutton and Barto give an example of an MDP where the optimal stochastic policy strictly dominates the best deterministic one, should answer all your questions
• 2,293

Only top scored, non community-wiki answers of a minimum length are eligible