Skip to main content
5 votes
Accepted

Why are there two different q-learning formulas?

The first equation (the one from the video) looks very wrong, for a few different reasons: It doesn't involve state-action values $Q(s, a)$, but only something that looks like "state values"...
Dennis Soemers's user avatar
  • 10.4k
2 votes
Accepted

Unclear line in prioritized sweeping algorithm

$ R + \gamma {max}_aQ(S^{\prime},a) $ is the target Q value for $(S,A)$. The present estimate $Q(S,A)$ is subtracted from it to arrive at the error. The absolute value is taken to arrive at the ...
foreverska's user avatar
  • 1,298
1 vote

Proof that Temporal-Difference TD(1) is Equivalent to Widrow-Hoff

The derivation is a result of manipulating sums. Starting with $$w=w+\sum_{t=1}^{m}\alpha\sum_{k=t}^{m}(P_{k+1}-P_{k})\nabla_{w}P_{t}$$ we focus on the double sum where $m=3$. Then, $$\sum_{t=1}^{3}\...
faunedepeluse's user avatar
1 vote

What is the difference between Q-learning, Deep Q-learning and Deep Q-network?

Q-learning Q-learning is a basic reinforcement learning algorithm. It uses a Q-table to store and update the value of each state-action pair. The algorithm updates the Q-values using the Bellman ...
Kristina_Poole's user avatar
1 vote
Accepted

What is the right DRL algorithm to use when the goal in an environment is not fixed?

There is a possibility that the agent can generalize to land in an arbitrary goal region if the goal region is varied at the start of each training episode. If the locations of the goal region and ...
DeepQZero's user avatar
  • 1,424
1 vote

Which niche textbook(s) should I read to master the math of AI?

It seems that you're looking for a book that covers the mathematics that would allow you to study and understand all AI and ML topics. First of all, if you have a CS and math background, you should ...
nbro's user avatar
  • 40.9k
1 vote
Accepted

What are disadvantages/limitations of Monte Carlo Tree Search in RL?

The hard to work around traits of Monte Carlo Tree Search (MCTS) are: It is model-based, and due to using a long look-ahead distance it can be sensitive to errors in the model (for learned models). ...
Neil Slater's user avatar
  • 32.7k
1 vote

Should the experience replay memory only contain unique experiences?

This question is related to some unresolved issues in reinforcement learning with approximation. The following is a partial answer, but is the advice you should initially follow, unless you have a ...
Neil Slater's user avatar
  • 32.7k
1 vote
Accepted

Are there problems where the optimal policy is stochastic?

Sutton and Barto give an example of an MDP where the optimal stochastic policy strictly dominates the best deterministic one, should answer all your questions
Alberto's user avatar
  • 2,293

Only top scored, non community-wiki answers of a minimum length are eligible