1 vote
Accepted

Doesn't the n-step Tree Backup algorithm negatively affect the DQN-Agent by creating inconsistent look-ahead targets?

Hence, I think we would in general look only up to n-steps ahead rather than always n-steps, since the n consecutive samples from the replay buffer don't guarantee that the transitions happended ...
  • 25.4k
1 vote
Accepted

How is $Q(s', a')$ calculated in SARSA and Q-Learning?

It seems that your problem is that you think that we must know the true value of $Q(s', a')$ in order to perform the SARSA update. This is not the case! SARSA is a reinforcement learning algorithm, ...
  • 36.3k

Only top scored, non community-wiki answers of a minimum length are eligible