Skip to main content
Share Your Experience: Take the 2024 Developer Survey
13 votes
Accepted

Can reinforcement learning be used for tasks where only one final reward is received?

RL can be used for cases where you have sparse rewards (i.e. at almost every step all rewards are zero), but, in such a setting, the experience the agent receives during the trajectory does not ...
nbro's user avatar
  • 40.8k
5 votes
Accepted

How to improve the reward signal when the rewards are sparse?

Andrew Y. Ng (yes, that famous guy!) et al. proved, in the seminal paper Policy invariance under reward transformations: Theory and application to reward shaping (ICML, 1999), which was then part of ...
nbro's user avatar
  • 40.8k
4 votes
Accepted

What are the pros and cons of sparse and dense rewards in reinforcement learning?

What are the pros and cons of sparse and dense rewards in reinforcement learning? It is unusual to refer to this difference as "pros and cons" because that term is often used to make ...
Neil Slater's user avatar
  • 32.5k
3 votes
Accepted

How to apply Q-learning when rewards is only available at the last state?

Having only a non-zero reward at the very end is not uncommon. When rewards are sparse, it becomes a bit harder to learn compared to having lots of different rewards along the way, but for your ...
Robby Goetschalckx's user avatar
3 votes
Accepted

Are there any reliable ways of modifying the reward function to make the rewards less sparse?

Doing something like the dense, distance-based reward signal you propose is possible... but you have to do it very carefully. If you're not careful, and do it in a naive manner, you are likely to ...
Dennis Soemers's user avatar
  • 10.3k
3 votes
Accepted

How do I update Q-values in Q-learning when rewards may only be received after many actions?

In the Tableau form of Q-learning the textbook way to attribute reward to a whole chain of state/action pairs leading to a reward is an eligibility trace. Instead of $Q(S_3,A_3) = Q(S_2,A_2) = Q(S_1,...
foreverska's user avatar
  • 1,288
2 votes

How do I compute the value function when the reward is only at the end in the context of actor-critic algorithms?

The reward is given only at the end of the episode (or when there is timeout there is no reward) This is a common case. E.g. winning a board game, or reaching a goal state. How could we learn the ...
Neil Slater's user avatar
  • 32.5k
1 vote
Accepted

A question about the reward calculation in the Hindsight Experience Replay algorithm

Meanwhile I can answer my own question. After figuring it out, what the HER algorithm really needs, I found out, that the right answer is 1: you need a reward function for calculating the new reward ...
Dave's user avatar
  • 155
1 vote

How to apply Q-learning when rewards is only available at the last state?

If by, I can compute the reward given $(a_1, a_2, \dots, a_n)$ you simply mean that your game is deterministic, this is absolutely fine. I feel another answer had assumed you were implying your ...
Ryan Rudes's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible