# Tag Info

Accepted

### What does $r : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$ mean in the article Hindsight Experience Replay, section 2.1?

This answer assumes that you only have a problem with this notation from the article: $r : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$ This is a standard notation, used in many ...
• 32.7k

### Why does HER not work with on-policy RL algorithms?

In principle replay mechanisms as HER, PER, etc cannot be applied to on-policy algorithms, like SARSA and Policy gradient as stated by @Neil Slater. Anyway, If you are willing to "adapt" ...
• 2,958
Accepted

### Why does HER not work with on-policy RL algorithms?

On policy algorithms contain policy and/or value update calculations that assume data was generated by the current policy. Breaking that assumption will cause them to miscalculate, or not function at ...
• 32.7k
1 vote
Accepted

### A question about the reward calculation in the Hindsight Experience Replay algorithm

Meanwhile I can answer my own question. After figuring it out, what the HER algorithm really needs, I found out, that the right answer is 1: you need a reward function for calculating the new reward ...
• 172
1 vote
Accepted

### What do the state features of KukaGymEnv represent?

Here's an incomplete answer, but it may help. Your state is read by the function getExtendedObservation(). This function makes two things : it calls the function <...
• 601
1 vote
Accepted

### What is the difference between success rate and reward when dealing with binary and sparse rewards?

Page 6 of the paper describes the exact reward functions, and why they were used: Goals: Goals describe the desired position of the object (a box or a puck depending on the task) with some fixed ...
• 9,267
1 vote

### How does Hindsight Experience Replay learn from unsuccessful trajectories?

Ignoring HER for now the $Q$ and $V$ functions operate on states and actions which are part of a Markov decision process which we call $M_0$. Back to HER, the $Q$ and $V$ functions now take a goal as ...
• 111

Only top scored, non community-wiki answers of a minimum length are eligible