Questions tagged [hindsight-experience-replay]

For questions about the Hindsight Experience Replay (HER), proposed in the paper "Hindsight Experience Replay" (2017) by Marcin Andrychowicz et al.

Filter by
Sorted by
Tagged with
0 votes
0 answers
19 views

When Hindsight Experience Replay is deployed, does the input need to be augmented as well?

In Hindsight Experience Replay (HER), we augment the state representation $s_t$ with some goal $g_T$, which corresponds to the state reached after $T$ steps, such that $s'_t = s_t || g_T$. Later, some ...
user avatar
  • 71
1 vote
0 answers
30 views

How can PPO be combined with HER?

I ask because PPO is apparently an on-policy algorithm & the HER paper says that it can be combine with any off-policy algorithm. Yet I see GitHub projects that have combined them somehow? How is ...
user avatar
  • 266
1 vote
1 answer
79 views

What does $r : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$ mean in the article Hindsight Experience Replay, section 2.1?

Taken from section 2.1 in the article: We consider the standard reinforcement learning formalism consisting of an agent interacting with an environment. To simplify the exposition we assume that the ...
user avatar
1 vote
0 answers
106 views

Why would DDPG with Hindsight Experience Replay not converge?

I am trying to train a DDPG agent augmented with Hindsight Experience Replay (HER) to solve the KukaGymEnv environment. The actor and critic are simple neural networks with two hidden layers (as in ...
user avatar
1 vote
1 answer
76 views

What do the state features of KukaGymEnv represent? [closed]

I trying to use DDPG augmented with Hindsight Experience Replay (HER) on pybullet's KukaGymEnv. To formulate the feature vector for the goal state, I need to know what the features of the state of the ...
user avatar
3 votes
0 answers
85 views

Is this a good approach to solving Atari's "Montezuma's Revenge"?

I'm new to Reinforcement Learning. For an internship, I am currently training Atari's "Montezuma's Revenge" using a double Deep Q-Network with Hindsight Experience Replay (HER) (see also ...
user avatar
4 votes
1 answer
139 views

How does the optimization process in hindsight experience replay exactly work?

I was reading the following research paper Hindsight Experience Replay. This is the paper that introduces a concept called Hindsight Experience Replay (HER), which basically attempts to alleviate the ...
user avatar
2 votes
1 answer
109 views

What is the difference between success rate and reward when dealing with binary and sparse rewards?

In OpenAI Gym "reward" is defined as: reward (float): amount of reward achieved by the previous action. The scale varies between environments, but the ...
user avatar
  • 233
2 votes
0 answers
50 views

How does Hindsight Experience Replay cope with multiple goals?

What if there are multiple goals? For example, let's consider the bit-flipping environment as described in the paper HER with one small change: now, the goal is not some specific configuration, but ...
user avatar
  • 61
3 votes
1 answer
225 views

How does Hindsight Experience Replay learn from unsuccessful trajectories?

I am confused by how HER learns from unsuccessful trajectories. I understand that from failed trajectories it creates 'fake' goals that it can learn from. Ignoring HER for now, if in the case where ...
user avatar
  • 173