Questions tagged [hindsight-experience-replay]

For questions about the Hindsight Experience Replay (HER), proposed in the paper "Hindsight Experience Replay" (2017) by Marcin Andrychowicz et al.

Filter by
Sorted by
Tagged with
1
vote
0answers
35 views

How exactly is hindsight experience replay related to potential-based reward shaping?

One of the reviewers of the HER paper (which was accepted as a NIPS conference paper) wrote Overall, I'd say that it's not a huge/deep idea, but a very nice addition to the learning toolbox. When it ...
1
vote
1answer
69 views

What does $r : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$ mean in the article Hindsight Experience Replay, section 2.1?

Taken from section 2.1 in the article: We consider the standard reinforcement learning formalism consisting of an agent interacting with an environment. To simplify the exposition we assume that the ...
1
vote
0answers
40 views

Why would DDPG with Hindsight Experience Replay not converge?

I am trying to train a DDPG agent augmented with Hindsight Experience Replay (HER) to solve the KukaGymEnv environment. The actor and critic are simple neural networks with two hidden layers (as in ...
1
vote
1answer
57 views

What do the state features of KukaGymEnv represent?

I trying to use DDPG augmented with Hindsight Experience Replay (HER) on pybullet's KukaGymEnv. To formulate the feature vector for the goal state, I need to know what the features of the state of the ...
3
votes
0answers
52 views

Is this a good approach to solving Atari's “Montezuma's Revenge”?

I'm new to Reinforcement Learning. For an internship, I am currently training Atari's "Montezuma's Revenge" using a double Deep Q-Network with Hindsight Experience Replay (HER) (see also ...
3
votes
1answer
71 views

How does the optimization process in hindsight experience replay exactly work?

I was reading the following research paper Hindsight Experience Replay. This is the paper that introduces a concept called Hindsight Experience Replay (HER), which basically attempts to alleviate the ...
2
votes
1answer
66 views

What is the difference between success rate and reward when dealing with binary and sparse rewards?

In OpenAI Gym "reward" is defined as: reward (float): amount of reward achieved by the previous action. The scale varies between environments, but the ...
2
votes
0answers
43 views

How does Hindsight Experience Replay cope with multiple goals?

What if there are multiple goals? For example, let's consider the bit-flipping environment as described in the paper HER with one small change: now, the goal is not some specific configuration, but ...
3
votes
1answer
202 views

How does Hindsight Experience Replay learn from unsuccessful trajectories?

I am confused by how HER learns from unsuccessful trajectories. I understand that from failed trajectories it creates 'fake' goals that it can learn from. Ignoring HER for now, if in the case where ...