Questions tagged [hindsight-experience-replay]
For questions about the Hindsight Experience Replay (HER), proposed in the paper "Hindsight Experience Replay" (2017) by Marcin Andrychowicz et al.
10
questions
0
votes
0
answers
19
views
How to establish baselines, with different training loops
My objective is to test out a new algorithm that I designed. However, I am confused whether my methodology to train the networks is correct.
I am just concerned about the training loops:
In the first ...
- 1
2
votes
0
answers
247
views
How can PPO be combined with HER?
I ask because PPO is apparently an on-policy algorithm & the HER paper says that it can be combine with any off-policy algorithm. Yet I see GitHub projects that have combined them somehow?
How is ...
- 326
1
vote
1
answer
96
views
What does $r : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$ mean in the article Hindsight Experience Replay, section 2.1?
Taken from section 2.1 in the article:
We consider the standard reinforcement learning formalism consisting of an agent interacting with an environment. To simplify the exposition we assume that the ...
1
vote
0
answers
162
views
Why would DDPG with Hindsight Experience Replay not converge?
I am trying to train a DDPG agent augmented with Hindsight Experience Replay (HER) to solve the KukaGymEnv environment. The actor and critic are simple neural networks with two hidden layers (as in ...
- 105
1
vote
1
answer
90
views
What do the state features of KukaGymEnv represent? [closed]
I trying to use DDPG augmented with Hindsight Experience Replay (HER) on pybullet's KukaGymEnv.
To formulate the feature vector for the goal state, I need to know what the features of the state of the ...
- 105
3
votes
0
answers
151
views
Is this a good approach to solving Atari's "Montezuma's Revenge"?
I'm new to Reinforcement Learning. For an internship, I am currently training Atari's "Montezuma's Revenge" using a double Deep Q-Network with Hindsight Experience Replay (HER) (see also ...
- 91
4
votes
1
answer
197
views
How does the optimization process in hindsight experience replay exactly work?
I was reading the following research paper Hindsight Experience Replay. This is the paper that introduces a concept called Hindsight Experience Replay (HER), which basically attempts to alleviate the ...
- 91
2
votes
1
answer
187
views
What is the difference between success rate and reward when dealing with binary and sparse rewards?
In OpenAI Gym "reward" is defined as:
reward (float): amount of reward achieved by the previous action. The
scale varies between environments, but the ...
- 263
2
votes
0
answers
56
views
How does Hindsight Experience Replay cope with multiple goals?
What if there are multiple goals? For example, let's consider the bit-flipping environment as described in the paper HER with one small change: now, the goal is not some specific configuration, but ...
- 61
3
votes
1
answer
252
views
How does Hindsight Experience Replay learn from unsuccessful trajectories?
I am confused by how HER learns from unsuccessful trajectories. I understand that from failed trajectories it creates 'fake' goals that it can learn from.
Ignoring HER for now, if in the case where ...
- 173