I am confused by how HER learns from unsuccessful trajectories. I understand that from failed trajectories it creates 'fake' goals that it can learn from.
Ignoring HER for now, if in the case where the robotic arm reaches the goal correctly, then the value functions (V) and action-value functions (Q) that correspond to the trajectories that get to the goal quicker will increase. These high Q and V values are ultimately important for getting the optimal policy.
However if you create 'fake' goals from unsuccessful trajectories - that would increase the Q and Vs of the environment that lead to getting the 'fake' goal. Those new Q and Vs would be unhelpful and possibly detrimental for the robotic arm to reach the real goal.
What am I misunderstanding?