I am confused by how HER learns from unsuccessful trajectories. I understand that from failed trajectories it creates 'fake' goals that it can learn from.
Ignoring HER for now, if in the case where the robotic arm reaches the goal correctly, then the value functions ($V$) and action-value functions ($Q$) that correspond to the trajectories that get to the goal quicker will increase. These high $Q$ and $V$ values are ultimately important for getting the optimal policy.
However, if you create 'fake' goals from unsuccessful trajectories - that would increase the $Q$ and $V$s of the environment that lead to getting the 'fake' goal. Those new $Q$ and $V$s would be unhelpful and possibly detrimental for the robotic arm to reach the real goal.
What am I misunderstanding?