Great question! I did some research and found out that Generally capable agents emerge from open ended play by Deepmind is using ES in form of population based training:
We also explored the question, what distribution of training tasks will produce the best possible agent, especially in such a vast environment? The dynamic task generation we use allows for continual changes to the distribution of the agent’s training tasks: every task is generated to be neither too hard nor too easy, but just right for training. We then use population based training (PBT) to adjust the parameters of the dynamic task generation based on a fitness that aims to improve agents’ general capability. And finally we chain together multiple training runs so each generation of agents can bootstrap off the previous generation.
But this didn't really answer on their reasoning, so I dug some deeper and found a great article on lesswrong.com about the use of PBT. I will quote the essence, but highly recommend to read the linked Chapter on PBT:
What does the evolutionary selection give us that we don't already
have? What problem does this let us avoid?
There are several answers to this question.
The more narrow answer is that this allows the dynamic task generation
hyper parameters themselves to shift in a direction that promotes
general competence. Neither of the optimization levels beneath us
include any way of changing these parameters. But the ideal filtering
parameters for the production of general competence might be different
at the beginning, or at the middle of training. Or they might be
different from agent to agent. Without something like
population-based-training, they would have no way of changing and this
would hurt performance.
The less narrow answer, I think, is that this ensures that agents are
developing broad competence in a way the innermost loop cannot do.
[...] each agent in our population of agents will learn to get better at
some distribution of tasks, then, but without
population-based-training they might not spread themselves broadly
across the entire span of this distribution. Like a student who
advances wildly at subjects she prefers, while ignoring subjects she
is not good at, our agents might not approach our desired ideal of
general competence. Population-based-training helps prevent the
scenario by multiplying agent / teacher pairs that do well generally