NEAT is an evolutionary algorithm. When would you want to use NEAT over more traditional/common RL algorithms like PPO or SAC etc. What advantage does it give you?
In my opinion, this shouldn't be an either/or question. Both NEAT and rl with fixed network topology has their own advantage when solving decision problem.
NEAT is good to solve simple problem fast with minimum network topology and without local optimum issue. While RL with fixed topology suffers local optimum but learning more directly, and policy-gradient based Rl would be even more adaptive to environment with stochasticity in the context of statisics.
Then, why not just combine them? At present, the best paper on this topic is this： https://dl.acm.org/doi/10.1145/3205455.3205536 In this paper, it is suggested that doing NEAT at first, then KEEP DOING rl forever, though I don't agree on this special routine. I think, both NEAT and rl should do interactively during the WHOLE training process.
The problem is, how to combine these 2 in an effective way. One problem I met is for rl like SAC, which has 2 outputs(one for policy and one for Q-value), And topology of the Q-value output has no contribution when doing NEAT. Then, how to deal with the Q-value topology? If both the 2 outputs shares some layers at first, then the learning would become more unstable, since doing NEAT would dramatically change these 2 outputs.
The paper above takes a fixed output layers for both 2 outputs, which I think is just a workaround,not the best way.