In my opinion, this shouldn't be an either/or question. Both NEAT and rl with fixed network topology has their own advantage when solving decision problem.

NEAT is good to solve simple problem fast with minimum network topology and without local optimum issue. While RL with fixed topology suffers local optimum but learning more directly, and policy-gradient based Rl would be even more adaptive to environment with stochasticity in the context of statisics.

Then, why not just combine them? At present, the best paper on this topic is this： https://dl.acm.org/doi/10.1145/3205455.3205536 In this paper, it is suggested that doing NEAT at first, then KEEP DOING rl forever, though I don't agree on this special routine. I think, both NEAT and rl should do interactively during the WHOLE training process.

The problem is, how to combine these 2 in an effective way. One problem I met is for rl like SAC, which has 2 outputs(one for policy and one for Q-value), And topology of the Q-value output has no contribution when doing NEAT. Then, how to deal with the Q-value topology? If both the 2 outputs shares some layers at first, then the learning would become more unstable, since doing NEAT would dramatically change these 2 outputs.

The paper above takes a fixed output layers for both 2 outputs, which I think is just a workaround,not the best way.

In my opinion, this shouldn't be an either/or question. Both NEAT and rl with fixed network topology has their own advantage when solving decision problem.

NEAT is good to solve a simple problem fast with minimum network topology and without local optimum issue. While RL with fixed topology suffers local optimum but learning more directly, and policy-gradient based Rl would be even more adaptive to an environment with stochasticity in the context of statistics.

Then, why not just combine them? At present, the best paper on this topic is this： https://dl.acm.org/doi/10.1145/3205455.3205536 In this paper, it is suggested that doing NEAT at first, then KEEP DOING RL forever, though I don't agree with this special routine. I think, both NEAT and RL should do interactively during the WHOLE training process.

The problem is, how to combine these 2 in an effective way. One problem I met is for RL, like SAC, which has 2 outputs(one for policy and one for Q-value), And the topology of the Q-value output has no contribution when doing NEAT. Then, how to deal with the Q-value topology? If both the 2 outputs shares some layers at first, then the learning would become more unstable, since doing NEAT would dramatically change these 2 outputs.

The paper above takes a fixed output layer for both 2 outputs, which I think is just a workaround, not the best way.

In my opinion, this shouldn't be an either/or question. Both NEAT and rl with fixed network topology has their own advantage when solving decision problem.

NEAT is good to solve simple problem fast with minimum network topology and without local optimum issue. While RL with fixed topology suffers local optimum but learning more directly, and policy-gradient based Rl would be even more adaptive to environment with stochasticity in the context of statisics.

Then, why not just combine them? At present, the best paper on this topic is this： https://dl.acm.org/doi/10.1145/3205455.3205536 In this paper, it is suggested that doing NEAT at first, then KEEP DOING rl forever, though I don't agree on this special routine. I think, both NEAT and rl should do interactively during the WHOLE training process.

The problem is, how to combine these 2 in an effective way. One problem I met is for rl like SAC, which has 2 outputs(one for policy and one for Q-value), And topology of the Q-value output has no contribution when doing NEAT. Then, how to deal with the Q-value topology? If both the 2 outputs shares some layers at first, then the learning would become more unstable, since doing NEAT would dramatically change these 2 outputs.

In my opinion, this shouldn't be an either/or question. Both NEAT and rl with fixed network topology has their own advantage when solving decision problem.

NEAT is good to solve simple problem fast with minimum network topology and without local optimum issue. While RL with fixed topology suffers local optimum but learning more directly, and policy-gradient based Rl would be even more adaptive to environment with stochasticity in the context of statisics.

Then, why not just combine them? At present, the best paper on this topic is this： https://dl.acm.org/doi/10.1145/3205455.3205536 In this paper, it is suggested that doing NEAT at first, then KEEP DOING rl forever, though I don't agree on this special routine. I think, both NEAT and rl should do interactively during the WHOLE training process.

The problem is, how to combine these 2 in an effective way. One problem I met is for rl like SAC, which has 2 outputs(one for policy and one for Q-value), And topology of the Q-value output has no contribution when doing NEAT. Then, how to deal with the Q-value topology? If both the 2 outputs shares some layers at first, then the learning would become more unstable, since doing NEAT would dramatically change these 2 outputs.

The paper above takes a fixed output layers for both 2 outputs, which I think is just a workaround,not the best way.