I'm working on a project where a non differentiable loss is there. I'm thinking about how should I deal with them.

My model is a very big lstm model (about 1M parameter), and after 500 steps (not sure if it's enough) of differential evolution, the loss didn't decrease at all.

I'm thinking that actor critic can see as follows: The policy network updates by value function, which is calculated from a reward estimator (Critic).

I'm wondering if I can take the thought of "estimator", and use it on loss estimation, then I can replace the non differentiable loss as that estimator, then also train the estimator each step. Maybe someone has already done that, if so, I want to know the name of the method.

  • $\begingroup$ what's your loss? what differential evolution method did you use? CE/CMA-ES? $\endgroup$
    – Alberto
    Commented Jan 24 at 9:54
  • $\begingroup$ The part make the loss function not differentiable is "argmax" and "split the list by a special element". And I think DE is an algorithm just like cma-es. $\endgroup$
    – TWTom
    Commented Jan 24 at 10:36
  • 1
    $\begingroup$ i mean you can definitely try reinforce, not quite sure how to help without a formal definition of the loss $\endgroup$
    – Alberto
    Commented Jan 24 at 12:27


You must log in to answer this question.