I'm using genetic algorithms to train deep reinforcement learning (DRL) agents, similarly to what was done in this paper. DRL policies are therefore represented by deep neural networks, which map states into deterministic actions. My state space consists of three state variables $v_1, v_2$ and $v_3$. Variable $v_1$ is extremely noisy and seems to be degrading the performance (i.e. the return or discounted cumulative reward) of my RL agent but, for certain reasons, I have to include it. The return is precisely my fitness function.

Currently, my DNN looks like this:

enter image description here

There is only 1 output since the action space is 1-dimensional (one degree of freedom, which is used to control a system).

The DNN tends to overfit more quickly when $v_1$ is present. I'm considering creating a custom NN that looks like this:

enter image description here

By doing this I would reduce the complexity of the influence of the variable $v_1$ on the output, since the number of layers between $v_1$ and the output node would be reduced.

I have reasons to believe that the optimal control depends linearly or (something close to linearity) on $v_1$.

Does this make any practical sense and are there are reasons why one should avoid doing this?

  • 1
    $\begingroup$ Yes, you could have a subset of the neural network that uses a linear activation function that gets as input $v_1$, but I don't know if this will really be beneficial. Let us know when you try that. $\endgroup$ – nbro Jan 30 at 20:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.