I'm using genetic algorithms to train deep reinforcement learning (DRL) agents, similarly to what was done in this paper. DRL policies are therefore represented by deep neural networks, which map states into deterministic actions. My state space consists of three state variables $v_1, v_2$ and $v_3$. Variable $v_1$ is extremely noisy and seems to be degrading the performance (i.e. the return or discounted cumulative reward) of my RL agent but, for certain reasons, I have to include it. The return is precisely my fitness function.
Currently, my DNN looks like this:
There is only 1 output since the action space is 1-dimensional (one degree of freedom, which is used to control a system).
The DNN tends to overfit more quickly when $v_1$ is present. I'm considering creating a custom NN that looks like this:
By doing this I would reduce the complexity of the influence of the variable $v_1$ on the output, since the number of layers between $v_1$ and the output node would be reduced.
I have reasons to believe that the optimal control depends linearly or (something close to linearity) on $v_1$.
Does this make any practical sense and are there are reasons why one should avoid doing this?
