Skip to main content
3 of 3
info in the comments added to the post
nbro
  • 41.4k
  • 12
  • 114
  • 205

If one of the inputs to a neural network (that represents a policy) is noisy and degrades the performance, would this architecture solve the issue?

I'm using genetic algorithms to train deep reinforcement learning (DRL) agents, similarly to what was done in this paper. DRL policies are therefore represented by deep neural networks, which map states into deterministic actions. My state space consists of three state variables $v_1, v_2$ and $v_3$. Variable $v_1$ is extremely noisy and seems to be degrading the performance (i.e. the return or discounted cumulative reward) of my RL agent but, for certain reasons, I have to include it. The return is precisely my fitness function.

Currently, my DNN looks like this:

enter image description here

There is only 1 output since the action space is 1-dimensional (one degree of freedom, which is used to control a system).

The DNN tends to overfit more quickly when $v_1$ is present. I'm considering creating a custom NN that looks like this:

enter image description here

By doing this I would reduce the complexity of the influence of the variable $v_1$ on the output, since the number of layers between $v_1$ and the output node would be reduced.

I have reasons to believe that the optimal control depends linearly or (something close to linearity) on $v_1$.

Does this make any practical sense and are there are reasons why one should avoid doing this?

BGa
  • 239
  • 1
  • 6