I've been working on a reinforcement learning (RL) problem in a highly stochastic environment where the effect of the noise far outweighs the impact of the agent's actions. To illustrate, consider the following example:
$ s' = s + a + \epsilon $
Where:
$ \epsilon \sim N(0, 0.3)$ is Gaussian noise with mean 0 and standard deviation 0.3.
$ a \in \{-0.01, 0, 0.01\}$ is the action the agent can take.
In this setup, the noise $\epsilon $ dominates the dynamics, and the effect of the agent's actions is negligible in comparison. Consequently, learning with standard Q-learning is proving to be inefficient as the noise overwhelms the learning signal.
Question: How can I efficiently learn in environments where the stochasticity (or noise) has a much stronger influence than the agent’s actions? Are there alternative RL algorithms or approaches better suited to handle such cases?
PS: Adding extra information to the state is an option but may not be favorable as it will increase the state space which I am trying to avoid for now.
Any suggestions on how to approach this problem or references to similar work would be greatly appreciated! Has anyone encountered similar issues, and how did you address them? Thank you in advance!
