I am using Open AI's code to do a RL task on an environment that I built myself.
I tried some network architectures, and they all converge, faster or slower on CartPole.
On my environment, the reward seems not to converge, and keeps flickering forever.
I suspect the neural network is too small, but I want to confirm my belief before going the route of researching the architecture.
How can I confirm that the architecture is the problem and not anything else in a neural network reinforcement learning task?