I am training a trading bot with TD3 and SAC algorithms. During the first 10k steps it takes uniformly random actions before running policy learnt so far. The agent starts to do gradient descent updates after the first 3k steps. For some reason after the initial exploration, i.e. first 10k steps (~26 episodes), episode returns drop, see below.

enter image description here

I changed learning rate, number of units in the layers, mini-batch size, added more layers but with no success. Has anyone any idea what is going on?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.