0
$\begingroup$

I am using TD3 to train custom gym environment, but the problem is action values stick to the end. Sticking to the end values makes reward negative, to be positive it must find action values somewhere in the mid. But, the agent doesn't learn that and keeps action values to maximum. I am using one step termination environment (environment needs actions once for each episode). How can I improve my model? I want action values to be roughly within 80% of maximum values.

In DDPG, we have inverted gradients but could something similar applied to TD3 to make action values search within legal action space more.

Score decreases as episodes increases.

$\endgroup$
0
$\begingroup$

I found the solution, it was changing reward function and using reward scaling. A little bit change in architecture and learning rate fixed the problem.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.