I have used the stable-baseline3 implementation of the SAC algorithm to train policies in a custom gym environment. So far the results look promising. However, I would like to test the robustness of the results. What are common ways to test robustness? So far, I have considered testing different seeds. Which other tests are recommended?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
This depends on your definition of robust.
Robust to what exactly?
Testing different random seeds will test the robustness of the algorithm on stochasticity of the environment and the algorithm's optimization procedure.
Trying different hyperparameters would test the robustness of the algorithm to hyperparameter changes.
Some RL benchmarks have their own definition of robustness:
The L2RPN benchmark (https://competitions.codalab.org/competitions/25426) defines robustness as a policy that is able to respond properly when unexpected events or adversarial attacks happen. In benchmarks such as Atari or Procgen, which have multiple tasks, a robust algorithm is one that can solve all the tasks.
If you mean robust, as in, the algorithm do indeed learn some inherent pattern on how to solve the task, as opposed to superficially memorizing sequences of actions, you could add noise to the observations (e.g. simple gaussian noise), try to add some sort of adversarial attacks, or try the sticky action idea used in Atari benchmarks.
