This depends on your definition of robust.
Robust to what exactly?
Testing different random seeds will test the robustness of the algorithm on stochasticity of the environment and the algorithm's optimization procedure.
Trying different hyperparameters would test the robustness of the algorithm to hyperparameter changes.
Some RL benchmarks have their own ...