1
$\begingroup$

I am trying to reproduce results presented in this paper. On page 4, the authors state:

... we train for 50 epochs (one epoch consists of 19*2*50 = 1900 full episodes), which amounts to a total of 4.75*10^6 timesteps.

The 1900 episodes are broken down into Rollouts per MPI worker (2) * Number of MPI Workers (19) * Cycles per epoch (50), as shown in the hyper parameters section on page 10.

When testing on my local machine, using the GitHub Baselines repo, I am using 1 MPI worker and the following hyperparams:

'n_cycles': 50,  # per epoch
'rollout_batch_size': 2,  # per mpi thread

By the same calculation, this means that I should have 1*50*2 = 100 episodes per epoch.

However when I run her on FetchReach-v1 turns out I only have 10 episodes per epoch. Here is a log sample:

Training...
---------------------------------
| epoch              | 0        |
| stats_g/mean       | 0.893    |
| stats_g/std        | 0.122    |
| stats_o/mean       | 0.269    |
| stats_o/std        | 0.0392   |
| test/episode       | 10       |
| test/mean_Q        | -0.602   |
| test/success_rate  | 0.5      |
| train/episode      | 10       |  <-- 10 episodes/epoch
| train/success_rate | 0        |
---------------------------------

Why is there this discrepancy? Any suggestions would be appreciated.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.