I have questions regarding on how to implement PBT as described in Algorithm 1 (on page 5) in the paper, Population Based Training of Neural Networks to train agents in a MARL (multi-agent reinforcement learning) environment.
In a single agent RL environment, the environments can be distributed & agents trained in parallel & there will be a need to maintain some sort of centralized population of weights & hyperparameters. (Please correct me if I'm wrong.)
In a MARL context, do I need to also maintain a centralized population for all agents in all environments or do I need to maintain a separate population for agents in each distributed environment? Which is a correct or more effective approach?
Any pointers would be appreciated. Thank you.
