Say I have a simple multi-agent reinforcement learning problem using vanilla policy gradient methods (i.e. REINFORCE) that is currently running with one network per agent. If I can say that each of my agents:
- are all of the same class
- have ~equivalent environmental contexts (on average)
- have no privileged state relative to other agents
- performs updates equally to all other agents
- DO use LSTMs (but store and reset memory states separately)
- DO receive rewards for individual actions based on their individual states
...is it possible to use one network for all agents so as to minimize training time? And if so, how do I combine rewards and generate losses? For instance, if I calculate loss as the -logprob * reward (in the REINFORCE case), could I simply sum or average this over all agents and then backprop accordingly?