I am trying to implement my first RL program where there are multiple agents, rather than just one. The environment I am using is the connect four game, which is turn-based.
In DQ-Learning, an agent takes a state, performs an action, gets a reward, and goes to the next state (SARS). This experience is then appended to a buffer, and the experiences are used to train the DQ network. In my implementation, each agent gets their own DQN.
My question is what happens if agent 1 wins the game? Sure, it is given an experience to be appended to a buffer, but what about the second agent? That agent should get a reward of -1 for losing, but can no longer perform actions to the board. So how do I give that agent an experience? My idea was to potentially introduce a 'null' action, which is only used when an agent is given a board that has been beaten by the other player (or draw), but I can't help but feel like there is a better solution.