I am confused on a conceptual scale how I would be able to model a multi-agent reinforcement learning problem when each agent performing an action would take different durations to complete the action. This means that a certain action is performed over multiple steps and the learning sample would have that action attached to it (with different observations and rewards, possibly).
An example of this situation would be where vehicles on a 2-lane road can perform lane changing actions, but each of these actions may take anywhere between 2 - 5 seconds (or learning steps) to complete.
So, what action would need to be passed at every step? I am using RLlib framework. Is it even possible to do this? Or do all these agents have to have the same action duration / step length for any RL algorithm to work?
I would greatly appreciate if anyone could point me in the right direction on bypassing this mental block, it is driving me crazy.