# Is there multi-agent reinforcement learning model in which (some of the) reward is given by other agent and not by the external environment?

The traditional setting of multiagent reinforcement learning (MARL) is the mode in which there is set of agents and external environment. And the reward is given to each agent - individually or collectively - by the external environment.

My question is - is there MARL model in which the reward is given by one agent to the other agent, meaning that one agent is incurring costs and other agent - revenue (or maybe even a profit?

Effectively that means distributed supervision: only some agents face the environment with real reward/supervision and then this supervision is more or less effectively propgated to other agents that learn/do their own specialized tasks that are part of collective task ececuted/solved distributively in MARL.

• arxiv.org/abs/1901.08492 seems to be some part of the answer - hierarchical supervision, mutually-supervised agent learning etc. seems to be the terms used for research suggested in my question.
– TomR
Jan 1 at 4:13

This is mostly an implementation architecture problem, and the thing is that basically you can implement anything in the traditional setting. To do so instead of having Env<->Agent1<->Agent2, you should have Agent1<->SuperEnv<->Agent2 where SuperEnv contains Env, and simply uses the reward given to SuperEnv by Agent1 and passes it to Agent2.
• Not aware of any project with this quite unusual setup. It is just matter of changing the environment step function essentially though (and giving the environment object an attribute that stores the reward returned by Agent1 in order to then feed this reward at the next step to Agent2) Jan 3 at 7:32