2
$\begingroup$

The traditional setting of multiagent reinforcement learning (MARL) is the mode in which there is set of agents and external environment. And the reward is given to each agent - individually or collectively - by the external environment.

My question is - is there MARL model in which the reward is given by one agent to the other agent, meaning that one agent is incurring costs and other agent - revenue (or maybe even a profit?

Effectively that means distributed supervision: only some agents face the environment with real reward/supervision and then this supervision is more or less effectively propgated to other agents that learn/do their own specialized tasks that are part of collective task ececuted/solved distributively in MARL.

Improve this question
$\endgroup$
1
  • $\begingroup$ arxiv.org/abs/1901.08492 seems to be some part of the answer - hierarchical supervision, mutually-supervised agent learning etc. seems to be the terms used for research suggested in my question. $\endgroup$
    – TomR
    Jan 1, 2021 at 4:13

1 Answer 1

Reset to default
1
$\begingroup$

This is mostly an implementation architecture problem, and the thing is that basically you can implement anything in the traditional setting. To do so instead of having Env<->Agent1<->Agent2, you should have Agent1<->SuperEnv<->Agent2 where SuperEnv contains Env, and simply uses the reward given to SuperEnv by Agent1 and passes it to Agent2.

I know this might seem a little counter-intuitive when comparing the implementation to the real-world problem setting, but the consistency of the RL structure (i.e. Environment that interacts with all the agent) is very important for your solutions to be easily understandable by others.

Improve this answer
$\endgroup$
2
  • $\begingroup$ It would be nice if you could provide a link to some project or research paper that has done something similar to what you're suggesting. $\endgroup$
    – nbro
    Jan 2, 2021 at 22:57
  • $\begingroup$ Not aware of any project with this quite unusual setup. It is just matter of changing the environment step function essentially though (and giving the environment object an attribute that stores the reward returned by Agent1 in order to then feed this reward at the next step to Agent2) $\endgroup$
    – Federico Malerba
    Jan 3, 2021 at 7:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .