Let me first summarize the question to make sure, that we are talking about the same issue. Reinforcement learning agent #1 is able to solve the inverted pendulum problem. He can execute the correct actions to maintain the pendulum in upright position. Agent #2, which is also an RL agent, doesn't control the environment but it is sending commands to Agent #1. Which means that the agent has to learn how to communicate with a different agent. Agent #1 plays the social role of the environment and the task is to optimal control this environment.
This sounds a bit like a command chain, in which the workload is distributed among different AI entities. Or to explain it easier, it's a hierarchical reinforcement learning issue in which the idea of learning is task is used recursively. Let us go back to the example problem and describe how a meaningful communication would look like. Agent #1 has the ability to maintain the balance of a pendulum. He accepts the following input commands:
- run (bring Pendulum upward)
- stop (do nothing)
The question for the manager agent is, when to execute such a command. Is it the right time to balance the pendulum or can agent #1 remain in the sleep mode and do nothing? This kind of decision is made on higher hierarchical level. Which means, agent #2 doesn't know how to balance the pendulum itself, but he knows only how to motivate a different computer program.
According to the picture, it's mainly a language parsing problem. The social role of each agent depends on the words which he accepts as input. The language space of agent #1 has to do with sending left/right statements to the environment, while the discursive sphere of agent #2 is about activating a task.