1
$\begingroup$

I am currently exploring multi-agent reinforcement learning. I have multiple agents that communicate with each other and a central service that maintains the environment state.

The central service dispatches some information at regular intervals to all the agents (Lets call this information as energy). The information can be very different for all the agents.

The agents on reception of this information select a particular action. The execution of the action should leave the agent as well as the environment in a positive state. The action requires a limited amount of energy which might change on every timestep. If a agent does not have sufficient energy to it may request for energy from other agents. The other agents may grant or deny this request.

If all the agents are able to successfully perform their actions and leave the environment in a positive state they get a positive reward.

As the environment is stochastic, where a agent's behavior is dependent on another agent can approximate Q Learning be used here?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Not particularly sure what you are asking, so the question that I will be answering is this:

Can Q learning be used to estimate a value that depends on another value in the Q Learning Matrix even though there is a certain amount of unpredictability involved?

The answer is yes!

I will use the example of a robotic arm trying to reach a point in space since that is what I am most familiar with.

Imagine a robotic arm with a shoulder, elbow, and wrist joint. The desired elbow value depends very much on the shoulder value which is also being learned. Given enough iterations, the Q learning algorithm will come up with a solution (out of possibly many) for the elbow joint based on where the shoulder joint is at that time.

The intrinsic unpredictability (air drag, motor power, etc in this case) is countered intuitively by the Q-learning algorithm iteratively learning what works best.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I know the question is difficult to understand with less context. But the way you have framed the question is exactly what I was trying to ask "unpredictability". Thanks! Could you add a few more comments as to where or in which environment will Q-learning fail? $\endgroup$
    – amitection
    Commented Jun 18, 2018 at 16:57
  • $\begingroup$ I am not an expert in the limits of reinforcement learning, but I know that time can be limiting factor. It takes a long time to train and will go through many iterations before the output becomes useful. $\endgroup$
    – Joe S
    Commented Jun 18, 2018 at 17:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .