I read some light material earlier about the possibility of building AI agent trees, which leaf agents optimizing for primitive tasks, while higher level agents optimizing for orchestrating direct descendant agents.

According to my understanding, each agent would have different objective functions, and possible moves.

I wonder if someone could give some insight on how to implement such complex agent, hopefully with an example.


The idea of a hierarchical agent comes from reinforcement learning. Standard learning algorithm like Q-learning or neural networks have a good performance on simple tasks. For example, if they should bring a robot to a goal. But they fail on complex tasks, for example the robot should go to an object, pickup the object and then bring the object to a goal. So called Hierarchical Reinforcement Learning works usually with layers, called modes. A mode is equal to a heuristic and is described in natural language. A possible example is, to define primitives for walking, pickup and drop-object. Each mode has a separate policy which is stored in the q-table or in the neural network.

Under the term multi-modal learning some papers were published since 2015 which are explaining the details of this concept. Usually, a hierarchical agent works very good. A recent example is an agent which can solve the “Montezuma's Revenge” game, which was previously known as to complicated for reinforcement learning because the game contains subtasks like climbing the ladder and pickup a key. The bottleneck of an agent tree is, that the modes aka heuristics has to be defined manually. That means before the algorithm can learn to solve the task, the programmer has to figure out which subtasks are in the game and how the agent should combine this to a overall plan. Some newer research goes into the direction to get this information with inverse reinforcement learning, that means to retrieve the subtasks from human-demonstration.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.