The idea of a hierarchical agent comes from reinforcement learning. Standard learning algorithm like Q-learning or neural networks have a good performance on simple tasks. For example, if they should bring a robot to a goal. But they fail on complex tasks, for example the robot should go to an object, pickup the object and then bring the object to a goal. So called Hierarchical Reinforcement Learning works usually with layers, called modes. A mode is equal to a heuristic and is described in natural language. A possible example is, to define primitives for walking, pickup and drop-object. Each mode has a separate policy which is stored in the q-table or in the neural network.
Under the term multi-modal learning some papers were published since 2015 which are explaining the details of this concept. Usually, a hierarchical agent works very good. A recent example is an agent which can solve the “Montezuma's Revenge” game, which was previously known as to complicated for reinforcement learning because the game contains subtasks like climbing the ladder and pickup a key. The bottleneck of an agent tree is, that the modes aka heuristics has to be defined manually. That means before the algorithm can learn to solve the task, the programmer has to figure out which subtasks are in the game and how the agent should combine this to a overall plan. Some newer research goes into the direction to get this information with inverse reinforcement learning, that means to retrieve the subtasks from human-demonstration.