0
$\begingroup$

I am reading AI: A Modern Approach. In the 2nd chapter when introducing different agent types, i.e., reflex, utility-based, goal-based, and learning agents, I understood that all types of agents, except learning agents, receive feedback and choose actions using the performance measure. But they do so in different ways. Model-based reflex agents possess an internal state (like a memory), while goal-based agents predict the outcome of actions and choose the one serving the goal. Lastly, utility-based functions measure the 'happiness' of each state using the utility function, which is again an internalization of the performance measure, hence all have similar nature overall. The learning agents, however, can be wrapped around the entire structure of previous agents. The entire agent's architecture is now called a Performance Element, and learning agent has an additional Learning Element, which modifies each component of the agent so as to bring the components into closer agreement with the available feedback information. But the feedback information in learning agents comes not from the performance measure embedded in the agent's structure, but from a fixed external performance standard which is part of the Critic element.

For the purpose of illustration, the structure of a utility-based agent and that of a learning agent are presented in the figure: Left: Utility-based agent, Right: Learning agent

What boggles my mind is figuring out the actual difference and interaction between performance standard and performance measure, which is perhaps related to those between learning agents and other ones. Here are my thoughts thus far:

  1. Other agents aim for maximizing the performance measure, causing them to do perfect actions. On the other hand, learning agents have the freedom of doing sub-optimal actions, which allow them to discover better actions on the long run using the performance standard.

  2. Through the performance standard's feedback (which comes from the critic as shown in figure), the learning agent can also learn a utility function or reflex component.

For providing examples, the book states that giving tip to an automated taxi is considered a performance standard. And also

hard-wired performance standards such as pain and hunger in animals can be understood in this way.

But I am still not sure about the discrepancy and interaction between the performance measure and performance standard. For instance, in the automated taxi, when confronting a road junction, the utility-based agent chooses a path that maximizes its utility function. The learning agent, however, must check different roads and after testing them, it receives feedback from outside so that eventually it would detect the user's preference. But what if we wrap a learning agent around a utility-based agent in such condition? Which has more effect, the utility function from inside, or the performance standard from outside (critic)? If they happen to contradict each other, which one would have the prevalent effect?

$\endgroup$
2
  • $\begingroup$ Could you please put your specific question in the title? If you have more than one question, split this post into multiple ones, please, so that people can focus on one problem/question at a time. In fact, it seems that you have multiple questions here, so please split this post into multiple ones: one for each question. $\endgroup$ – nbro Apr 9 at 13:03
  • $\begingroup$ I specified the question in the title. However, I'm afraid I don't know how to split the question since all of the contents are related and serve a single inquiry of mine. $\endgroup$ – Hamed Vaheb Apr 9 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.