What is a learning agent?

Question

What is a learning agent, and how does it work? What are examples of learning agents (e.g., in the field of robotics)?

Answer 1

A learning agent can be defined as an agent that, over time, improves its performance (which can be defined in different ways depending on the context) based on the interaction with the environment (or experience).

The human is an example of a learning agent. For example, a human can learn to ride a bicycle, even though, at birth, no human possesses this skill.

Section 2.4.6 Learning agents (p. 54) of the AIMA book (3rd edition), by Norvig and Russell, define a learning agent as follows.

A learning agent can be divided into four conceptual components, as shown in Fig 2.15.

The four components are

1. learning element: makes improvements to the performance element (an example would be Q-learning)
2. performance element: chooses the actions to take in the environment (this is analogous to a model, e.g. a neural network, that contains the knowledge or rules to act in the environment)
3. critic: provides feedback (based on some performance metric) to the learning element in order for it to improve the performance element (so this is how you evaluate the potential improvements)
4. problem generator: suggests actions that will lead to new informative experiences (this would be a behavior policy in reinforcement learning)

At first glance, this definition might seem unrelated to the definition given above, but they are equivalent. Norvig and Russell's definition of a learning agent builds on top of their definition of an agent. Moreover, as I wrote above, if you are familiar with RL, these four components can be associated with common concepts in reinforcement learning (such as Q-learning, value function/policy, target, behavior policy). The book just uses different names to refer to the same or similar concepts.

To make the definition clearer, Norvig and Russell also provide an example

To make the overall design more concrete, let us return to the automated taxi example. The performance element consists of whatever collection of knowledge and procedures the taxi has for selecting its driving actions. The taxi goes out on the road and drives, using this performance element. The critic observes the world and passes information along to the learning element. For example, after the taxi makes a quick left turn across three lanes of traffic, the critic observes the shocking language used by other drivers. From this experience, the learning element is able to formulate a rule saying this was a bad action, and the performance element is modified by installation of the new rule. The problem generator might identify certain areas of behavior in need of improvement and suggest experiments, such as trying out the brakes on different road surfaces under different conditions.

This answer provides more definitions of the machine learning field and an ML algorithm, which is not exactly the same thing as a learning agent, given that the concept of an agent also implies or emphasizes the usage of a body with sensors, actuators, and an agent program (which converts the observations into actions), but the definitions given in this answer and in the other answer are consistent with each other.