While studying artificial intelligence, I have often encountered the term "agent" (often autonomous, intelligent). For instance, in fields such as Reinforcement Learning, Multi-Agent Systems, Game Theory, Markov Decision Processes.
In an intuitive sense, it is clear to me what an agent is; I was wondering whether in AI it had a rigorous definition, perhaps expressed in mathematical language, and shared by the various AI-related fields.
What is an agent in Artificial Intelligence?
The acclaimed book Artificial Intelligence: A Modern Approach (by Stuart Russell and Peter Norvig) gives a definition of an agent
An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators.
This definition is illustrated by the following figure
This definition (and illustration) of an agent thus does not seem to include the agent as part of the environment, but this is debatable and can be a limitation, given that the environment also includes the agent.
According to this definition, humans, robots, and programs are agents. For example, a human is an agent because it possesses sensors (e.g. the eyes) and actuators (e.g. the hands, which, in this case, are also sensors) and it interacts with an environment (the world).
A percept (or perception) is composed of all perceptual inputs of the agents. The specific definition of the percept changes depending on the specific agent. For example, in the case of a human, the percept consists of all perceptual inputs from all sense organs of the human (eyes, ears, tongue, skin, and nose). In the case of a robot only equipped with a camera, the percept consists only of the camera frame (at a certain point in time). A percept sequence is a sequence of percepts.
An action is anything that has an effect on the environment. For example, in the case of a legged robot, an action can be "move forward".
An action is chosen by the agent function (which is illustrated by the white box with a black question mark in the figure above), which can also be called policy. The agent function highly determines the intelligent or intellectual capabilities of the agent and differentiates it from other agents.
Therefore, there are different agents depending on the sensors and actuators they possess, but, more importantly, depending on their policy, which highly affects their intellectual characteristics. A possible categorization of agents is
rational agents do the "right" thing (where "right", of course, depends on the context)
simple reflex agents select actions only based on the current percept (thus ignoring previous percepts)
model-based reflex agents build a model of the world (sometimes called a state) that is used to deal with cases where the current percept is insufficient to take the most appropriate action
goal-based agents possess some sort of goal information that describes situations that are desirable; for example, in the case of a human, a situation that is desirable is to have food
utility-based agents associate value with certain actions more than others; for example, if you need immediate energy, chocolate might have more value than some vegetable
learning agents update their e.g. model based on the experience or interaction with the environment
More details regarding these definitions can be found in section 2 of the book mentioned above (3rd edition). However, note that there are other possible categorizations of agents.
A reinforcement learning (RL) agent is an agent that interacts with an environment and can learn a policy (a function that determines how the agent behaves) or value (or utility) function (from which the policy can be derived) from this interaction, where the agent takes an action from the current state of the environment, and the environment emits a percept, which, in the case of RL, consists of a reinforcement (or reward) signal and the next state. The goal of the RL agent is to maximize the cumulative reward (or reinforcement) signal. An RL agent can thus be considered a rational, goal, utility-based, and learning agent. It can also be (or not) a simple reflex and model-based agent.
