# What does the agent in reinforcement learning exactly do?

What is an agent in reinforcement learning (RL)? I think it is not the neural network behind. What does the agent in RL exactly do?

The agent in RL is the component that makes the decision of what action to take.

In order to make that decision, the agent is allowed to use any observation from the environment, and any internal rules that it has. Those internal rules can be anything, but typically in RL, it expects the current state to be provided by the environment, for that state to have the Markov property, and then it processes that state using a policy function $$\pi(a|s)$$ that decides what action to take.

In addition, in RL we usually care about handling a reward signal (received from the environment) and optimising the agent towards maximising the expected reward in future. To do this, the agent will maintain some data which is influenced by the rewards it received in the past, and use that to construct a better policy.

One interesting thing about the definition of an agent, is that the agent/environment boundary is usually considered to be very close to the abstract decision making unit. For instance, for a robot, the agent is typically not the whole robot, but the specific program running on the robot's CPU that makes the decision on the action. All the relays/motors and other parts of the physical body of the robot are parts of the environment in RL terms. Although often loose language is used here, as the distinction might not matter in most descriptions - we would say that "the robot moves its arm to achieve the goal" when in stricter RL terms we should say that "the agent running on the robot CPU instructs the arm motors to move to achieve the goal".

I think it is not the Neural Net behind?

That is correct, the agent is more than the neural network. One or more neural networks might be part of an agent, and take the role of estimating the value of a state, or state/action pair, or even directly driving the policy function.

The agent replaces the human operator. He controls instead of him the game. The colloquial term for such a software is Aimbot. In the context of Reinforcement Learning the inner working of an aimbot is not realized with scripting languages like AutoIt, but with machine learning algorithm like q-learning and neural networks.

• I have to downvote this answer. I have read a decent amount of resources on RL and I never heard the term "aimbot" being used instead of "agent". This is not the first answer that you give where you seem to be out of context. No need to introduce more ambiguous terms as synonyms for terms that are not yet understood. Also, the agent is not just used in games and the relation between humans and agents doesn't necessarily exist. – nbro Nov 18 '18 at 14:07
• @nbro Aimbots have even a Wikipedia entry. They are described as a, quote: “computer game bot for automated target acquisition”. If a software, which is able to play a game autonomously, is out of context of reinforcement learning (RL), that the definition is too tight. Such a restricted definition is available in academic AI in which only mathematical equations are important but no real implementations. – Manuel Rodriguez Nov 18 '18 at 15:47
• aimbots don't play games autonomously.... they only /aim/ for the player autonomously. further aimbots don't employ the agent/environment paradigm that RL does, they mostly just do "(computer) memory hacking" in order to achieve their tasks. nothing related to AI/ML here! – k.c. sayz 'k.c sayz' Nov 20 '18 at 4:13