Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
Can you describe this reward system in more detail?
I understand that the environment sends a signal indicating whether or not the action taken by the agent was 'good' or not, but it seems too simple.
Basically, can you detail the nitty-gritty workings of this system? I dunno, I may just be overthinking things.
In this case, the word "system" refers to a Markov decision process (MDP), which is the mathematical model used to represent the reinforcement learning (RL) problem or, in general, a decision making problem. Recall that, in RL, the problem consists in finding an (optimal) policy, which is a policy that allows the agent to collect the highest amount of reward (in the long run). Hence, in RL, the MDP is the problem and the optimal policy (for that MDP) is the solution.
The MDP is composed of the set of states of the environment $S$, the set of possible actions that the RL agent can take $A$, a transition function, $P(s_{t+1}=s'\mid s_{t}=s, a_t = a)$, which is a probability distribution and describes the dynamics of the environment, and a reward function $R_a(s', s)$, which is a function that describes the "reward system" of the environment. $R_a(s', s)$ can be thought of as the reward (signal) that the agent receives after having taken action $a$ in state $s$ and having landed in state $s'$.
In other words, the reward system is just this function $R_a(s', s)$, which is often the hardest function to define when modelling an RL problem. However, in certain cases, this function can be easily defined. For example, in the case of chess, you could define $R_a(s', s) = 1$, if $s'$ is the terminal state (checkmate), else $R_a(s', s) = 0$. Nonetheless, we could define the reward function for a chess environment differently. For example, we could give a reward of $0.5$ for certain "clever" moves. So, the reward function needs to be defined (by the programmer) in order to solve an RL problem and it highly affects the way the agent will learn.
The reward function can also be denoted by $R_a(s)$, which can be thought of as the reward that the agent receives after having taken action $a$ in state $s$ (no matter which state it lands in), or $R(s)$, which can be thought of as the reward the agent receives either when it enters or exits state $s$.
