For questions related to reinforcement learning, i.e. a machine learning technique where we imagine an agent that interacts with an environment (composed of states) in time steps by taking actions and receiving rewards (or reinforcements), then, based on these interactions, the agent tries to find a policy (i.e. a behavioural strategy) that maximizes the cumulative reward (in the long run), so the goal of the agent is to maximize the reward.
Reinforcement Learning (RL) is a machine learning technique influenced by behavioral psychology. It is based on agents associating reward signals to actions they made on a particular state in the environment.
The theory behind reinforcement learning frames problems as Markov Decision Processes (MDPs). Learning problems that fit to the MDP framework are suitable for using RL as a solution. This means that they should have the following components:
- An Environment which follows a consistent set of rules, and which evolves over a series of time steps.
- An Agent that is in the Environment, and can make meaningful decisions about how to act.
- Multiple States which influence how the Environment rules will be applied.
- Multiple Actions which the Agent may select on each time step, and which also influence the Environment.
- A system of scalar Reward values, that provide feedback on a task that the Agent is performing.
For RL theory to hold, the State should possess the Markov property, which is to say that a single state should have fixed distributions of next State and Reward, dependent only on the current State and Action.
In RL control problems, the goal of optimization is to maximize a long-term measure of reward. Most commonly this is the discounted sum of all rewards, or the average reward per time step.
There are many RL algorithms, based on different approaches to solving MDPs. They include Monte Carlo Control, SARSA, Q Learning, REINFORCE, Actor-Critic. Implementations with deep neural networks include DQN, A3C, A2C, PPO, TRPO, DPPG.
