Return to Answer

There are several important researchers (such as Sutton and Szepesvari) in the field of reinforcement learning and bandits that distinguish between the two.

The book Reinforcement learning: an introduction by Sutton and Barto describes bandit problems as a special case of the general RL problem.

The first chapter of this part of the book describes solution methods for the special case of the reinforcement learning problem in which there is only a single state, called bandit problems. The second chapter describes the general problem formulation that we treat throughout the rest of the book — finite Markov decision processes — and its main ideas including Bellman equations and value functions.

This means that you can represent your bandit problem as an MDP with a single state and possibly multiple actions.

In section 1.1.2 of the book Bandit Algorithms (2020), Szepesvari and Lattimore describe the differences between bandits and reinforcement learning

One of the distinguishing features of all bandit problems studied in this book is that the learner never needs to plan for the future. More precisely, we will invariably make the assumption that the learner's available choices and rewards tomorrow are not affected by their decisions today. Problems that do require this kind of long-term planning fall into the realm of reinforcement learning

This definition is different than the one by Sutton and Barto. In this case, only bandit problems where the learner doesn't need to plan for the future are considered.

In any case, bandit problems and RL problems have a lot of similarities. For example, both attempt to deal with the exploration-exploitation trade-off (or dilemma).

Several important researchers distinguish between bandit problems and the general reinforcement learning problem.

The book Reinforcement learning: an introduction by Sutton and Barto describes bandit problems as a special case of the general RL problem.

The first chapter of this part of the book describes solution methods for the special case of the reinforcement learning problem in which there is only a single state, called bandit problems. The second chapter describes the general problem formulation that we treat throughout the rest of the book — finite Markov decision processes — and its main ideas including Bellman equations and value functions.

This means that you can represent your bandit problem as an MDP with a single state and possibly multiple actions.

In section 1.1.2 of the book Bandit Algorithms (2020), Szepesvari and Lattimore describe the differences between bandits and reinforcement learning

One of the distinguishing features of all bandit problems studied in this book is that the learner never needs to plan for the future. More precisely, we will invariably make the assumption that the learner's available choices and rewards tomorrow are not affected by their decisions today. Problems that do require this kind of long-term planning fall into the realm of reinforcement learning

This definition is different than the one by Sutton and Barto. In this case, only bandit problems where the learner doesn't need to plan for the future are considered.

In any case, bandit problems and RL problems have a lot of similarities. For example, both attempt to deal with the exploration-exploitation trade-off and, in both cases, the underlying problem can be formulated as a Markov decision process.

There are several important researchers (such as Sutton, Barto, Szepesvari and Lattimore) in the field of reinforcement learning and bandits that distinguish between the two.

The book Reinforcement learning: an introduction by Sutton and Barto describes bandit problems as a special case of the general RL problem.

The first chapter of this part of the book describes solution methods for the special case of the reinforcement learning problem in which there is only a single state, called bandit problems. The second chapter describes the general problem formulation that we treat throughout the rest of the book — finite Markov decision processes — and its main ideas including Bellman equations and value functions.

This means that you can represent your bandit problem as an MDP with a single state and possibly multiple actions.

In section 1.1.2 of the book Bandit Algorithms (2020), Szepesvari and Lattimore describe the differences between bandits and reinforcement learning

One of the distinguishing features of all bandit problems studied in this book is that the learner never needs to plan for the future. More precisely, we will invariably make the assumption that the learner's available choices and rewards tomorrow are not affected by their decisions today. Problems that do require this kind of long-term planning fall into the realm of reinforcement learning

I have not yet read this book, but this distinction doesn't necessarily imply that all bandit problems do not care about the future, but that the bandit problems studied in the cited book are associated with this setting.

The fact that there's a book completely dedicated to bandits suggests that you should distinguish bandit problems from RL problems. However, clearly, bandit problems and RL problems have a lot of similarities. For example, both attempt to deal with the exploration-exploitation trade-off (or dilemma), which is also a reason why certain RL books (like the one cited above) start with the bandits.

There are several important researchers (such as Sutton and Szepesvari) in the field of reinforcement learning and bandits that distinguish between the two.

The book Reinforcement learning: an introduction by Sutton and Barto describes bandit problems as a special case of the general RL problem.

The first chapter of this part of the book describes solution methods for the special case of the reinforcement learning problem in which there is only a single state, called bandit problems. The second chapter describes the general problem formulation that we treat throughout the rest of the book — finite Markov decision processes — and its main ideas including Bellman equations and value functions.

This means that you can represent your bandit problem as an MDP with a single state and possibly multiple actions.

In section 1.1.2 of the book Bandit Algorithms (2020), Szepesvari and Lattimore describe the differences between bandits and reinforcement learning

One of the distinguishing features of all bandit problems studied in this book is that the learner never needs to plan for the future. More precisely, we will invariably make the assumption that the learner's available choices and rewards tomorrow are not affected by their decisions today. Problems that do require this kind of long-term planning fall into the realm of reinforcement learning

This definition is different than the one by Sutton and Barto. In this case, only bandit problems where the learner doesn't need to plan for the future are considered.

In any case, bandit problems and RL problems have a lot of similarities. For example, both attempt to deal with the exploration-exploitation trade-off (or dilemma).

There are several important researchers (such as Sutton, Barto, Szepesvari and Lattimore) in the field of reinforcement learning and bandits that distinguish between the two.

The book Reinforcement learning: an introduction by Sutton and Barto describes bandit problems as a special case of the general RL problem.

The first chapter of this part of the book describes solution methods for the special case of the reinforcement learning problem in which there is only a single state, called bandit problems. The second chapter describes the general problem formulation that we treat throughout the rest of the book — finite Markov decision processes — and its main ideas including Bellman equations and value functions.

This means that you can represent your bandit problem as an MDP with a single state and possibly multiple actions.

In section 1.1.2 of the book Bandit Algorithms (2020), Szepesvari and Lattimore describe the differences between bandits and reinforcement learning

One of the distinguishing features of all bandit problems studied in this book is that the learner never needs to plan for the future. More precisely, we will invariably make the assumption that the learner's available choices and rewards tomorrow are not affected by their decisions today. Problems that do require this kind of long-term planning fall into the realm of reinforcement learning

I have not yet read this book, but this distinction doesn't necessarily imply that all bandit problems do not care about the future, but that the bandit problems studied in the cited book are associated with this setting.

The fact that there's a book completely dedicated to bandits suggests that you should distinguish bandit problems from RL problems. However, clearly, bandit problems and RL problems have a lot of similarities. For example, both attempt to deal with the exploration-exploitation trade-off (or dilemma), which is also a reason why certain RL books (like the one cited above) start with the idea of the bandits.

There are several important researchers (such as Sutton, Barto, Szepesvari and Lattimore) in the field of reinforcement learning and bandits that distinguish between the two.

The book Reinforcement learning: an introduction by Sutton and Barto describes bandit problems as a special case of the general RL problem.

The first chapter of this part of the book describes solution methods for the special case of the reinforcement learning problem in which there is only a single state, called bandit problems. The second chapter describes the general problem formulation that we treat throughout the rest of the book — finite Markov decision processes — and its main ideas including Bellman equations and value functions.

This means that you can represent your bandit problem as an MDP with a single state and possibly multiple actions.

In section 1.1.2 of the book Bandit Algorithms (2020), Szepesvari and Lattimore describe the differences between bandits and reinforcement learning

One of the distinguishing features of all bandit problems studied in this book is that the learner never needs to plan for the future. More precisely, we will invariably make the assumption that the learner's available choices and rewards tomorrow are not affected by their decisions today. Problems that do require this kind of long-term planning fall into the realm of reinforcement learning

I have not yet read this book, but this distinction doesn't necessarily imply that all bandit problems do not care about the future, but that the bandit problems studied in the cited book are associated with this setting.

The fact that there's a book completely dedicated to bandits suggests that you should distinguish bandit problems from RL problems. However, clearly, bandit problems and RL problems have a lot of similarities. For example, both attempt to deal with the exploration-exploitation trade-off (or dilemma), which is also a reason why certain RL books (like the one cited above) start with the bandits.