Several important researchers distinguish between bandit problems and the general reinforcement learning problem.
The book Reinforcement learning: an introduction by Sutton and Barto describes bandit problems as a special case of the general RL problem.
The first chapter of this part of the book describes solution methods for the special case of the reinforcement learning problem in which there is only a single state, called bandit problems. The second chapter describes the general problem formulation that we treat throughout the rest of the book — finite Markov decision processes — and its main ideas including Bellman equations and value functions.
This means that you can represent your bandit problem as an MDP with a single state and possibly multiple actions.
In section 1.1.2 of the book Bandit Algorithms (2020), Szepesvari and Lattimore describe the differences between bandits and reinforcement learning
One of the distinguishing features of all bandit problems studied in this book is that the learner never needs to plan for the future. More precisely, we will invariably make the assumption that the learner's available choices and rewards tomorrow are not affected by their decisions today. Problems that do require this kind of long-term planning fall into the realm of reinforcement learning
This definition is different than the one by Sutton and Barto. In this case, only bandit problems where the learner doesn't need to plan for the future are considered.
In any case, bandit problems and RL problems have a lot of similarities. For example, both attempt to deal with the exploration-exploitation trade-off and, in both cases, the underlying problem can be formulated as a Markov decision process.