There are several important researchers (such as Sutton, Barto, Szepesvari and Lattimore) in the field of reinforcement learning and bandits that distinguish between the two.

The book [Reinforcement learning: an introduction][2] by Sutton and Barto describes bandit problems as a special case of the general RL problem.

> The first chapter of this part of the book describes solution methods for the **special case of the reinforcement learning problem** in which there is **only a single state**, called **bandit problems**. The second chapter describes the **general problem formulation** that we treat throughout the rest of the book — **finite Markov decision processes** — and its main ideas including Bellman equations and value functions.

This means that you can represent your bandit problem as an MDP with a single state and possibly multiple actions.

In [section 1.1.2 of the book Bandit Algorithms][1] (2020), Szepesvari and Lattimore describe the differences between bandits and reinforcement learning

> One of the distinguishing features of all bandit problems studied in this book
is that **the learner never needs to plan for the future**. More precisely, we will invariably make the assumption that the learner's available choices and rewards tomorrow are not affected by their decisions today. **Problems that do require this kind of long-term planning fall into the realm of reinforcement learning**

I have not yet read this book, but this distinction doesn't necessarily imply that all bandit problems do not care about the future, but only the bandit problems studied in the cited book are concerned with this setting.

The fact that there's a book completely dedicated to bandits suggests that you should distinguish bandit problems from RL problems. However, clearly, bandit problems and RL problems have a lot of similarities. For example, both attempt to deal with the exploration-exploitation trade-off (or dilemma).

 [1]: https://tor-lattimore.com/downloads/book/book.pdf#page=21
 [2]: http://incompleteideas.net/book/RLbook2020.pdf#page=45