You should start with the general definition of [Reinforcement Learning][1] problem. And what [Markov Decision Process][2] is. 

DQN, A3C, PPO and REINFORCE are algorithms for solving reinforcement learning problems. These algorithms have their strengths and weaknesses depending on the details of the underlying problem.

[Multi-Armed Bandit][3] is not even an algorithm - it is a subclass of reinforcement learning problems, where your environment (usually) doesn't have any state transitions and your actions are just a single choice from (usually) fixed and finite set of choices.

Multi-Armed Bandit is used as an introductory problem to reinforcement learning, because it illustrates some basic concepts in the field: exploration-exploitation tradeoff, policy, target an estimate, learning rate and gradient optimization. All these concepts are basic vocabulary in RL. I recommend reading (and, very importantly, doing all the exercises) the [Sutton and Barto book][4] chapter two to get familiarized with it. 
  
*Edit: since the answer got popular, I'll address the comments and the question edit*.

Being a special simplified subset of Markov Decision Processes, Multi Armed Bandit problems allow deeper theoretical understanding. For example, (as per @NeilSlater comment) the optimal policy would be to always go for the best arm. So it makes sense to introduce "regret" $\rho$ - the difference between a potential optimal reward and the actual collected reward by agent following your strategy:

$$\rho(T) = \mathbb{E}\left[T\mu^* -\sum_{t=1}^T\mu(a_t)\right]$$

One can then study asymptotic behavior of this regret as a function of $T$ and devise strategies with different asymptotic properties. As you can see, the reward here is not discounted ($\gamma=1$) - we usually can study the behavior of it as a function of $T$ without this regularization. 

Although, there is one famous result that uses discounted rewards - the [Gittins index policy][5] (note, though, that they [use $\beta$ instead of $\gamma$][6] to denote the factor). 

[1]:https://en.wikipedia.org/wiki/Reinforcement_learning
[2]:https://en.wikipedia.org/wiki/Markov_decision_process
[3]:https://en.wikipedia.org/wiki/Multi-armed_bandit
[4]:http://incompleteideas.net/book/the-book.html

[5]:https://en.wikipedia.org/wiki/Gittins_index
[6]:https://en.wikipedia.org/wiki/Gittins_index#Mathematical_definition