Say, we have $n$ actions to choose from, assume that after an action has been chosen, the state of the system stays the same, only our knowledge gets updated. What exactly is the goal of the multi arm bandit problem?

1. Is it that we want to find the action that has highest expected mean reward, so that after sufficiently long amount of time, we have gotten one ideal action? Here it seems that one should always go for exploration, for each state perform the experiment very high number of times, to get an accurate estimate of the expected reward and then we are guaranteed to find the best action to always go for in the future.

2. Is it that I want to maximize the total reward accumulated over time due to repetition of the experiment? Here, I will have to be careful about my strategy and hence I need to ponder over exploitation vs exploration.


**Background**


I am reading chapter 2 of Sutton's Reinforcement learning. But it seems very verbose, confusing and convoluted. Please provide any general suggestions that you might have, should I stick to the book or try something else? I think I might be too much used to definition, theorem, proof style of presentation.