Skip to main content
Artificial Intelligence

Return to Question

please do not add irrelevant clutter - be concise and to the point
Source Link
desertnaut
desertnaut
  • 1k
  • 11
  • 19

Question

Say, we have $n$ actions to choose from, assume that after an action has been chosen, the state of the system stays the same, only our knowledge gets updated. What exactly is the goal of the multi arm bandit problem: 1.) Is it that we want to find the action that has highest expected mean reward, so that after sufficiently long amount of time, we have gotten one ideal action. Here it seems that one should always go for exploration, for each state perform the experiment very high number of times, to get an accurate estimate of the expected reward and then we are guaranteed to find the best action to always go for in the future.?

2.) Is it that I want to maximize the total reward accumulated over time due to repetition of the experiment. Here, I will have to be careful about my strategy and hence I need to ponder over exploitation vs exploration.

  1. Is it that we want to find the action that has highest expected mean reward, so that after sufficiently long amount of time, we have gotten one ideal action? Here it seems that one should always go for exploration, for each state perform the experiment very high number of times, to get an accurate estimate of the expected reward and then we are guaranteed to find the best action to always go for in the future.

  2. Is it that I want to maximize the total reward accumulated over time due to repetition of the experiment? Here, I will have to be careful about my strategy and hence I need to ponder over exploitation vs exploration.

Background

I am reading chapter 2 of Sutton's Reinforcement learning. I come from maths background and contrary to the popular view, I am not liking the book, perhaps because I am new to the field. But it seems very verbose, confusing and convoluted. Please provide any general suggestions that you might have, should I stick to the book or try something else.? I think I might be too much used to definition, theorem, proof style of presentation.

Question

Say, we have $n$ actions to choose from, assume that after an action has been chosen, the state of the system stays the same, only our knowledge gets updated. What exactly is the goal of the multi arm bandit problem: 1.) Is it that we want to find the action that has highest expected mean reward, so that after sufficiently long amount of time, we have gotten one ideal action. Here it seems that one should always go for exploration, for each state perform the experiment very high number of times, to get an accurate estimate of the expected reward and then we are guaranteed to find the best action to always go for in the future.

2.) Is it that I want to maximize the total reward accumulated over time due to repetition of the experiment. Here, I will have to be careful about my strategy and hence I need to ponder over exploitation vs exploration.

Background

I am reading chapter 2 of Sutton's Reinforcement learning. I come from maths background and contrary to the popular view, I am not liking the book, perhaps because I am new to the field. But it seems very verbose, confusing and convoluted. Please provide any general suggestions that you might have, should I stick to the book or try something else. I think I might be too much used to definition, theorem, proof style of presentation.

Say, we have $n$ actions to choose from, assume that after an action has been chosen, the state of the system stays the same, only our knowledge gets updated. What exactly is the goal of the multi arm bandit problem?

  1. Is it that we want to find the action that has highest expected mean reward, so that after sufficiently long amount of time, we have gotten one ideal action? Here it seems that one should always go for exploration, for each state perform the experiment very high number of times, to get an accurate estimate of the expected reward and then we are guaranteed to find the best action to always go for in the future.

  2. Is it that I want to maximize the total reward accumulated over time due to repetition of the experiment? Here, I will have to be careful about my strategy and hence I need to ponder over exploitation vs exploration.

Background

I am reading chapter 2 of Sutton's Reinforcement learning. But it seems very verbose, confusing and convoluted. Please provide any general suggestions that you might have, should I stick to the book or try something else? I think I might be too much used to definition, theorem, proof style of presentation.

Became Hot Network Question
Source Link
Debu
Debu
  • 123
  • 3

Confusion in the "goal" of multi arm bandit problem

Question

Say, we have $n$ actions to choose from, assume that after an action has been chosen, the state of the system stays the same, only our knowledge gets updated. What exactly is the goal of the multi arm bandit problem: 1.) Is it that we want to find the action that has highest expected mean reward, so that after sufficiently long amount of time, we have gotten one ideal action. Here it seems that one should always go for exploration, for each state perform the experiment very high number of times, to get an accurate estimate of the expected reward and then we are guaranteed to find the best action to always go for in the future.

2.) Is it that I want to maximize the total reward accumulated over time due to repetition of the experiment. Here, I will have to be careful about my strategy and hence I need to ponder over exploitation vs exploration.

Background

I am reading chapter 2 of Sutton's Reinforcement learning. I come from maths background and contrary to the popular view, I am not liking the book, perhaps because I am new to the field. But it seems very verbose, confusing and convoluted. Please provide any general suggestions that you might have, should I stick to the book or try something else. I think I might be too much used to definition, theorem, proof style of presentation.