Questions tagged [multi-armed-bandits]

For questions related to the multi-armed bandit (MAB) problem, in which a fixed limited set of resources must be allocated between competing (alternative) choices in a way that maximizes their expected gain, when each choice's properties are only partially known at the time of allocation.

Filter by
Sorted by
Tagged with
5 votes
2 answers
2k views

Are bandits considered an RL approach?

If a research paper uses multi-armed bandits (either in their standard or contextual form) to solve a particular task, can we say that they solved this task using a reinforcement learning approach? Or ...
user5093249's user avatar
4 votes
1 answer
494 views

Why do we have two similar action selection strategies for UCB1?

In the literature, there are at least two action selection strategies associated with the UCB1's action selection strategy/policy. For example, in the paper Algorithms for the multi-armed bandit ...
nbro's user avatar
  • 40.1k
3 votes
3 answers
641 views

Why aren't exploration techniques, such as UCB or Thompson sampling, used in full RL problems?

Why aren't exploration techniques, such as UCB or Thompson sampling, typically used in bandit problems, used in full RL problems? Monte Carlo Tree Search may use the above-mentioned methods in its ...
Mika's user avatar
  • 341
1 vote
1 answer
64 views

If the probabilities with which each task is selected for you do not change over time, why would it appear as a single stationary k-armed bandit task?

Sutton-Barto (Section 2.9-Associative Search (Contextual Bandits), page 41): As an example, suppose there are several different k-armed bandit tasks, and that on each step you confront one of these ...
user3489173's user avatar
1 vote
1 answer
499 views

How do we reach at the formula for UCB action-selection in multi-armed bandit problem?

I came across the formula for Upper Confidence Bound Action Selection (while studying multi-armed bandit problem), which looks like: $$ A_t \dot{=} \operatorname{argmax}_a \left[ Q_t(a) + c \sqrt{ \...
SAGALPREET SINGH's user avatar