Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [multi-armed-bandits]

For questions related to the multi-armed bandit (MAB) problem, in which a fixed limited set of resources must be allocated between competing (alternative) choices in a way that maximizes their expected gain, when each choice's properties are only partially known at the time of allocation.

30 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3 votes
0 answers
26 views

Mapping given probabilities to empirical probabilities

Consider following problem statement: You have given $n$ actions. You can perform any of them. Each action gives you success with some probability. The challenge is to perform given finite number of ...
Rnj's user avatar
  • 221
2 votes
0 answers
129 views

Is it better to model a Contextual Multi-Armed Bandit problem as an MDP with a non-zero discount factor than treating it as it is?

I'd like to ask if it is, generally, better to model a problem that naturally appears as a Contextual Multi-Armed Bandit like Recommender Systems as a Markov Decision Process with a non-zero discount ...
Daviiid's user avatar
  • 575
2 votes
0 answers
57 views

Multi-armed bandit problem without getting rewards

In a 2-armed-bandit problem, an agent has an opportunity to see n reward for each action. Now the agent should choose actions m times and maximize the expected reward in these m decisions. but it cant ...
lighting's user avatar
  • 121
2 votes
0 answers
88 views

Is there a UCB type algorithm for linear stochastic bandit with lasso regression?

Why is there no upper confidence bound algorithm for linear stochastic bandits that uses lasso regression in the case that the regression parameters are sparse in the features? In particular, I don't ...
PJORR's user avatar
  • 21
2 votes
0 answers
164 views

Solving multi-armed bandit problems with continuous action space

My problem has a single state and an infinite amount of actions on a certain interval (0,1). After quite some time of googling I found a few paper about an algorithm called zooming algorithm which can ...
Peter's user avatar
  • 21
1 vote
2 answers
102 views

References on Theoretical Bandit Problem

I am going to start learning the bandit problem and algorithm, especially how to bound the regret. I found the book ``Bandit Algorithms'' but it is not easy to follow. It is based on advanced ...
Amin's user avatar
  • 481
1 vote
0 answers
50 views

Data Imbalance in Contextual Bandit with Thompson Sampling

I'm working with the Online Logistic Regression Algorithm (Algorithm 3) of Chapelle and Li in their paper, "An Empirical Evaluation of Thompson Sampling" (https://papers.nips.cc/paper/2011/...
MABQ's user avatar
  • 11
1 vote
0 answers
65 views

Policy gradient (or more general, RL algorithms) for the problems where actions does not determine next state (next state is independent to action)

I am pretty new in RL. Could anyone suggest results/paper about whether or not policy gradient (or more general RL algorithms) can be applied to the problems where actions does not determine next ...
Penn's user avatar
  • 11
1 vote
0 answers
85 views

Can vanilla multi armed bandit problems be solved by RL algorithms like A2C and PPO?

Let's say we have N bandit machines with some distributions (assume some are gaussian, some are uniform, some are chi squared). We want to maximize rewards in X amount of time. I am aware that ...
Prasanjit Rath's user avatar
1 vote
0 answers
113 views

Is there a paper/article on contextual $\epsilon$-greedy algorithm?

I am reading the paper A Contextual-Bandit Approach to Personalized News Article Recommendation, where it refers to $\epsilon$-greedy (disjoint) algorithm. I suspect, that it is just a version of a K-...
d56's user avatar
  • 243
1 vote
0 answers
58 views

Multi-armed Bandit in optimization on graph edges selection

I have the problem, which I described below. I wonder if there exists a class of multi-armed bandit approaches that is related to it. I am working on computer networking optimization. In the simplest ...
Ramon's user avatar
  • 21
1 vote
0 answers
102 views

Which RL algorithm would be suitable for this multi-dimensional and continuous action space?

Is there an RL approach/algorithm that would be suited for the following kind of problem? There is a continuous action space with an action value $A_{a,t}$ for each action dimension $a$. The ...
shimeji42's user avatar
  • 111
1 vote
0 answers
242 views

UCB-like algorithms: how do you compute the exploration bonus?

My question concerns Stochastic Combinatorial Multiarmed Bandits. More specifically, the algorithm called CombUCB1 presented in this paper. It is a UCB-like algorithm. Essentially, in each round of ...
Adam's user avatar
  • 11
1 vote
0 answers
95 views

How do I learn the value function for a POMDP with a single-step horizon (bandit)?

Consider a POMDP with a finite number of environment states, $|\mathcal{S}| = N$, but the number of belief states is uncountably infinite. The belief state space is the convex hull of an $N$ simplex. ...
jdizzle's user avatar
  • 61
1 vote
0 answers
43 views

Multi-armed bandits: reducing stochastic multi-armed bandits to bernoulli bandits

Agrawal and Goyal (http://proceedings.mlr.press/v23/agrawal12/agrawal12.pdf page 3) discussed how we can extend Thompson sampling for bernoulli bandits to Thompson sampling for stochastic bandits in ...
Felix P.'s user avatar
  • 297
1 vote
0 answers
805 views

Understanding GLIE conditions for epsilon greedy approach

I was going through this course on reinforcement learning (the course has two lecture videos and corresponding slides) and I had a doubt. On slide 18 of this pdf, it states following condition for an ...
Mahesha999's user avatar
1 vote
0 answers
32 views

Customized food for persons based on their profile using Reinforcement learning

I am newbie to Reinforcement Learning, this is my idea - Agent(food provider) has to select a food based on the environment(based on the user profile). Here the reward will be given to the agent based ...
Bala u1's user avatar
  • 11
1 vote
0 answers
73 views

How do I determine the optimal policy in a bandit problem with missing contexts?

Suppose I learn an optimal policy $\pi(a|c)$ for a contextual multi-armed bandit problem, where the context $c$ is a composite of multiple context variables $c = c_1, c_2, c_3$. For example, the ...
user31663's user avatar
1 vote
0 answers
34 views

Name of a multiarmed bandit with only some levers available

In order to model a card game, as an exercise, I was thinking of an elementary setting as a multiarmed bandit, each lever being the distribution of expected rewards of a specific card. But, of course,...
arivero's user avatar
  • 51
0 votes
0 answers
11 views

What is the equation to calculate discounted reward of an action in a multi-state multi-armed bandit?

I have a multi-armed bandit environment and I would like to calculate the accuracy of a Q-value based model with discounted rewards. For a multi-armed bandit (MAB) with a single state I reasoned out ...
foreverska's user avatar
  • 1,288
0 votes
1 answer
31 views

Can we use MAB in problems that reset after some time?

I have this scheduling problem. There are $n$ jobs, one machine and $T$ time slots. To be satisfied, each job $i=1,\ldots,n$ must receive at least the quantity $v_i$ from the machine. The machine can ...
zdm's user avatar
  • 301
0 votes
0 answers
25 views

Can RL solve scheduling problems with unknown function

I have the following scheduling problem. There are $n$ tasks and $m>n$ machines. Each task $i$ has a requirement $t_i$ that should be guaranteed. Any task can be scheduled on at least one machine ...
Jika's user avatar
  • 101
0 votes
0 answers
15 views

Do Bernoulli bandits need a different treatment if the rewards are sparse?

I have a problem where, effectively, my slot machines have very low payout probability (on the order of 1% for the "best" slot machines) and my goal is to minimize the number of actions to ...
Alexander Soare's user avatar
0 votes
0 answers
31 views

The impact of allowing the reward to be negative in a contextual bandit problem

I posted this question on stats.stackexchange.com 3 days ago. But there has been no answer. I would like to try my luck here. It seems the contextual bandits problems, as shown in these two papers ...
Hans's user avatar
  • 101
0 votes
0 answers
20 views

Finding an optimal action score function for Multi-Armed Bandit Problem

Considering a multi-armed bandit problem where there are : ...
MohammadAli Zeraatkar's user avatar
0 votes
1 answer
76 views

What reinforcement learning algorithm should I use for the following problem?

Environment I have a static timeseries environment meaning the environment is the same. This problem is a multi armed bandit problem. Time t0 t1 t2 State s0 s1 s2 Score 10 0.1 0.2 Class 1 0 0 ...
adamwest's user avatar
0 votes
0 answers
52 views

How to model the probability of click-through in bandit problem?

Consider a bandit problem in which we want to maximize the probability of click-through based on bid values ($b$ is the value of bid and $\Pr(b)$ shows the probability that a customer clicks on a link ...
Amin's user avatar
  • 481
0 votes
0 answers
55 views

Is there a variant of Thompson Sampling that works with variable bandits?

Does there exist a variant of TS, such that, while computing the returns of multi-armed bandits, we have the possibility of introducing an extra bandit? For instance, while we are applying TS to 3 ...
desert_ranger's user avatar
0 votes
0 answers
77 views

Are Best-Arm Bandits considered to be reinforcement learning?

Multi Armed Bandits (MABs) are a broad field of research pursuing different streams. In addition to the common objective of maximizing the cumulative reward, there are also so-called Best-Arm (...
D. B.'s user avatar
  • 101
0 votes
0 answers
98 views

Which algorithms work in a non-stationary stochastic environment?

Currently, I am reading into the Multi-Armed-Bandit problem and found the special case of non-stationary (environment and its attributes, like the reward distribution, change over time) stochastic ...
paperplan3's user avatar