Questions tagged [contextual-bandit]

For questions about the contextual bandit problem, which is a generalization of the (context-free) multi-armed bandit problem, where there is more than one situation (or state) and the optimal action to take in one state may be different than the optimal action to take in another state, but where the actions do not affect states (as e.g. in the reinforcement learning problem), but only the rewards.

Filter by
Sorted by
Tagged with
1
vote
0answers
40 views
+50

How do I learn the value function for a POMDP with a single-step horizon (bandit)?

Consider a single-step POMDP, where an agent receives a belief state $b(s)$ over some fixed set of contexts $s$, selects a single action, and receives a single reward, before the episode ends. I ...
1
vote
1answer
277 views

How can I incorporate domain knowledge to choose actions in the case of large action spaces in multi-armed bandits?

Suppose one is using a multi-armed bandit, and one has relatively few "pulls" (i.e. timesteps) relative to the action set. For example, maybe there are 200 timesteps and 100 possible actions....
4
votes
1answer
69 views

Can you convert a MDP problem to a Contextual Multi-Arm Bandits problem?

I'm trying to get a better understanding of Multi-Arm Bandits, Contextual Multi-Arm Bandits and Markov Decision Process. Basically, Multi-Arm Bandits is a special case of Contextual Multi-Arm Bandits ...
2
votes
0answers
24 views

Is there a UCB type algorithm for linear stochastic bandit with lasso regression?

Why is there no upper confidence bound algorithm for linear stochastic bandits that uses lasso regression in the case that the regression parameters are sparse in the features? In particular, I don't ...
1
vote
1answer
256 views

Can I apply DQN or policy gradient algorithms in the contextual bandit setting?

I have a problem which I believe can be described as a contextual bandit. More specifically, in each round, I observe a context from the environment consisting of five continuous features, and, ...
2
votes
2answers
91 views

Are bandits considered an RL approach?

If a research paper uses multi-armed bandits (either in their standard or contextual form) to solve a particular task, can we say that they solved this task using a reinforcement learning approach? Or ...
1
vote
0answers
36 views

How do I determine the optimal policy in a bandit problem with missing contexts?

Suppose I learn an optimal policy $\pi(a|c)$ for a contextual multi-armed bandit problem, where the context $c$ is a composite of multiple context variables $c = c_1, c_2, c_3$. For example, the ...
2
votes
1answer
439 views

How to implement a contextual reinforcement learning model?

In a reinforcement learning model, states depend on the previous actions chosen. In the case in which some of the states -but not all- are fully independent of the actions -but still obviously ...
1
vote
0answers
30 views

Name of a multiarmed bandit with only some levers available

In order to model a card game, as an exercise, I was thinking of an elementary setting as a multiarmed bandit, each lever being the distribution of expected rewards of a specific card. But, of course,...