0
$\begingroup$

Currently, I am reading into the Multi-Armed-Bandit problem and found the special case of non-stationary (environment and its attributes, like the reward distribution, change over time) stochastic environments. Since this is an adversarial MAB problem, no context is available at the moment. I've read that $\epsilon$-greedy, Exp3 and the FPL algorithm work. But some tutorials like TensorFlow ones use LinUCB and other algorithms, which are never mentioned in any papers.

So, my question is, basically: which algorithms work on the non-stationary stochastic environments?

Improve this question
$\endgroup$
3
  • $\begingroup$ What do you mean by "LinUCB and other algorithms, which are never mentioned in any papers."? I've seen LinUCB being mentioned in several papers that I've come across on bandits. So, which papers do you have in mind? Do you mean papers related to non-stationary environments? Also, can you clarify this "Since this is an adversarial MAB problem, no context is available at the moment."? Are you referring to the "context" as in "contextual bandits"? I don't see the connection between being "an adversarial MAB problem" and not having access to "context" (whatever you mean by this). $\endgroup$
    – nbro
    Jan 23, 2022 at 18:09
  • $\begingroup$ Hi, yes I've been unclear in this points. By "LinUCB and other algorithms, which are never mentioned in any papers." I do mean papers related to non-stationary environments. And by "no context" I mean the base case of the MAB problem, so without any context like in contextual bandits. I just want to make the example simpler to understand it better. Thats why I am excluding the context. I hope this made it clearer. $\endgroup$
    – paperplan3
    Jan 24, 2022 at 10:21
  • $\begingroup$ Thanks for the clarifications. Can you please include this info directly into your post? I would also recommend that you provide the link to 1-2 papers that you came across on non-stationary environments. $\endgroup$
    – nbro
    Jan 24, 2022 at 11:18

0

Reset to default

You must log in to answer this question.