Many of multi-armed bandit(MAB) algorithms are used when the total reward is the sum of all rewards. However, in RL, the discounted reward is mainly used. Why is the discounted reward not prevailing in MAB problem, and in what cases is this type of modeling valid and might be better?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
One of the reasons a discount factor is used, is to make sure the reward maximization is a well-defined problem and to make the sum of all rewards convergent.
In the MAB problem, the number of trials is typically finite owing to some sort of budget in the number of trials. Hence, this is less of problem. However, by all means discounts are still valid and helpful in cases where the analysis in asymptotic in the number of trials.
-
$\begingroup$ Thanks for your response. Except for convergence analysis, in what cases discounted MAB can be meaningful? Is it possible to have an optimal policy in the discounted case different from the undiscounted setting? $\endgroup$– AminCommented Jun 5, 2020 at 6:26