2
$\begingroup$

I'm developing a multi-armed bandit which learns the best information to display to persuade someone to donate to charity.

Suppose I have treatments A, B, C, D (which are each one paragraph of text). The bandit selects one treatment to show to a person. The person is given $1 and has to decide how much (if any) to donate, in increments of one cent. The donation decision is recorded and fed to the multi-armed bandit, who will then re-optimize before another person is shown a treatment selected by the bandit.

How should I program the bandit if my objective is to maximize total donations? For example, can I use Thompson sampling, and if a participant donates $0.80, I count that as 80 successes and 20 failures?

$\endgroup$
0
3
$\begingroup$

It does not matter to the bandit algorithm that rewards are quantised or fractional, or that they can vary. This is true for pretty much all bandit optimisation algorithms.

So just treat the $0.80 donation as a real valued reward of 0.8, that occurs on a single timestep.

Treating a single reward on a single timestep as if it were multiple rewards across multiple timesteps might cause problems, depending on which algorithm you were using, even if the reward averaged to the same value. For instance it may skew the maths away from theoretical assumptions used in upper confidence bound action selection or gradient-based solutions.


It occurs to me that your $1 experiment is supposed to be some kind of proxy for the real deployment, where donations will be sparse and may vary more. Whilst this may help a little, essentially you are just setting some kind of prior based on results from a test. The real bandit algorithm starts once the system is deployed into production, and could well have very different results when the goal is to collect real and larger donations.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.