# How to model the probability of click-through in bandit problem?

Consider a bandit problem in which we want to maximize the probability of click-through based on bid values ($$b$$ is the value of bid and $$\Pr(b)$$ shows the probability that a customer clicks on a link given bid $$b$$). I am wondering what the possible modeling approaches are? If we consider a discrete set of bids ($$b\in \{b_1,b_2,\cdots,b_k \}$$), it is possible to model the probability as a Bernoulli distribution for each bid $$\Pr(b_k)=p_k$$ (with Beta or Dirichlet prior, e.g., $$p_k \sim Beta(\alpha_k,\beta_k)$$ ). Another case is model as a Logistic bandit in which it is possible to extend it to continuous values for bids, $$\Pr(b_k)=\frac{1}{1+\exp\left(a_0 + a_1 b_k \right)}$$.

I am wondering what other approaches are possible? Can we model it in form $$\Pr(b_k)=\beta_0 + \beta_1 b_k$$ with Normal prior on $$\beta_0$$ and $$\beta_1$$? If yes, what should we do if $$\beta$$ is estimated such that the probability exceeds 1?

• Can you define more precisely (i.e. mathematically) what you mean by "click-through"? I know that the click-through rate may depend on the bid that you pay, but why do you think it only depends on the bid? Moreover, can you explain why you want to model is as a bandit problem (at least according to the title), but then you're wondering if you can model is as a logistic model? What is your main question exactly? Please, highlight that, because otherwise people may not know what you're really asking, given that you seem to be asking many questions, actually.
– nbro
Jun 17, 2022 at 22:07
• If we consider the values of bid to be discrete values, then we consider each value as an arm. Because the reward is in form of probability, I have considered logistic function as the output to make sure that it is between 0 and 1. There are some Logistic Bandit papers and I have used their idea.
– Amin
Jun 18, 2022 at 5:44
• I am used to bandit problems where the task is to learn quickly, or with minimal regret, through trial and error attempts on action choice, with a goal of obtaining the maximum reward per action. It is not clear whether this is your task, or what the action and reward is here. If the action is to be bid choice, and the reward a constant on obtaining a click-through, then most of your models imply maximising the bid directly and no exploration would be required. However that feels wrong, and I think you are missing some important details of your setting. Please link a description Jun 18, 2022 at 7:36