Lets take an ad recommendation problem for 1 slot. Feedback is click/no click. I can solve this by contextual bandits. But I can also introduce exploration in supervised learning, I learn my model from collected data every k hours.

What can contextual bandits give me in this example which supervised learning + exploration cannot do?

  • $\begingroup$ Hello. Can you please clarify what you mean by "I can also introduce exploration in supervised learning"? What's your idea more specifically? $\endgroup$
    – nbro
    Jun 29 at 12:37
  • $\begingroup$ I will learn a supervised model from the logging policy/model 's data and I will use it in a stochastic way which will promote exploration. If I do this iteratively(i.e. learn model with IPW supervised loss and collect data with exploration), i hope i can achieve what a batch learning with bandit feedback setup will also achieve. I will use inverse propensity weights in bandits and supervised learning loss as well. then my question is that for this specific problem is, will bandit give me anything i cannot achieve with a IPW supervised loss. $\endgroup$
    – dksahuji
    Jun 30 at 2:52
  • $\begingroup$ the whole point is working in a practical setup and trying to learn from partial observations i.e. in recommendation you dont have data for what you dont recommend. learned stuff from this ( cs.cornell.edu/courses/cs7792/2018fa ) . batch learning with bandit feedback seems exciting to me. But how do i convince myself that for simple problem as explained in the question exploration plus inverse propensity weighted supervised loss is equal to a batch learned bandit. IPW supervised loss will obviously require less iterations over data. $\endgroup$
    – dksahuji
    Jun 30 at 2:59
  • $\begingroup$ In the example in question description, i can have a softmax over candidates for that slot. exploration will let me try different candidates and hence collecting data by exploration. $\endgroup$
    – dksahuji
    Jun 30 at 3:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.