0
$\begingroup$

I was learning ML, and I learnt a new section called, Reinforcement Learning. After some research on web, I found that it is a trial and error technique by which the agent learns from the environment by some interactions (actions), and it get rewarded if the action is accurate.

I wonder why it is better than A/B testing ? We can simply perform A/B testing between the variants of any product and can calculate the Interaction Rate, and can choose which one of the variant is better.

  1. What is the difference between A/B testing and Reinforcement Learning?

  2. Why RL is better than A/B testing?

$\endgroup$
1
  • $\begingroup$ Please, next time, put your specific question in the title. Thanks. $\endgroup$
    – nbro
    Commented Jul 26, 2023 at 10:10

1 Answer 1

2
$\begingroup$

What is the difference between A/B testing and Reinforcement Learning?

A/B testing can be thought of as a very basic subset of reinforcement learning. A/B testing is usually more closely related to context-free bandits (also a kind of subset of reinforcement learning), because it typically does not consider anything stateful about user interactions (e.g. what site the user arrived from, and demographic data from user profile), or any time domain changes (e.g. what has been presented and clicked/not clicked before), although both are possible whilst still following a simple "collect data, predict best choice later" design.

The main differences are:

  • Training of simple A/B testing is batched, whilst reinforcement learning, and bandit learning algorithms are online.
  • That means with reinforcement learning and bandit algorithms, decisions can be made in real time based on knowledge so far. You can choose, by tuning the algorithm, to optimise for better results during learning, reducing the risk of losing revenue from possible bad choices.
  • Reinforcement learning (unlike A/B testing or bandit algorithms) can attempt to optimise longer-term goals, if there is some kind of state that can be tracked, and that is influenced by decisions that have been made. So for example you can optimise for overall revenue rather than short-term results from each site visit independently.
  • Outside of promoting user choices on the web, and other "single interaction, single choice" scenarios, reinforcement learning is a superior theoretical model for agents which play games, control robots or manage any long term interactive system. RL solves problems involving sequential action choice, whilst A/B testing cannot really do that as a framework.

Why RL is better than A/B testing?

It isn't inherently better at all things. A/B testing may be an appropriate model and system design for some scenarios.

However, RL is the more general purpose learning system, whilst A/B testing is a simpler tool that is more specific.

Usually the "upgrade" from simple A/B testing would be to look into context-free or contextual bandits. These can improve revenue in advertising or product promotion scenarios due to the online learning factor. These improvements occur independently of whatever batch learning timing you may have set for offline training of A/B testing, and can be left on continuously in case something changes after the initial trial. Bandit algorithms add complexity, but are otherwise a drop in replacement for existing A/B systems. In that sense they can be viewed as better than simple A/B testing.

$\endgroup$
3
  • $\begingroup$ A/B testing takes more time because it is trained in offline before deployment and Reinforcement Learning techniques (e.g, Upper Confidence Bound) can be deployed directly, because it is trained by trial and error method , right ? $\endgroup$ Commented Jul 25, 2023 at 9:51
  • $\begingroup$ But it A/B testing is very simple because we just have to calculate the no. of votes for each variations and can deploy them afterwards $\endgroup$ Commented Jul 25, 2023 at 9:52
  • 1
    $\begingroup$ @mainakmukherjee - it is not just "takes more time". This time has often cost you money directly because one or more of the choices is not optimal, and A/B testing won't allow you to adjust the ratios in response to early stages of learning. In addition, if you train/deploy and then consider all trials done, you may miss out on changes to preferences for any reason. You can adapt A/B testing for those things (check early results, always have a 10% chance of showing worst afterwards and monitor routinely), but the more you do so, the more you have effectively changed A/B testing into RL. $\endgroup$ Commented Jul 25, 2023 at 10:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .