What is the difference between A/B testing and Reinforcement Learning?
A/B testing can be thought of as a very basic subset of reinforcement learning. A/B testing is usually more closely related to context-free bandits (also a kind of subset of reinforcement learning), because it typically does not consider anything stateful about user interactions (e.g. what site the user arrived from, and demographic data from user profile), or any time domain changes (e.g. what has been presented and clicked/not clicked before), although both are possible whilst still following a simple "collect data, predict best choice later" design.
The main differences are:
- Training of simple A/B testing is batched, whilst reinforcement learning, and bandit learning algorithms are online.
- That means with reinforcement learning and bandit algorithms, decisions can be made in real time based on knowledge so far. You can choose, by tuning the algorithm, to optimise for better results during learning, reducing the risk of losing revenue from possible bad choices.
- Reinforcement learning (unlike A/B testing or bandit algorithms) can attempt to optimise longer-term goals, if there is some kind of state that can be tracked, and that is influenced by decisions that have been made. So for example you can optimise for overall revenue rather than short-term results from each site visit independently.
- Outside of promoting user choices on the web, and other "single interaction, single choice" scenarios, reinforcement learning is a superior theoretical model for agents which play games, control robots or manage any long term interactive system. RL solves problems involving sequential action choice, whilst A/B testing cannot really do that as a framework.
Why RL is better than A/B testing?
It isn't inherently better at all things. A/B testing may be an appropriate model and system design for some scenarios.
However, RL is the more general purpose learning system, whilst A/B testing is a simpler tool that is more specific.
Usually the "upgrade" from simple A/B testing would be to look into context-free or contextual bandits. These can improve revenue in advertising or product promotion scenarios due to the online learning factor. These improvements occur independently of whatever batch learning timing you may have set for offline training of A/B testing, and can be left on continuously in case something changes after the initial trial. Bandit algorithms add complexity, but are otherwise a drop in replacement for existing A/B systems. In that sense they can be viewed as better than simple A/B testing.
