From the article Dangerous Feedback Loops in ML

Let’s say our model has leads from Facebook, Google, and Bing. If our first model decides that the probability of conversion is 3%, 5%, and 1% from these given sources, and we have finite amount of callbacks we can make, we will only callback the 5% probability. Now fast forward two months. The second model finds these probabilities are now: 0.5%, 8.5%, and 0%. What happened?

Because we started only calling Google leads back, we increased our chances of converting these leads, and likewise, because we stopped calling Facebook and Bing leads, these leads never converted because we never called them. This is an example of a real world feedback loop

How can we solve this problem?

  • 1
    $\begingroup$ This question needs to be clarified. We should not need to read that linked article in order to understand what is going on, i.e. provide some context or summarise what is written in that article. $\endgroup$ – nbro Jul 13 at 11:25

I read the article you linked, and what you are missing are that the given conversion probabilities are assessed pre-callback - i.e. they include an assessment of whether you will even call them back or not. So of course the probabilities change if you change your behaviour. The writer of the article has created a bit of a straw man argument by defining a model and decision process that don't go well together. They should have used a model that predicted conversion rates after callback, then it could be used as they wanted.

So the sales example is pretty easy to solve. Make the model accept the lead source as a feature, and predict the conversion rate after callback. That will give you the "action value" of choosing to callback, which is much more useful value to have if you are deciding whether or not to callback.

To cover the possibility that probabilities change over time, you have to be willing to test that hypothesis by calling back at least some leads that the model predicts have a low probability of success, in order to update the model. This is related to exploration vs exploitation trade off in bandit problems or in reinforcement learning. The callback optimisation problem looks a lot like a bandit problem in fact.

Many problems of choosing how to act optimally, based on an updating statistical model of results, can be re-framed successfully as bandit problems or reinforcement learning problems. This is one way to try and address the feedback loops issue, because these include theory around decision making. It is not a magic fix for all these problems, but contextual bandits are very strong in advertising strategies that are very similar to the example you give.

To be able to address feedback loops like this well does require that source data is unbiased, so that adding conditional actions to your model helps obtain better ground truth. So the other examples that the linked article gives - recidivism and recruitment - are much harder to address, because biased data is inserted as ground truth even if you add the conditions (e.g. for exploring and tracking results in a statistically neutral way). For some, adding these conditions may be enough to help, it will make the model accurate enough that it starts to address bias. For others, there is no really good solution other than to be aware of weakness of the model.

You can attempt to de-bias the model by using some abstract ideal. A common technique used in recruiting for instance is to remove features that might be causing bias: Literally remove race, sex and related determiners from the CV on the grounds that these should not be part of any fair model, regardless of whether or not a ML model could make of use them to predict results. This might accidentally throw out data that would improve accuracy too, but can be used to achieve a decision process that observers agree should be free of contentious soruces of bias.

Which features to remove or adjust is an open and subjective question. Unfortunately ML models can often determine protected characteristics from other data, and with "black box" ones such as neural networks it is not clear whether or not they are doing so. It's still an open question, and the main defence against biased models is basically awareness that it is a problem and that feeding all your data into a computer to get a statistical model does not in any way "purify" it or achieve some higher level of objective truth.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.