I recently read an article about how artificial intelligence replicates human stereotypes when applied to biased datasets.

What techniques exist to prevent bias in artificial intelligence systems?

  • $\begingroup$ I wasn't sure what tags to use for this question, so please edit it liberally. $\endgroup$ – user6698 Apr 17 '17 at 19:08
  • 3
    $\begingroup$ There was also the disaster in which Microsoft made a Twitter bot to emulate millennials and it learned/tweeted things that were, uh, less than desirable. $\endgroup$ – Ben N Apr 17 '17 at 19:23
  • $\begingroup$ I think the question is too broad, depends on what A.I. system you have in mind and the specific domain, as they exist today you would have to somehow program in human stereotypes as part of the learning set to identify and avoid them, in self learning ones you would have to recreate theory of mind which in itself is not fully understood. $\endgroup$ – Keno Apr 18 '17 at 20:18
  • $\begingroup$ @Keno I feel like your comment is a valid answer! $\endgroup$ – DukeZhou Apr 19 '17 at 16:31
  • 1
    $\begingroup$ To some extend the article has given an answer. It says, "The findings of the study do not come as a massive surprise, as machine learning systems are only able to interpret the data they are trained upon". So, a valid thing to do feed non-biased data to the ML algorithms. $\endgroup$ – Ugnes Apr 19 '17 at 20:00

It's important to note that ultimately, the statistical methods we currently use in ML research are just that: statistical methods. So when they show some "bad behaviour" it's not because of problems with the statistical methods, but with the data we give it. But if the data we give it are as "genuine and unfiltered" as it gets, then it probably shows something about us.

From a cognitive science perspective, its probably the case that the same heuristics and biases that creates stereotypes are also the ones that make us powerful agents(note the similarity between categories and stereotypes), so at least at this moment its unclear how we can segregate desired from undesired behaviour.

To combine the points mode above, it seems we can only either: 1. Remove "bad content" by curating the data by hand or by some metric that we don't know of yet 2. Accept that our methods will produce AI as "bad as we are" cause that's what we are, and let it operate under the knowledge that it might produce undesired behavior sometimes.

Unless we have some crazy new theory of mind that we can begin to analyze this in a more rigour manner, it seems like there is no clear cut solution.


Although the question is broad, the field of statistical psychology offers many methodologies to remove bias from datasets, or rather gather a dataset with minimal unknown biases. This will be the responsility of the programmer, an AI that learns from datasets will not be able to find bias in those datasets.

  • $\begingroup$ In other words, fudge the data so it gives the results that the researcher wants instead of the "real" results. It seems like this is the where the future of scientific research is heading. $\endgroup$ – Dunk Jul 10 '17 at 21:46
  • $\begingroup$ @Dunk Biased data is already fudged: otherwise there'd be no need to debias it. $\endgroup$ – Jeremy List Sep 25 '19 at 2:16
  • 1
    $\begingroup$ @JeremyList - Fair enough, the data could be biased and it is reasonable to 'debias' the data, as long as a concrete explanation justifying why the data is biased can be determined first. Otherwise, debiasing data just to give more desirable results is essentially falsifying the data to give the answers the researcher is looking for. All the computer is going to do is find patterns and the most significant relationships in the data. Just because someone doesn't like what the computer finds doesn't automatically mean the data is biased. $\endgroup$ – Dunk Sep 25 '19 at 13:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy