# How is it that AI can become biased, and what are the proposals to mitigate this?

This is not meant to be negative or a joke but rather looking for a productive solution on AI development, engineering and its impact on human life:

Lately with my Google searches, the AI model keeps auto filling the ending of my searches with:

“...in Vietnamese”

And

“...in a Vietnamese home”

The issue is I have never searched for that but because of my last name the model is creating this context.

The other issue is that I’m a halfy and my dad is actually third generation, I grew up mainstream American and don’t even speak Vietnamese. I’m not even sure what a Vietnamese home means.

My buddy in a similar situation of South Asian and noticed the same exact thing more so with YouTube recommended videos.

We already have enough issues in the US with racism, projections of who others expect us to be based on any number of things, stereotyping and putting people in boxes to limit them - I truly believe AI is adding to the problem, not helping.

How can we fix this. Moreover, how can we use AI to bring out peoples true self, talents and empower and free them them to create their life how they like ?

There is huge potential here to harness AI in ways that can bring us more freedom, joy and beauty so people can be the whole of themselves and with who they really are. Then meet peoples needs, wishes, dreams and hope. Given them shoulders to stand on to create their reality, not live someone else's projection of themselves.

• The real crime here is Google shadily collects all your personal info. – DuttaA Feb 15 '19 at 19:59
• @DuttA: Creators of NLP and face recognition algorithms do recognise unwanted bias creeping into their models. These models reflect the biases of the media that they consume, statistically. For instance, if you do word "math" you end up finding the female analog to a male doctor is a nurse (and many other examples where NLP model subtly demotes female roles). There are things that can be done about it at the model-building end, that may also apply here. – Neil Slater Feb 16 '19 at 13:37
• @NeilSlater IMO if we remove the statistical nature of ML then it is not ML anymore. But here I have told the OP to make sure there are no other ways of bias creeping in like 1.) Search history of family members. 2.) Google reading emails. 3.) Google location tracking with the people you are hanging out with and then recommending based on their family members. Basically I think Google is very good at connecting dots rather than ML approach as it'll require continuous intensive training of huge amts of data. (It's a speculation tho) – DuttaA Feb 16 '19 at 13:44
• @DuttaA: You can remove unwanted bias systematically from a statistical model if you also have a statistical model of the bias. For instance any NLP system that associates non-gendered words (like "doctor") in skewed ways to gender pronouns can have a loss function adjusted to prevent this effect, assuming it is unwanted for the purpose of the model. This can be done, and have seen in a training course - may write an answer about this – Neil Slater Feb 16 '19 at 13:52

Lately with my Google searches, the AI model keeps auto filling the ending of my searches with:

“...in Vietnamese”

I can see how this would be annoying.

I don't think Google's auto-complete algorithm and training data is publicly available. Also it changes frequently as they work to improve the service. As such, it is hard to tell what exactly is leading it to come up with this less-than-useful suggestion.

The whole thing is based around statistical inference. At no point does any machine "know" what Vietnamese - or in fact any of the words in your query - actually means. This is a weakness of pretty much all core NLP work in AI, and is called the grounding problem. It is why, for instance, that samples of computer generated text produce such surreal and comic material. The rules of grammar are followed, but semantics and longer term coherence are a mess.

Commercial chatbot systems work around this with a lot of bespoke coding around some subject area, such as booking tickets, shopping etc. These smaller domains are possible for human developers to "police", connecting them back to reality, and avoiding the open-ended nature of the whole of human language. Search engine text autocomplete however, cannot realistically use this approach.

• Wait it out. The service will improve. Whatever language use statistics are at work here are likely change over time. Your own normal use of the system without using the suggestions will be part of that data stream of corrections.

• Send a complaint to Google. Someone, somewhere in Google will care about these results, and view them as errors to be fixed.

Neither of these approaches guarantee results in any time frame sadly.

We already have enough issues in the US with racism, projections of who others expect us to be based on any number of things, stereotyping and putting people in boxes to limit them - I truly believe AI is adding to the problem, not helping.

You are not alone in having these worries. The statistics-driven nature of machine learning algorithms and use of "big data" to train them means that machines are exposing bias and prejudice that are long buried in our language. These biases are picked up by machinery then used by companies that don't necessarily want to reflect those attitudes.

A similar example occurs in natural language processing models with word embeddings. A very interesting feature of LSTM neural networks that learn statistical language models is that you can look at word embeddings, mathematical representations of words, and do "word math":

$$W(king) - W(man) + W(woman) \approx W(queen)$$

$$W(he) - W(male) + W(female) \approx W(she)$$

This is very cool, and implies that the learned embeddings really are capturing semantics up to some depth. However, the same model can produce results like this:

$$W(doctor) - W(male) + W(female) \approx W(nurse)$$

This doesn't reflect modern sensibilities of gender equality. There is obviously a deep set reason for this, as it has appeared from non-prejudiced statistical analysis of billions of words of text from all sorts of sources. Regardless of this though, engineers responsible for these systems would prefer that their models did not have these flaws.

How can we fix this. Moreover, how can we use AI to bring out peoples true self, talents and empower and free them them to create their life how they like ?

Primarily by recognising that statistical ML and AI doesn't inherently have prejudice or any agenda at all. It is reflecting back ugliness already in the world. The root problem is to fix people (beyond scope of this answer, if I had solid ideas about this I would not be working in software engineering, but in something more people-focussed).

However, we can remove some of the unwanted bias from AI systems. Broadly the steps toward this go:

• Recognise that a particular AI system has captured and is using unwanted gender, racial, religious etc bias.

• Reach a consensus about how an unbiased model should behave. It must still be useful for purpose.

• Add the desired model behaviour into the training and assessment routines of the AI.

For instance in your case, there are possibly some users of Google's system who would prefer to read articles in Vietnamese, or have English translated into Vietnamese, and are finding it awkward that the default assumption is that everything should be presented in English. These users don't necessarily need to use the search text for this, but presumably are for some reason. A reasonable approach is to figure out how their needs could be met without spamming "in Vietnamese" on the end of every autocomplete suggestion, and perhaps in general move suggestions to localise searches by cultural differences out of autocomplete into a different part of the system.

For the case of gender bias in NLP systems, Andrew Ng's Coursera course on RNNs shows how this can be achieved using the embeddings themselves. Essentially it can be done by identifying a bias direction from a set of words (e.g. "he/she", "male/female"), and removing deviations in that direction for most other words, preserving it only for words where it is inherently OK to reflect the differences (such as "king" and "queen" for gender bias).

Each case of unwanted bias though needs to be discovered by people and oversight of this as a political and social issue, not primarily a technical one.

• Does Google have enough computational power to train on such huge amounts of data regularly to build a NLP model? – DuttaA Feb 17 '19 at 15:27
• @DuttaA: Yes I suspect they do have that level of compute power and more, and routinely re-train their main models. However, the decision becomes a more complex cost/benefit analysis, and does not only involve time and electricity, but development work. The specific case of gender bias adjustment does not require re-training as such but is a post-processing step. Actual fixes will vary a lot. – Neil Slater Feb 17 '19 at 15:46
• "Statistical ML doesn't have any agenda": check. "Statistical ML doesn't have any prejudice other than that in the data": nope. There are a few papers which prove that, except for very specific cases and even if the data have been correctly collected, a classifier cannot be at the same time calibrated and fair, and if fair under a certain fairness metric, it will be unfair under another one. See papers.nips.cc/paper/7151-on-fairness-and-calibration.pdf for one example. – DeltaIV Feb 18 '19 at 8:12
• @DeltaIV: Fairness and prejudice are not always opposites. The paper is strongly related to this question, but not talking about quite the same thing as my answer when I refer systemic bias in NLP systems. I will try to incorporate it into the answer though . . . – Neil Slater Feb 18 '19 at 9:43
• @neil-slater: amazing answer, thank you. The root issue of fixing humans is eye opening. Sure the models have no agenda but either do bullets. This means there is huge potential here to harness AI in ways that can bring us more freedom, joy and beauty so people can be the whole of themselves and with who they really are. Then meet peoples needs, wishes, dreams and hope. Given them shoulders to stand on to create their reality, not live someone else's projection of themselves. – P.S. Feb 24 '19 at 23:28

Another fallacy that appears common to most search engines is that anything a person searches on is an aspect of their own identity. I once searched on walk-in tubs for a very elderly relative, and was followed all over the web by ads for aids for the infirm elderly. Users who recognize that Google uses their searches to build their profile can alter their searches accordingly. It's also fun to mess with Google's model. Try searching on "dragon images" and see how fast Google and advertisers decide you are a teenage female. Have fun with it. Do your best to turn Google's model of you into self-contradictory garbage.

• Yeah good point, wtf? Is there an AI app that can just send out random queries throughout the day? – P.S. Mar 16 '19 at 19:22

The key I think is teaching the algorythm by providing better data. The only thing an AI can use is the data available for itself. Figuring out whatever it can is not bias, as it's based on objective facts.

If it knows 98% of Nguyens are interested in X, knowing nothing else about you personally, showing you X might be good. If you consistently click on downvote/not interested, etc. buttons on the site, your personal data will override the default, and you won't see X anymore.

As a user you could give better reviews for better results, and as a developer you can give better ways to get this: by logging what you click on, search, and showing "not interested/interested/upvote/downvote/like" etc. buttons.

Note that I'm using youtube from different, unlinked machines/browsers, and I get different suggestions from all of these, probably because I've trained the AI with different data.

You can also use services with less intrusive data collection, e.g. duckduckgo, bitchute, etc.

• thanks, good points. would love to see the developers add more feedbac/training hooks for users. – P.S. Feb 24 '19 at 23:30