How Computer Algorithms Expose Our Hidden Biases And How To Fix - - PowerPoint PPT Presentation

How Computer Algorithms Expose Our Hidden Biases

And How To Fix Them

Victor Zimmermann

• LXIV. StuTS

Computational Linguistics Department Heidelberg University

The Shitstorm cometh.

What happened?

• lxiv. stuts | white man explains racism

1

The Netflix Artwork Controversy Why are the tabloids up in arms over Netflix adverts? Welcome to a stereotypical machine learning controversy.

• lxiv. stuts | white man explains racism

2

The algorithm

“If the artwork representing a title captures something compelling to you, then it acts as a gateway into that title and gives you some visual ‘evidence’ for why the title might be good for you.” [Cha+17]

Figure 1: Different artworks for romance and comedy viewers.

• lxiv. stuts | white man explains racism

3

Twitter outrage

• lxiv. stuts | white man explains racism

4

Netflix’ Response

“We don’t ask members for their race, gender or ethnicity so we cannot use this information to personalise their individual Netflix

• experience. The only information we use is a member’s viewing

history.” [Iqb18]

• lxiv. stuts | white man explains racism

5

Nobody expects the Patriarchy.

Sources of Bias

There are some obvious reasons for bias in machine learning:

• Your training data is bad.
• Your algorithm is bad.
• You are bad. And you should feel bad.
• lxiv. stuts | white man explains racism

6

Bad Training Data

Human Language

Spoiler: All human language is biased. Bias in not necessarily performance based. [Tan90][GMS98] Instead it can also be encoded in orthography, lexicography or grammar of a language.

• Asymmetrically marked gender (generic masculine, e.g. actor vs

actress)

• Quantity of gendered insults 1 [Sta77]
• Naming conventions (e.g. Chastity vs. Bob) [Swe13]

1Wikipedia lists 22 misogynistic and 5 misandric slurs.

• lxiv. stuts | white man explains racism

7

Word Embeddings

What are Word Embeddings? Condensed mathematical representations of collocations. [Mik+13]

CHICAGO – Former President Barack Obama campaigned in Chicago and northwest Indiana on Sunday, just days ahead of Tuesday’s midterm elec- tions. Obama spoke Sunday afternoon at a get-out-the-vote rally in Gary, Indiana, supporting Democrat U.S. Sen. Joe

• Donnelly. The rally ended at about 3

p.m. and then spoke a rally at ...

⇒

− − − − − → Obama(0.2, 0.6, ...) − − − − → speaks(0.1, 0.8, ...) − − − − − → Chicago(0.3, 0.2, ...) − − − → press(0.0, 0.5, ...) . . .

⇒ Now you can do maths with words!? − − → King − − − → Man + − − − − − → Woman = − − − − → Queen − − − → Berlin − − − − − − − → Germany + − − − − → France = − − − → Paris − − − − − − − − − → Programmer − − − → Man + − − − − − → Woman = − − − − − − − − → Homemaker − − − − − → Surgeon − − − → Man + − − − − − → Woman = − − − → Nurse

• lxiv. stuts | white man explains racism

8

Word Embeddings

What are Word Embeddings used for?

• Similarity Measures [Kus+15]
• Machine Translation [Zou+13]
• Sentence Classification [Kim14]
• Part-of-Speech-Tagging [SZ14][RRZ18]
• Dependency Parsing [CM14]
• Semantic Modelling [Fu+14]
• Coreference Resolution [Lee+17]

Basically the entire field of Computational Linguistics.

• lxiv. stuts | white man explains racism

9

Mathematical Sledgehammer

What if we just remove gender?

Figure 2: Mind = Blown

• lxiv. stuts | white man explains racism

10

Mathematical Sledgehammer

• Take “good” analogies,

e.g. man-woman, he-she, king-queen, etc.

• Extract some average “gender vector” from their embeddings.
• Substract this new vector from all other relations.
• Not applicable to most other kinds of bias.
• lxiv. stuts | white man explains racism

11

Mathematical Sledgehammer (in beautiful)

Word sets W, defining subsets D1, D2, ..., Dn ⊂ W, embedding {w ∈ Rd}w∈W, integer parameter k ≥ 1, with µi := ∑

w∈Di

w/|Di| being the means of the defining subsets. Bias subspace B consists of the first k rows of SVD(C), where C :=

n

∑

i=1

∑

w∈Di

(w − µi)T(w − µi)/|Di|. Words to neutralise N ∈ W, family of equality sets ε:= {E1, E2, ..., Em}, Ei ⊆ W, with reembedded words w ∈ N defined as w := (w − wB)/|w − wB| . For each set E ∈ ε, let µ := ∑

w∈E

w/|E| v := µ − µB For each w ∈ E, w := v + √ 1 − |v|2 wB − µB |wB − µB|

• lxiv. stuts | white man explains racism

12

Bad Algorithms

Google’s Image Recognition Controversy

Google automatically labels pictures according to their content. Problem: Their algorithm is bad.

Source: @jackyalcine on Twitter

• lxiv. stuts | white man explains racism

13

Google’s Image Recognition Controversy

Their solution:

Source: www.theverge.com (visited on 2018-11-06)

• lxiv. stuts | white man explains racism

14

No easy solutions.

Not one of these solutions is really good.

• Total avoidance of problem. [Iqb18]
• Limited applicability. [Bol+16]
• Exploitation of false classification. [BGO16]
• Introduction of even more priors and meta parameters. [Zha+17]
• lxiv. stuts | white man explains racism

15

Bad People

Facebook

Actual Quote from an actual Facebook Employee “We started out of a college dorm. I mean, c’mon, we’re Facebook. We never wanted to deal with this shit.” [Sha16]

• lxiv. stuts | white man explains racism

16

Facebook

Possible cause of this apathy: (Don’t quote me on this.)

• lxiv. stuts | white man explains racism

17

Help, my Chatbot joined the KKK!

Microsoft Tay

• lxiv. stuts | white man explains racism

18

Microsoft Tay

• lxiv. stuts | white man explains racism

19

Microsoft Tay

What can we learn from this?

• Tay is a chat bot.Tay is a chat bot.
• Tay is down with the kids?Tay is down with the kids?
• Tay learns from Twitter data.
• lxiv. stuts | white man explains racism

20

Microsoft Tay

The absolutely expected happens...

Source: www.theguardian.com (visited on 2018-11-19)

• lxiv. stuts | white man explains racism

21

What should you take away from this talk?

• Just because something uses “machine learning”

doesn’t mean it is unbiased.

• All language is implicitly prejudiced.
• Training data does make a difference.
• Diverse staff makes a difference.
• Testing your system makes a difference.
• lxiv. stuts | white man explains racism

22

What should you take away from this talk? Don’t listen to chat bots. They may act human.

• lxiv. stuts | white man explains racism

23

Appendix

Language Classification

Common language identification systems use extensive news corpora for training. + Big corpora in most languages. + Mostly unbiased “unbiased” texts.

• Written in main dialect.
• Privileged writing staff.

Problem: African American English is 20% less likely to be classified as English than Standard English. [BO17]

• lxiv. stuts | white man explains racism

Language Classification

Solution by Blodgett, Green, and O’Connor (2016):

• 1. Use US Census data und geolocated tweets to estimate race of

user,

• 2. Train classifier to identify “race” of a given tweet, based on high

AA tweets from first set. Result:

• Build new corpus from high AA tweets.
• (Find out that “Asian” captures all foreign languages and use that

fact for classification.)

• lxiv. stuts | white man explains racism

References

[Ang+16] Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. “Machine bias: There’s software used across the country to predict future criminals. and it’s biased against blacks”. In: ProPublica, May 23 (2016). [BGO16] Su Lin Blodgett, Lisa Green, and Brendan O’Connor. “Demographic dialectal variation in social media: A case study of African-American English”. In: arXiv preprint arXiv:1608.08868 (2016).

• lxiv. stuts | white man explains racism

References

[BO17] Su Lin Blodgett and Brendan O’Connor. “Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American English”. In: arXiv preprint arXiv:1707.00061 (2017). [Bol+16] Tolga Bolukbasi, Kai-wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. “Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings”. In: Nips (2016), pp. 1–9. [CBN17] Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. “Semantics derived automatically from language corpora contain human-like biases”. In: Science 356.6334 (2017),

• pp. 183–186. issn: 10959203. arXiv: 1608.07187.
• lxiv. stuts | white man explains racism

References

[Cha+17] Ashok Chandrashekar, Fernando Amat, Justin Basilico, and Tony Jebara. “Artwork Personalization at Netflix”. In: Netflix Techblog (Dec. 7, 2017). url: https: //medium.com/netflix-techblog/artwork- personalization-c589f074ad76 (visited on 11/05/2018). [CM14] Danqi Chen and Christopher Manning. “A fast and accurate dependency parser using neural networks”. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 2014, pp. 740–750. [Duh12] Charles Duhigg. “How companies learn your secrets”. In: The New York Times 16 (2012), p. 2012.

• lxiv. stuts | white man explains racism

References

[FD14] Manaal Faruqui and Chris Dyer. “Improving vector space word representations using multilingual correlation”. In: Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics. 2014, pp. 462–471. [Fu+14] Ruiji Fu, Jiang Guo, Bing Qin, Wanxiang Che, Haifeng Wang, and Ting Liu. “Learning semantic hierarchies via word embeddings”. In: Proceedings of the 52nd Annual Meeting

• f the Association for Computational Linguistics (Volume 1:

Long Papers). Vol. 1. 2014, pp. 1199–1209.

• lxiv. stuts | white man explains racism

References

[Gar+17] Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. “Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes”. In: 115.16 (2017). issn: 0027-8424. arXiv: 1711.08412. url: http://arxiv.org/abs/1711.08412. [GMS98] Anthony G Greenwald, Debbie E McGhee, and Jordan LK Schwartz. “Measuring individual differences in implicit cognition: the implicit association test.”. In: Journal

• f personality and social psychology 74.6 (1998), p. 1464.

[Gre95] Gregory Grefenstette. “Comparing Two Language Identification Schemes”. In: JADT. 1995.

• lxiv. stuts | white man explains racism

References

[Han+15] C Hansen, M Tosik, G Goossen, C Li, L Bayeva, F Berbain, and M Rotaru. “How to get the best word vectors for resume parsing”. In: SNN Adaptive Intelligence/Symposium: Machine Learning. 2015. [Iqb18] Nosheen Iqbal. “Film fans see red over Netflix ‘targeted’ posters for black viewers”. In: The Guardian (Oct. 20, 2018). url: https://www.theguardian.com/media/2018/

• ct/20/netflix-film-black-viewers-

personalised-marketing-target (visited on 11/05/2018).

• lxiv. stuts | white man explains racism

References

[Kil+17] Niki Kilbertus, Mateo Rojas-Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf. “Avoiding Discrimination through Causal Reasoning”. In: Nips (2017), pp. 1–11. issn: 10495258. arXiv: 1706.02744. url: http://arxiv.org/abs/1706.02744. [Kim14] Yoon Kim. “Convolutional neural networks for sentence classification”. In: arXiv preprint arXiv:1408.5882 (2014). [Kus+15] Matt Kusner, Yu Sun, Nicholas Kolkin, and Kilian Weinberger. “From word embeddings to document distances”. In: International Conference on Machine

• Learning. 2015, pp. 957–966.
• lxiv. stuts | white man explains racism

References

[Kus+17] Matt J. Kusner, Joshua R. Loftus, Chris Russell, and Ricardo Silva. “Counterfactual Fairness”. In: Nips (2017). issn: 10495258. arXiv: 1703.06856. url: http://arxiv.org/abs/1703.06856. [Lee+17] Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. “End-to-end neural coreference resolution”. In: arXiv preprint arXiv:1707.07045 (2017).

• lxiv. stuts | white man explains racism

References

[Meh+17] Rishabh Mehrotra, Ashton Anderson, Fernando Diaz, Amit Sharma, Hanna Wallach, and Emine Yilmaz. “Auditing Search Engines for Differential Satisfaction Across Demographics”. In: Proceedings of the 26th International Conference on World Wide Web Companion (2017),

• pp. 626–633. url: https:

//dl.acm.org/citation.cfm?id=3054197%7B%5C& %7DCFID=966931141%7B%5C&%7DCFTOKEN=80146118. [Mik+13] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. “Distributed representations of words and phrases and their compositionality”. In: Advances in neural information processing systems. 2013, pp. 3111–3119.

• lxiv. stuts | white man explains racism

References

[Nal+16] Eric Nalisnick, Bhaskar Mitra, Nick Craswell, and Rich Caruana. “Improving Document Ranking with Dual Word Embeddings”. In: Proceedings of the 25th International Conference Companion on World Wide Web. WWW ’16 Companion. Montreal, Quebec, Canada: International World Wide Web Conferences Steering Committee, 2016, pp. 83–84. isbn: 978-1-4503-4144-8. url: https://doi.org/10.1145/2872518.2889361. [RC11] Karen Ross and Cynthia Carter. “Women and news: A long and winding road”. In: Media, Culture & Society 33.8 (2011),

• pp. 1148–1165.
• lxiv. stuts | white man explains racism

References

[RRZ18] Ines Rehbein, Josef Ruppenhofer, and Victor Zimmermann. “A harmonised testsuite for POS tagging of German social media data”. In: (2018). [RVS17] Sebastian Ruder, Ivan Vulic, and Anders Søgaard. “A survey

• f cross-lingual embedding models”. In: CoRR,

abs/1706.04902 (2017).

• lxiv. stuts | white man explains racism

References

[Sha16] Aarti Shahani. “From Hate Speech To Fake News: The Content Crisis Facing Mark Zuckerberg”. In: NPR (Nov. 17, 2016). url: https: //www.npr.org/sections/alltechconsidered/ 2016/11/17/495827410/from-hate-speech-to- fake-news-the-content-crisis-facing-mark- zuckerberg?t=1542640881872 (visited on 11/19/2018). [SL17] Robert Speer and Joanna Lowry-Duda. “ConceptNet at SemEval-2017 Task 2: Extending Word Embeddings with Multilingual Relational Knowledge”. In: (2017). arXiv: 1704.03560. url: http://arxiv.org/abs/1704.03560.

• lxiv. stuts | white man explains racism

References

[Sta77] Julia Penelope Stanley. “Paradigmatic woman: The prostitute”. In: Papers in language variation (1977),

• pp. 303–321.

[Swe13] Latanya Sweeney. “Discrimination in online ad delivery”. In: Queue 11.3 (2013), p. 10. [SZ14] Cicero D Santos and Bianca Zadrozny. “Learning character-level representations for part-of-speech tagging”. In: Proceedings of the 31st International Conference on Machine Learning (ICML-14). 2014,

• pp. 1818–1826.

[Tan90] Dali Tan. “Sexism in the Chinese Language”. In: NWSA Journal 2.4 (1990), pp. 635–639.

• lxiv. stuts | white man explains racism

References

[Zha+17] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. “Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints”. In: (2017). arXiv: 1707.09457. url: http://arxiv.org/abs/1707.09457. [ZLM18] Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. “Mitigating Unwanted Biases with Adversarial Learning”. In: (2018). arXiv: 1801.07593. url: http://arxiv.org/abs/1801.07593.

• lxiv. stuts | white man explains racism

References

[Zou+13] Will Y Zou, Richard Socher, Daniel Cer, and Christopher D Manning. “Bilingual word embeddings for phrase-based machine translation”. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 2013, pp. 1393–1398.

• lxiv. stuts | white man explains racism