SLIDE 1
TEXT AND TEXT AND AUTOMATED BIASES AUTOMATED BIASES NATURAL - - PowerPoint PPT Presentation
TEXT AND TEXT AND AUTOMATED BIASES AUTOMATED BIASES NATURAL - - PowerPoint PPT Presentation
TEXT AND TEXT AND AUTOMATED BIASES AUTOMATED BIASES NATURAL LANGUAGES ARE THE NATURAL LANGUAGES ARE THE BASE HUMAN COMMUNICATION BASE HUMAN COMMUNICATION we learn from books of all kinds about complex topics and keep ourself updated
SLIDE 2
SLIDE 3
NATURAL LANGUAGES ARE THE NATURAL LANGUAGES ARE THE BASE HUMAN COMMUNICATION BASE HUMAN COMMUNICATION
we learn from books of all kinds about complex topics and keep ourself updated
SLIDE 4
USEFULL APPLICATIONS USEFULL APPLICATIONS
structure big amounts of text (by topics or certain words) understand the meaning of text voice recognition text generation (summaries, q&a systems)
SLIDE 5
COMMON TASKS COMMON TASKS
AllenNLP demos Spacy demos
SLIDE 6
HOW DO WE MAKE COMPUTERS HOW DO WE MAKE COMPUTERS TRY TO UNDERSTAND TRY TO UNDERSTAND LANGUAGE? LANGUAGE?
The langauge of each person is different Language is ambigious Language requires contextual information it's constantly evolving
SLIDE 7
APPROACHES IN THE PAST APPROACHES IN THE PAST
- 1. Rule based
- 2. probabilistic models and linear
classifiers.
- 3. deep learning
SLIDE 8
DEEP LEARNING DEEP LEARNING
SLIDE 9
SLIDE 10
SLIDE 11
SLIDE 12
HOW TO DEAL WITH SEQUENCES HOW TO DEAL WITH SEQUENCES
SLIDE 13
SLIDE 14
DEEP LEARNING FOR NLP DEEP LEARNING FOR NLP
from symbolic representations to tensors/vectors and embeddings how to represent words in the input layer?
SLIDE 15
ONE HOT ENCODING ONE HOT ENCODING
scales bad no relationship between words no context/semantic information
word : n dimensions (for dictionary size) car : 1 0 0 ... 0 dog : 0 1 0 ... 0 cat : 0 0 1 ... 0 apple : 0 0 0 ... 1
SLIDE 16
WORD EMBEDDINGS WORD EMBEDDINGS
much less dimensions then words in the dictionary relationship between words build from training language models
SLIDE 17
TRAINING WORD VECTORS TRAINING WORD VECTORS (WORD EMBEDDINGS) (WORD EMBEDDINGS)
SLIDE 18
SLIDE 19
SLIDE 20
SLIDE 21
LANGUAGE MODELS LANGUAGE MODELS
Predicting the next character / word in a sequence
SLIDE 22
SLIDE 23
The Unreasonable Effectiveness of Recurrent Neural Networks
SLIDE 24
SLIDE 25
SLIDE 26
SLIDE 27
WORD ASSOCIATIONS WORD ASSOCIATIONS
SLIDE 28
SLIDE 29
SLIDE 30
SLIDE 31
SLIDE 32
Demo time
SLIDE 33
SUMMARY SUMMARY
Language models builds Word Embeddings Word Embeddings are Word representations in tense spaces The contain semantical information about a word Association are reflected in the relationships of words
SLIDE 34
SUMMARY SUMMARY
Language models builds Word Embeddings Word Embeddings are Word representations in tense spaces The contain semantical information about a word Association are reflected in the relationships of words There are problematic associations
SLIDE 35
SLIDE 36
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
SLIDE 37
Word Embedding Association Test Semantics derived automatically from language corpora necessarily contain human biases
SLIDE 38
Biases are consolidated Historical bias underepresented group Are Emily and Greg More Employable than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination
SLIDE 39
QUESTIONS TO ASK QUESTIONS TO ASK
Who build the model From what dataset was it build Where is the model used?
SLIDE 40
REAL WORLD APPLICATIONS AND REAL WORLD APPLICATIONS AND THERE PROBLEMS THERE PROBLEMS
GOOGLE TRANSLATE GOOGLE TRANSLATE
SLIDE 41
SLIDE 42
Google Translate Keeps Spitting Out Creepy Religious Prophecies
SLIDE 43
CHATBOTS CHATBOTS
SLIDE 44
Can virtual humans be more engaging than real ones?
SLIDE 45
MICROSOFTS CHATBOT TAY MICROSOFTS CHATBOT TAY
SLIDE 46
WOEBOT WOEBOT
SLIDE 47
SLIDE 48
HOW EXTREME BIAS BECOMES WHEN HOW EXTREME BIAS BECOMES WHEN FED WITH BAD DATA FED WITH BAD DATA
SLIDE 49
Norman A.I
SLIDE 50
Bias is identical to meaning, and it is impossible to employ language meaningfully without incorporating human bias.
SLIDE 51
THANK YOU THANK YOU Get in touch transfluxus@posteo.de twitter.com/ramin__