SLIDE 1
Practical Unsupervised Learning INFO/CS 4300, Spring 2016 Jack - - PowerPoint PPT Presentation
Practical Unsupervised Learning INFO/CS 4300, Spring 2016 Jack - - PowerPoint PPT Presentation
Practical Unsupervised Learning INFO/CS 4300, Spring 2016 Jack Hessel Unsupervised Learning is Cool! But how can we use this in our projects? 1. 2. But first, lets look at our dataset... Data Dimensionality! Probability of Lung Cancer
SLIDE 2
SLIDE 3
But how can we use this in our projects?
1. 2.
SLIDE 4
But first, let’s look at our dataset...
SLIDE 5
Data Dimensionality!
X X X X X X X X X
Cigarettes Consumed Per Day Probability of Lung Cancer Developing
SLIDE 6
Data Dimensionality!
X X X X X X X X X
Cigarettes Consumed Per Day Probability of Lung Cancer Developing
SLIDE 7
Data Dimensionality!
X X X X X X X X X
Cigarettes Consumed Per Day Probability of Lung Cancer Developing
SLIDE 8
Data Dimensionality!
X X X X X X X X X
Cigarettes Consumed Per Day Probability of Lung Cancer Developing
SLIDE 9
Data Dimensionality!
X X X X X X X X X
Cigarettes Consumed Per Day Probability of Lung Cancer Developing
X
SLIDE 10
Data Dimensionality!
X X X X X X X X X
Cigarettes Consumed Per Day Probability of Lung Cancer Developing
SLIDE 11
Data Dimensionality!
X X X X X X X X X
Cigarettes Consumed Per Day Probability of Lung Cancer Developing
D i f f e r e n c e f r
- m
m e a n h e i g h t ?
SLIDE 12
Data Dimensionality!
X X X X X X X X X
Cigarettes Consumed Per Day Probability of Lung Cancer Developing
1 Dimension!
SLIDE 13
Words and documents are the same way...
Xtfidf
|V| |D|
SLIDE 14
Words and documents are the same way...
Xtfidf
|V| |D|
Pineapple
SLIDE 15
Words and documents are the same way...
Xtfidf
|V| |D|
Pineapple
X
SLIDE 16
Words and documents are the same way...
Xtfidf
|V| |D|
Pineapple
X
SLIDE 17
Words and documents are the same way...
Xtfidf
|V| |D|
Pineapple
X
(but, really -- a low dimensional subspace…)
SLIDE 18
Words and documents are the same way...
Xtfidf
|V| |D|
“ P i n e a p p l e s w e r e r e c a l l e d … ”
X
SLIDE 19
Words and documents are the same way...
Xtfidf
|V| |D|
“ P i n e a p p l e s w e r e r e c a l l e d … ”
X
SLIDE 20
Words and documents are the same way...
Xtfidf
|V| |D|
“ P i n e a p p l e s w e r e r e c a l l e d … ”
X
(but, really -- a low dimensional subspace…)
SLIDE 21
Key questions in unsupervised NLP:
SLIDE 22
Key questions in unsupervised NLP:
- 1. How many dimensions does our
dataset actually live in?
SLIDE 23
Key questions in unsupervised NLP:
- 1. How many dimensions does our
dataset actually live in?
- 2. How do we project our data down to
those dimensions?
SLIDE 24
Key questions in unsupervised NLP:
- 1. How many dimensions does our
dataset actually live in?
- 2. How do we project our data down to
those dimensions?
- 3. Does any of this stuff actually do
anything for our projects?
SLIDE 25
Key tool in Linear Algebra, NLP, Machine Learning, Data Science, Computer Vision, Algorithms, Matrix Computations, Optimization, Statistics...
SLIDE 26
Key tool in Linear Algebra, NLP, Machine Learning, Data Science, Computer Vision, Algorithms, Matrix Computations, Optimization, Statistics...
SLIDE 27
Key tool in Linear Algebra, NLP, Machine Learning, Data Science, Computer Vision, Algorithms, Matrix Computations, Optimization, Statistics...
SLIDE 28
Key tool in Linear Algebra, NLP, Machine Learning, Data Science, Computer Vision, Algorithms, Matrix Computations, Optimization, Statistics...
SLIDE 29
Xtfidf
|V| |D|
=
SLIDE 30
Xtfidf
|V| |D|
=
|D| |V|
1 1
SLIDE 31
Xtfidf
|V| |D|
=
|D| |V|
1 1
k
SLIDE 32
Xtfidf
|V| |D|
=
|D| |V|
1 1
k
1 1
SLIDE 33
Xtfidf
|V| |D|
=
|D| |V|
2 2 2 2 k1 0 0 k2
SLIDE 34
Xtfidf
|V| |D|
=
|D| |V|
2 2 2 2 k1 0 0 k2
SLIDE 35
SLIDE 36
Key questions in unsupervised NLP:
- 1. How many dimensions does our dataset actually live
in?
- 2. How do we project our data down to those dimensions?
SLIDE 37
Enough talk, time for some magic.
SLIDE 38
Xtfidf
|V| |D|
+ =
SLIDE 39
Xtfidf
|V| |D|
+ = Latent Semantic Indexing (LSI)
SLIDE 40
Xtfidf
|V| |D|
+ = Latent Semantic Indexing (LSI) = Latent Semantic Analysis (LSA)
SLIDE 41
As a side note...
Great first NLP paper to read! Highly accessible :-)
Scott Deerwester
“Indexing by latent semantic analysis” - Deerwester et al. 1990
SLIDE 42
What are topic models?
Xtfidf
|V| |D| |D| |V|
k k
SLIDE 43
What are topic models?
Xtfidf
|V| |D| |D| |V|
Latent semantic indexing (Deerwater et al. 1990) k k
SLIDE 44
What are topic models?
Xtfidf
|V| |D| |D| |V|
Latent semantic indexing (Deerwater et al. 1990) k k Non-negative matrix factorization (Lee and Seung 1999)
SLIDE 45
What are topic models?
Xtfidf
|V| |D| |D| |V|
Latent dirichlet allocation (Blei et al. 2003) Latent semantic indexing (Deerwater et al. 1990) k k Non-negative matrix factorization (Lee and Seung 1999)
SLIDE 46
Why do we care??
|D| |V|
k k
SLIDE 47
Why do we care??
|D| |V|
k k
SLIDE 48
Why do we care??
|D| |V|
k k
Interpretable, small number of features for text classification!
SLIDE 49
Why do we care??
|D| |V|
k k
Interpretable, small number of features for text classification!
SLIDE 50
Document length *matters a lot*
SLIDE 51
Document length *matters a lot*
Different regimes of supervised NLP
(Jack’s opinions only! Lots of caveats!)
SLIDE 52
Document length *matters a lot*
Different regimes of supervised NLP
(Jack’s opinions only! Lots of caveats!)
50-100 Words
Less words More words
SLIDE 53
Document length *matters a lot*
Different regimes of supervised NLP
(Jack’s opinions only! Lots of caveats!)
50-100 Words
Less words More words
Topic models fail Topic models work
SLIDE 54
Document length *matters a lot*
Different regimes of supervised NLP
(Jack’s opinions only! Lots of caveats!)
50-100 Words
Less words More words
Topic models fail Topic models work Naive bayes, n-gram features + linear classifier are almost always pretty good in practice :-)
SLIDE 55