DETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING - - PowerPoint PPT Presentation

▶
determinantal point processes
SMART_READER_LITE
LIVE PREVIEW

DETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING - - PowerPoint PPT Presentation

Nov 17, 2022 176 likes •1.73k views

DETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex Kulesza and Ben Taskar OUTLINE OUTLINE Motivation & background on DPPs OUTLINE Motivation & background on DPPs Large-scale

slide-1
SLIDE 1

DETERMINANTAL POINT PROCESSES

FOR

NATURAL LANGUAGE PROCESSING

Jennifer Gillenwater Joint work with Alex Kulesza and Ben Taskar

slide-2
SLIDE 2

OUTLINE

slide-3
SLIDE 3

OUTLINE

Motivation & background on DPPs

slide-4
SLIDE 4

OUTLINE

Motivation & background on DPPs Large-scale settings

slide-5
SLIDE 5

OUTLINE

Motivation & background on DPPs Large-scale settings Structured summarization

slide-6
SLIDE 6

OUTLINE

Motivation & background on DPPs Large-scale settings Structured summarization Other potential NLP applications

slide-7
SLIDE 7

MOTIVATION & BACKGROUND

slide-8
SLIDE 8

SUMMARIZATION

slide-9
SLIDE 9

SUMMARIZATION ...

slide-10
SLIDE 10

SUMMARIZATION ...

slide-11
SLIDE 11

SUMMARIZATION ...

Quality: relevance to the topic

slide-12
SLIDE 12

SUMMARIZATION ...

Quality: relevance to the topic Diversity: coverage of core ideas

slide-13
SLIDE 13

SUBSET SELECTION

slide-14
SLIDE 14

SUBSET SELECTION

slide-15
SLIDE 15

SUBSET SELECTION

slide-16
SLIDE 16

SUBSET SELECTION

slide-17
SLIDE 17

AREA AS SET-GOODNESS

slide-18
SLIDE 18

feature space

AREA AS SET-GOODNESS

slide-19
SLIDE 19

feature space

Bi Bj

AREA AS SET-GOODNESS

slide-20
SLIDE 20

feature space

quality = p B>

i Bi

similarity = B>

i Bj

Bi Bj

AREA AS SET-GOODNESS

slide-21
SLIDE 21

feature space

quality = p B>

i Bi

similarity = B>

i Bj

Bi + Bj Bi Bj

AREA AS SET-GOODNESS

a r e a = q k Bi k2

2kBjk2 2 (B> i Bj)2

slide-22
SLIDE 22

feature space

quality = p B>

i Bi

similarity = B>

i Bj

Bi + Bj Bi Bj

AREA AS SET-GOODNESS

a r e a = q k Bi k2

2kBjk2 2 (B> i Bj)2

slide-23
SLIDE 23

feature space

quality = p B>

i Bi

similarity = B>

i Bj

Bi + Bj Bi Bj

AREA AS SET-GOODNESS

a r e a = q k Bi k2

2kBjk2 2 (B> i Bj)2

slide-24
SLIDE 24

VOLUME AS SET-GOODNESS

area = q kBik2

2kBjk2 2 (B> i Bj)2

slide-25
SLIDE 25

VOLUME AS SET-GOODNESS

area = q kBik2

2kBjk2 2 (B> i Bj)2

slide-26
SLIDE 26

VOLUME AS SET-GOODNESS

area = q kBik2

2kBjk2 2 (B> i Bj)2

length = kBik2

slide-27
SLIDE 27

VOLUME AS SET-GOODNESS

area = q kBik2

2kBjk2 2 (B> i Bj)2

length = kBik2 volume = base × height

slide-28
SLIDE 28

VOLUME AS SET-GOODNESS

area = q kBik2

2kBjk2 2 (B> i Bj)2

= ||B1||2vol(proj⊥B1(B2:N)) vol(B) = height × base

length = kBik2 volume = base × height

slide-29
SLIDE 29

AREA AS A DET

area = q kBik2

2kBjk2 2 (B> i Bj)2

slide-30
SLIDE 30

AREA AS A DET

area = q kBik2

2kBjk2 2 (B> i Bj)2

||Bi||2

2

B>

i Bj

B>

i Bj

( )

= det

||Bj||2

2

1 2

slide-31
SLIDE 31

AREA AS A DET

area = q kBik2

2kBjk2 2 (B> i Bj)2

||Bi||2

2

B>

i Bj

B>

i Bj

( )

= det

||Bj||2

2

1 2

= det(

)

Bi Bj Bi Bj

1 2

slide-32
SLIDE 32

VOLUME AS A DET

= det(

)

Bi Bj Bi Bj

1 2

vol(B{i,j})

slide-33
SLIDE 33

VOLUME AS A DET

= det(

)

Bi Bj Bi Bj

1 2

vol(B{i,j}) vol(B) = det

1 2

B1 BN . . . B1 BN . . .

( )

vol(B)2 = det(B>B) = det(L)

slide-34
SLIDE 34

COMPLEX STATISTICS

slide-35
SLIDE 35

COMPLEX STATISTICS

slide-36
SLIDE 36

COMPLEX STATISTICS

slide-37
SLIDE 37

COMPLEX STATISTICS

P

slide-38
SLIDE 38

COMPLEX STATISTICS

P

slide-39
SLIDE 39

COMPLEX STATISTICS

P

slide-40
SLIDE 40

COMPLEX STATISTICS

P

slide-41
SLIDE 41

COMPLEX STATISTICS

P

slide-42
SLIDE 42

COMPLEX STATISTICS

P

slide-43
SLIDE 43

COMPLEX STATISTICS

P

slide-44
SLIDE 44

COMPLEX STATISTICS

P

slide-45
SLIDE 45

COMPLEX STATISTICS

N items = ⇒ 2N sets

slide-46
SLIDE 46

EFFICIENT COMPUTATION

slide-47
SLIDE 47

det

1 2

EFFICIENT COMPUTATION

slide-48
SLIDE 48

EFFICIENT COMPUTATION

2 det

slide-49
SLIDE 49

EFFICIENT COMPUTATION

2 P det

slide-50
SLIDE 50

EFFICIENT COMPUTATION

2 det O(N 3)

slide-51
SLIDE 51

POINT PROCESSES

Y = {1, . . . , N}

slide-52
SLIDE 52

POINT PROCESSES

Y = {1, . . . , N}

slide-53
SLIDE 53

POINT PROCESSES

Y = {1, . . . , N}

( )

P

slide-54
SLIDE 54

POINT PROCESSES

Y = {1, . . . , N}

( )

P

= 0.2

slide-55
SLIDE 55

DETERMINANTAL

slide-56
SLIDE 56

DETERMINANTAL

P({2, 3, 5}) ∝

slide-57
SLIDE 57

DETERMINANTAL

L11 L12 L13 L14 L15 L21 L22 L23 L24 L25 L35 L34 L33 L32 L31 L41 L42 L43 L44 L45 L55 L54 L53 L52 L51 P({2, 3, 5}) ∝

slide-58
SLIDE 58

DETERMINANTAL

L11 L12 L13 L14 L15 L21 L22 L23 L24 L25 L35 L34 L33 L32 L31 L41 L42 L43 L44 L45 L55 L54 L53 L52 L51 P({2, 3, 5}) ∝

slide-59
SLIDE 59

DETERMINANTAL

L22 L23 L25 L35 L33 L32 L55 L53 L52 P({2, 3, 5}) ∝

slide-60
SLIDE 60

DETERMINANTAL

L22 L23 L25 L35 L33 L32 L55 L53 L52 det(

)

P({2, 3, 5}) ∝

slide-61
SLIDE 61

DETERMINANTAL

L22 L23 L25 L35 L33 L32 L55 L53 L52 det(

)

P({2, 3, 5}) =

slide-62
SLIDE 62

DETERMINANTAL

L22 L23 L25 L35 L33 L32 L55 L53 L52 det(

)

P({2, 3, 5}) = det(L + I)

slide-63
SLIDE 63

EFFICIENT INFERENCE

slide-64
SLIDE 64

EFFICIENT INFERENCE

PL(Y = Y )

Normalizing:

slide-65
SLIDE 65

EFFICIENT INFERENCE

PL(Y = Y ) P(Y ⊆ Y)

Normalizing: Marginalizing:

slide-66
SLIDE 66

EFFICIENT INFERENCE

PL(Y = Y ) PL(Y = B | A ⊆ Y) PL(Y = B | A ∩ Y = ∅) P(Y ⊆ Y)

Normalizing: Marginalizing: Conditioning:

slide-67
SLIDE 67

EFFICIENT INFERENCE

PL(Y = Y ) PL(Y = B | A ⊆ Y) PL(Y = B | A ∩ Y = ∅) Y ∼ PL P(Y ⊆ Y)

Normalizing: Marginalizing: Conditioning: Sampling:

slide-68
SLIDE 68

EFFICIENT INFERENCE

PL(Y = Y ) PL(Y = B | A ⊆ Y) PL(Y = B | A ∩ Y = ∅) Y ∼ PL P(Y ⊆ Y) O(N 3)

Normalizing: Marginalizing: Conditioning: Sampling:

slide-69
SLIDE 69

LARGE-SCALE SETTINGS

slide-70
SLIDE 70

DUAL KERNEL

KULESZA AND TASKAR (NIPS 2010)

slide-71
SLIDE 71

DUAL KERNEL

B1 BN

. . .

B2 B3 B1 BN

. . .

B2 B3

L

KULESZA AND TASKAR (NIPS 2010)

slide-72
SLIDE 72

DUAL KERNEL

B1 BN

. . .

B2 B3 B1 BN

. . .

B2 B3

L

N × N =

KULESZA AND TASKAR (NIPS 2010)

slide-73
SLIDE 73

DUAL KERNEL

B1 BN

. . .

B2 B3 B1 BN

. . .

B2 B3

C

N × N =

KULESZA AND TASKAR (NIPS 2010)

slide-74
SLIDE 74

DUAL KERNEL

B1 BN

. . .

B2 B3 B1 BN

. . .

B2 B3

C

=

KULESZA AND TASKAR (NIPS 2010)

slide-75
SLIDE 75

DUAL KERNEL

B1 BN

. . .

B2 B3 B1 BN

. . .

B2 B3

= D × D

C

KULESZA AND TASKAR (NIPS 2010)

slide-76
SLIDE 76

DUAL INFERENCE

slide-77
SLIDE 77

DUAL INFERENCE

L = V ΛV > C = ˆ V Λ ˆ V >

slide-78
SLIDE 78

DUAL INFERENCE

L = V ΛV > C = ˆ V Λ ˆ V > V = B> ˆ V Λ 1

2

slide-79
SLIDE 79

DUAL INFERENCE

L = V ΛV > C = ˆ V Λ ˆ V > V = B> ˆ V Λ 1

2

O(D3) P

Y det(LY )

Normalizing

slide-80
SLIDE 80

DUAL INFERENCE

L = V ΛV > C = ˆ V Λ ˆ V > V = B> ˆ V Λ 1

2

O(D3) P

Y det(LY )

Normalizing

O(D3 + D2k2)

Marginalizing & Conditioning

slide-81
SLIDE 81

DUAL INFERENCE

L = V ΛV > C = ˆ V Λ ˆ V > V = B> ˆ V Λ 1

2

O(ND2k) Y ∼ PL

Sampling

O(D3) P

Y det(LY )

Normalizing

O(D3 + D2k2)

Marginalizing & Conditioning

slide-82
SLIDE 82

EXPONENTIAL N

slide-83
SLIDE 83

EXPONENTIAL N

N = O({sentence length}{sentence length})

We want to select a diverse set of parses.

slide-84
SLIDE 84

EXPONENTIAL N

N = O({sentence length}{sentence length})

We want to select a diverse set of parses.

N = O({node degree}{path length})

slide-85
SLIDE 85

i =

STRUCTURE FACTORIZATION

KULESZA AND TASKAR (NIPS 2010)

slide-86
SLIDE 86

i =

STRUCTURE FACTORIZATION

Bi = q(i)φ(i)

KULESZA AND TASKAR (NIPS 2010)

quality similarity

slide-87
SLIDE 87

i =

STRUCTURE FACTORIZATION

Bi = q(i)φ(i) i = {iα}α∈F α c = 1

KULESZA AND TASKAR (NIPS 2010)

quality similarity

slide-88
SLIDE 88

i =

STRUCTURE FACTORIZATION

Bi = q(i)φ(i) i = {iα}α∈F α c = 2

KULESZA AND TASKAR (NIPS 2010)

quality similarity

slide-89
SLIDE 89

i =

STRUCTURE FACTORIZATION

i = {iα}α∈F Bi =  Q

α∈F

q(iα)

  • φ(i)

α c = 2

KULESZA AND TASKAR (NIPS 2010)

slide-90
SLIDE 90

i =

STRUCTURE FACTORIZATION

i = {iα}α∈F Bi =  Q

α∈F

q(iα)  P

α∈F

φ(iα)

  • α

c = 2

KULESZA AND TASKAR (NIPS 2010)

slide-91
SLIDE 91

i =

STRUCTURE FACTORIZATION

i = {iα}α∈F Bi =  Q

α∈F

q(iα)  P

α∈F

φ(iα)

  • α

c = 2 O(ND2k) Y ∼ PL

KULESZA AND TASKAR (NIPS 2010)

slide-92
SLIDE 92

STRUCTURE FACTORIZATION

Bi =  Q

α∈F

q(iα)  P

α∈F

φ(iα)

  • M =

R = α c = 2 Y ∼ PL O(D2k3 + Dk2M cR)

KULESZA AND TASKAR (NIPS 2010)

slide-93
SLIDE 93

STRUCTURE FACTORIZATION

Bi =  Q

α∈F

q(iα)  P

α∈F

φ(iα)

  • M =

R = α c = 2 Y ∼ PL O(D2k3 + Dk2M cR)

KULESZA AND TASKAR (NIPS 2010)

M cR = 42 ⇤ 12 = 192 ⌧ N = 412 = 16,777,216

slide-94
SLIDE 94

LARGE FEATURE SETS?

slide-95
SLIDE 95

Large Exponential Small dual dual + structure

LARGE FEATURE SETS?

N = # of items D = # of features

slide-96
SLIDE 96

Large Exponential Small dual dual + structure Large

? ?

LARGE FEATURE SETS?

N = # of items D = # of features

slide-97
SLIDE 97

RANDOM PROJECTIONS

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

slide-98
SLIDE 98

RANDOM PROJECTIONS

N D Φ

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

slide-99
SLIDE 99

RANDOM PROJECTIONS

D Φ

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

M cR

slide-100
SLIDE 100

RANDOM PROJECTIONS

D d D × Φ

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

M cR

slide-101
SLIDE 101

RANDOM PROJECTIONS

D d D × d = Φ

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

M cR M cR

slide-102
SLIDE 102

RANDOM PROJECTIONS

d D

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

slide-103
SLIDE 103

RANDOM PROJECTIONS

d D

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

slide-104
SLIDE 104

VOLUME PRESERVATION

slide-105
SLIDE 105

VOLUME PRESERVATION

JOHNSON AND LINDENSTRAUSS (1984)

slide-106
SLIDE 106

VOLUME PRESERVATION

JOHNSON AND LINDENSTRAUSS (1984)

log N

slide-107
SLIDE 107

VOLUME PRESERVATION

JOHNSON AND LINDENSTRAUSS (1984)

log N

slide-108
SLIDE 108

VOLUME PRESERVATION

MAGEN AND ZOUZIAS (2008)

log N

slide-109
SLIDE 109

VOLUME PRESERVATION

MAGEN AND ZOUZIAS (2008)

log N

slide-110
SLIDE 110

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

DPP PRESERVATION

vol2 = det

slide-111
SLIDE 111

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

DPP PRESERVATION

≈

k = 1

vol2 = det

slide-112
SLIDE 112

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

DPP PRESERVATION

≈

k = 1

≈

k = 2

vol2 = det

slide-113
SLIDE 113

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

DPP PRESERVATION

≈

k = 1

≈

k = 2

≈

k = 3

vol2 = det

slide-114
SLIDE 114

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

DPP PRESERVATION

slide-115
SLIDE 115

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

DPP PRESERVATION

d = O ⇣ max n

k ✏ , log(1/)+log(N) ✏2

+ k

  • ⌘
slide-116
SLIDE 116

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

DPP PRESERVATION

d = O ⇣ max n

k ✏ , log(1/)+log(N) ✏2

+ k

  • ⌘

total # of items subset size

slide-117
SLIDE 117

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

DPP PRESERVATION

d = O ⇣ max n

k ✏ , log(1/)+log(N) ✏2

+ k

  • ⌘

w.p. 1 δ : kPk ˜ Pkk1  e6k✏ 1

total # of items subset size

slide-118
SLIDE 118

GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)

DPP PRESERVATION

d = O ⇣ max n

k ✏ , log(1/)+log(N) ✏2

+ k

  • ⌘

50 100 150 0.2 0.4 0.6 0.8 1 1.2 L1 variational distance Projection dimension 1 2 3 4 x 10

8

Memory use (bytes)

w.p. 1 δ : kPk ˜ Pkk1  e6k✏ 1

total # of items subset size

slide-119
SLIDE 119

STRUCTURED SUMMARIZATION

slide-120
SLIDE 120

NEWS THREADING

slide-121
SLIDE 121

NEWS THREADING

slide-122
SLIDE 122

NEWS THREADING

March 28: Health officials confirm Ebola outbreak in Guinea’s capital

slide-123
SLIDE 123

NEWS THREADING

March 28: Health officials confirm Ebola outbreak in Guinea’s capital August 8: World Health Organization declares Ebola epidemic an international health emergency

slide-124
SLIDE 124

NEWS THREADING

March 28: Health officials confirm Ebola outbreak in Guinea’s capital August 8: World Health Organization declares Ebola epidemic an international health emergency September 2: GlaxoSmithKlein begins Ebola vaccine drug trial

slide-125
SLIDE 125

NEWS THREADING

10360

March 28: Health officials confirm Ebola outbreak in Guinea’s capital August 8: World Health Organization declares Ebola epidemic an international health emergency September 2: GlaxoSmithKlein begins Ebola vaccine drug trial

M ≈ 35,000

slide-126
SLIDE 126

PROJECTING NEWS FEATURES

slide-127
SLIDE 127

PROJECTING NEWS FEATURES

φ(i) D = 36,356

slide-128
SLIDE 128

PROJECTING NEWS FEATURES

G

Gφ(i) φ(i) D = 36,356 d = 50

slide-129
SLIDE 129

30

slide-130
SLIDE 130

31

slide-131
SLIDE 131

DPP THREADS

slide-132
SLIDE 132

DPP THREADS

Jan 08 Jan 28 Feb 17 Mar 09 Mar 29 Apr 18 May 08 May 28 Jun 17 pope vatican church parkinson israel palestinian iraqi israeli gaza abbas baghdad

  • wen nominees senate democrats judicial filibusters

social tax security democrats rove accounts iraq iraqi killed baghdad arab marines deaths forces

slide-133
SLIDE 133

DPP THREADS

Jan 08 Jan 28 Feb 17 Mar 09 Mar 29 Apr 18 May 08 May 28 Jun 17 pope vatican church parkinson israel palestinian iraqi israeli gaza abbas baghdad

  • wen nominees senate democrats judicial filibusters

social tax security democrats rove accounts iraq iraqi killed baghdad arab marines deaths forces

Feb 24: Parkinson's Disease Increases Risks to Pope Feb 26: Pope's Health Raises Questions About His Ability to Lead Mar 13: Pope Returns Home After 18 Days at Hospital Apr 01: Pope's Condition Worsens as World Prepares for End of Papacy Apr 02: Pope, Though Gravely Ill, Utters Thanks for Prayers Apr 18: Europeans Fast Falling Away from Church Apr 20: In Developing World, Choice [of Pope] Met with Skepticism May 18: Pope Sends Message with Choice of Name

slide-134
SLIDE 134

System ROUGE-1F R-SU4F Coherence

QUANTITATIVE RESULTS

slide-135
SLIDE 135

System k-means ROUGE-1F 16.5 R-SU4F 3.76 Coherence 2.73

QUANTITATIVE RESULTS

slide-136
SLIDE 136

System k-means DTM ROUGE-1F 16.5 14.7 R-SU4F 3.76 3.44 Coherence 2.73 3.2

QUANTITATIVE RESULTS

slide-137
SLIDE 137

System k-means DTM DPP ROUGE-1F 16.5 14.7 17.2 R-SU4F 3.76 3.44 3.98 Coherence 2.73 3.2 3.3

QUANTITATIVE RESULTS

slide-138
SLIDE 138

System k-means DTM DPP ROUGE-1F 16.5 14.7 17.2 R-SU4F 3.76 3.44 3.98 Coherence 2.73 3.2 3.3 Runtime (s) 626 19,434 252

QUANTITATIVE RESULTS

slide-139
SLIDE 139

OTHER POTENTIAL NLP APPLICATIONS

slide-140
SLIDE 140

POTENTIAL APP: RE-RANKING

slide-141
SLIDE 141
  • Parser: simple model with local features defines

basic scores for all possible parse trees

POTENTIAL APP: RE-RANKING

slide-142
SLIDE 142
  • Parser: simple model with local features defines

basic scores for all possible parse trees

  • Re-ranker: more complex model with non-local

features provides more refined scores

POTENTIAL APP: RE-RANKING

slide-143
SLIDE 143
  • Parser: simple model with local features defines

basic scores for all possible parse trees

  • Re-ranker: more complex model with non-local

features provides more refined scores

  • Typical pipeline: find the k highest-scoring parses

under the simple model, then score these k with the more complex model and output the best

POTENTIAL APP: RE-RANKING

slide-144
SLIDE 144
  • Parser: simple model with local features defines

basic scores for all possible parse trees

  • Re-ranker: more complex model with non-local

features provides more refined scores

  • Typical pipeline: find the k highest-scoring parses

under the simple model, then score these k with the more complex model and output the best

  • Issue: the k may be largely redundant, so re-

ranker does not get to consider significantly different parses

POTENTIAL APP: RE-RANKING

slide-145
SLIDE 145

IDEA: USE DPPS FOR SELECTING RE-RANKER INPUT

slide-146
SLIDE 146

IDEA: USE DPPS FOR SELECTING RE-RANKER INPUT

N = O({sentence length}{sentence length})

We want to select a diverse set of parses.

slide-147
SLIDE 147

IDEA: USE DPPS FOR SELECTING RE-RANKER INPUT

N = O({sentence length}{sentence length})

We want to select a diverse set of parses.

Quality: standard parser scores

slide-148
SLIDE 148

IDEA: USE DPPS FOR SELECTING RE-RANKER INPUT

N = O({sentence length}{sentence length})

We want to select a diverse set of parses.

Quality: standard parser scores Diversity: edge lengths, POS pairs, etc.

slide-149
SLIDE 149

POTENTIAL APP: WORD SENSE INDUCTION

slide-150
SLIDE 150

POTENTIAL APP: WORD SENSE INDUCTION

  • Goal: identify all possible senses of ambiguous

words (e.g. river bank vs bank deposit)

slide-151
SLIDE 151

POTENTIAL APP: WORD SENSE INDUCTION

  • Goal: identify all possible senses of ambiguous

words (e.g. river bank vs bank deposit)

  • Typical approach: unsupervised clustering,

cluster centers represent word senses

slide-152
SLIDE 152

POTENTIAL APP: WORD SENSE INDUCTION

  • Goal: identify all possible senses of ambiguous

words (e.g. river bank vs bank deposit)

  • Typical approach: unsupervised clustering,

cluster centers represent word senses

  • Why DPPs fit: can re-express finding cluster

centers as the problem of finding a high-quality, diverse set

slide-153
SLIDE 153

POTENTIAL APP: WORD SENSE INDUCTION

  • Goal: identify all possible senses of ambiguous

words (e.g. river bank vs bank deposit)

  • Typical approach: unsupervised clustering,

cluster centers represent word senses

  • Why DPPs fit: can re-express finding cluster

centers as the problem of finding a high-quality, diverse set Quality: centrality (density

  • f points nearby)
slide-154
SLIDE 154

POTENTIAL APP: WORD SENSE INDUCTION

  • Goal: identify all possible senses of ambiguous

words (e.g. river bank vs bank deposit)

  • Typical approach: unsupervised clustering,

cluster centers represent word senses

  • Why DPPs fit: can re-express finding cluster

centers as the problem of finding a high-quality, diverse set Quality: centrality (density

  • f points nearby)

Diversity: same as standard WSI features

slide-155
SLIDE 155

QUESTIONS?