Word representations and modelling ambiguity: A case study of - - PowerPoint PPT Presentation

▶
word representations and modelling ambiguity a case study
SMART_READER_LITE
LIVE PREVIEW

Word representations and modelling ambiguity: A case study of - - PowerPoint PPT Presentation

Jul 17, 2023 482 likes •791 views

Word representations and modelling ambiguity: A case study of metaphor Ekaterina Shutova ILLC University of Amsterdam 2 May 2018 Word representations and modelling ambiguity: A case study of metaphor 1 / 29 Polysemy and word senses The

slide-1
SLIDE 1

Word representations and modelling ambiguity: A case study of metaphor

Ekaterina Shutova ILLC University of Amsterdam 2 May 2018

1 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-2
SLIDE 2

Polysemy and word senses The children ran to the store If you see this man, run! Service runs all the way to Cranbury She is running a relief operation in Sudan the story or argument runs as follows Does this old car still run well? Interest rates run from 5 to 10 percent Who’s running for treasurer this year? They ran the tapes over and over again These dresses run small

2 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-3
SLIDE 3

Polysemy

homonymy: unrelated word senses. bank (raised land) vs bank (financial institution) bank (financial institution) vs bank (in a casino): related but distinct senses. regular polysemy and sense extension

zero-derivation, e.g. tango (N) vs tango (V), or rabbit, turkey, halibut (meat / animal) metaphorical senses, e.g. swallow [food], swallow [information], swallow [anger] metonymy, e.g. he played Bach; he drank his glass.

vagueness: nurse, lecturer, driver cultural stereotypes: nurse, lecturer, driver

No clearcut distinctions. Dictionaries are not consistent.

3 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-4
SLIDE 4

What is metaphor?

4 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-5
SLIDE 5

What is metaphor? “A political machine” “The wheels of the regime were well oiled and already turning” “Time to mend our foreign policy” “20 Steps towards a Modern, Working Democracy”

5 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-6
SLIDE 6

How does it work? Conceptual Metaphor Theory (Lakoff and Johnson, 1980) Metaphorical associations between concepts POLITICALSYSTEM

  • target

is a MECHANISM

  • source

Cross-domain knowledge projection and inference Reasoning about the target domain in terms of the properties

  • f the source

6 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-7
SLIDE 7

Computational metaphor processing tasks

1

Learn metaphorical associations from corpora “POLITICAL SYSTEM is a MECHANISM”

2

Identify metaphorical language in text “mend the policy”

3

Interpret the metaphorical language “mend the policy” means “improve the policy; address the downsides of the policy"

7 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-8
SLIDE 8

Metaphor in the distributional space

N: game N: politics 1170 play 31 dominate 202 win 30 play 99 miss 28 enter 76 watch 16 discuss 66 lose 13 leave 63 start 12 understand 42 enjoy 8 study 22 finish 6 explain ... 5 shape 20 dominate 4 influence 18 quit 4 change 17 host 4 analyse 17 follow ... 17 control 2 transform ... ...

8 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-9
SLIDE 9

Metaphor in the distributional space

N: game N: politics 1170 play 31 dominate 202 win 30 play 99 miss 28 enter 76 watch 16 discuss 66 lose 13 leave 63 start 12 understand 42 enjoy 8 study 22 finish 6 explain ... 5 shape 20 dominate 4 influence 18 quit 4 change 17 host 4 analyse 17 follow ... 17 control 2 transform ... ...

9 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-10
SLIDE 10

Metaphor in the distributional space

N: game N: politics 1170 play 31 dominate 202 win 30 play 99 miss 28 enter 76 watch 16 discuss 66 lose 13 leave 63 start 12 understand 42 enjoy 8 study 22 finish 6 explain ... 5 shape 20 dominate 4 influence 18 quit 4 change 17 host 4 analyse 17 follow ... 17 control 2 transform ... ...

10 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-11
SLIDE 11

Metaphor in the distributional space

N: game N: politics 1170 play 31 dominate 202 win 30 play 99 miss 28 enter 76 watch 16 discuss 66 lose 13 leave 63 start 12 understand 42 enjoy 8 study 22 finish 6 explain ... 5 shape 20 dominate 4 influence 18 quit 4 change 17 host 4 analyse 17 follow ... 17 control 2 transform ... ...

11 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-12
SLIDE 12

Metaphor in the distributional space

N: game N: politics 1170 play 31 dominate 202 win 30 play 99 miss 28 enter 76 watch 16 discuss 66 lose 13 leave 63 start 12 understand 42 enjoy 8 study 22 finish 6 explain ... 5 shape 20 dominate 4 influence 18 quit 4 change 17 host 4 analyse 17 follow ... 17 control 2 transform ... ...

12 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-13
SLIDE 13

Metaphor in the distributional space

N: game N: politics 1170 play 31 dominate 202 win 30 play 99 miss 28 enter 76 watch 16 discuss 66 lose 13 leave 63 start 12 understand 42 enjoy 8 study 22 finish 6 explain ... 5 shape 20 dominate 4 influence 18 quit 4 change 17 host 4 analyse 17 follow ... 17 control 2 transform ... ...

13 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-14
SLIDE 14

Metaphor in the distributional space

N: game N: politics 1170 play 31 dominate 202 win 30 play 99 miss 28 enter 76 watch 16 discuss 66 lose 13 leave 63 start 12 understand 42 enjoy 8 study 22 finish 6 explain ... 5 shape 20 dominate 4 influence 18 quit 4 change 17 host 4 analyse 17 follow ... 17 control 2 transform ... ...

14 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-15
SLIDE 15

Metaphor in the distributional space

N: game N: politics 1170 play 31 dominate 202 win 30 play 99 miss 28 enter 76 watch 16 discuss 66 lose 13 leave 63 start 12 understand 42 enjoy 8 study 22 finish 6 explain ... 5 shape 20 dominate 4 influence 18 quit 4 change 17 host 4 analyse 17 follow ... 17 control 2 transform ... ... NEED TO FIND A WAY TO PARTITION THE SPACE

15 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-16
SLIDE 16

Learning metaphorical associations by soft clustering

Unsupervised Metaphor Identification Using Hierarchical Graph Factorization

  • Clustering. Shutova and Sun, 2013. NAACL.

... 16 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-17
SLIDE 17

Learning metaphorical associations by soft clustering

GAME SPORT

...

...

MACHINE

... ...

function break repair

... DEMOCRACY AUTOCRACY

…

AUTOCRACY ENGINE MACHINE

...

DEMOCRACY GAME ENGINE SPORT

play compete ...

17 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-18
SLIDE 18

Creating the graph Experimental setup

ALGORITHM: Hierarchical graph

factorization clustering (Yu, Yu, Tresp, 2006)

DATASET: 2000 frequent nouns FEATURES: verbs in subject, direct and

indirect object relations (Gigaword corpus)

LEVELS: 10

v1 v7 v6 v9 v8 v2 v3 v4 v5 u1 u2 u3 q1 q2 18 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-19
SLIDE 19

Hierarchical clustering using graph factorization

v1 v6 v2 v4 v3 v5 v7 v8 v9

Similarity matrix W: wij = JSD(vi, vj)

19 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-20
SLIDE 20

Hierarchical clustering using graph factorization

v1 v6 v2 v4 v3 v5 v7 v8 v9 v1 v7 v6 v9 v8 v2 v3 v4 v5 u1 u2 u3

Similarity matrix W: wij = JSD(vi, vj)

20 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-21
SLIDE 21

Hierarchical clustering using graph factorization

v1 v6 v2 v4 v3 v5 v7 v8 v9 v1 v7 v6 v9 v8 v2 v3 v4 v5 u1 u2 u3

Similarity matrix W: wij = JSD(vi, vj) W ′: w′

ij = m p=1 bipbjp λp

bip – connection weight λi = n

i=1 bip 21 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-22
SLIDE 22

Hierarchical clustering using graph factorization

v1 v6 v2 v4 v3 v5 v7 v8 v9 v1 v7 v6 v9 v8 v2 v3 v4 v5 u1 u2 u3

u3 u1 u2

v1

v6

v2

v4

v3

v5 v7 v8 v9

Similarity matrix W: wij = JSD(vi, vj) W ′: w′

ij = m p=1 bipbjp λp

bip – connection weight λi = n

i=1 bip 22 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-23
SLIDE 23

Hierarchical clustering using graph factorization

v1 v6 v2 v4 v3 v5 v7 v8 v9 v1 v7 v6 v9 v8 v2 v3 v4 v5 u1 u2 u3

u3 u1 u2

v1

v6

v2

v4

v3

v5 v7 v8 v9 u1 u2 u3

Similarity matrix W: wij = JSD(vi, vj) W ′: w′

ij = m p=1 bipbjp λp

bip – connection weight λi = n

i=1 bip 23 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-24
SLIDE 24

Hierarchical clustering using graph factorization

v1 v6 v2 v4 v3 v5 v7 v8 v9 v1 v7 v6 v9 v8 v2 v3 v4 v5 u1 u2 u3

u3 u1 u2

v1

v6

v2

v4

v3

v5 v7 v8 v9 u1 u2 u3 v1 v7 v6 v9 v8 v2 v3 v4 v5 u1 u2 u3 q1 q2

Similarity matrix W: wij = JSD(vi, vj) W ′: w′

ij = m p=1 bipbjp λp

bip – connection weight λi = n

i=1 bip 24 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-25
SLIDE 25

Identifying metaphorical associations in the graph

start with the source concept, e.g. "fire"

  • utput a ranking of potential target concepts

v1 v7 v6 v9 v8 v2 v3 v4 v5 u1 u2 u3 q1 q2

SOURCE: fire TARGET: sense hatred emotion passion enthusiasm sentiment hope inter-

est feeling resentment optimism hostility excitement anger

TARGET: coup violence fight resistance clash rebellion battle drive fighting

riot revolt war confrontation volcano row revolution struggle

SOURCE: disease TARGET: fraud outbreak offence connection leak count crime violation

abuse conspiracy corruption terrorism suicide

TARGET: opponent critic rival

25 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-26
SLIDE 26

Identifying metaphorical expressions

GAME SPORT

...

...

MACHINE

... ...

function break repair

... DEMOCRACY AUTOCRACY

…

AUTOCRACY ENGINE MACHINE

...

DEMOCRACY GAME ENGINE SPORT

play compete ... 26 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-27
SLIDE 27

Identifying metaphorical expressions

GAME SPORT

...

...

MACHINE

... ...

function break repair

... DEMOCRACY AUTOCRACY

…

AUTOCRACY ENGINE MACHINE

...

DEMOCRACY GAME ENGINE SPORT

play compete ... 27 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-28
SLIDE 28

Metaphorical expressions retrieved

FEELING IS FIRE

anger blazed passion flared fuel resentment anger crackled etc.

CRIME IS A DISEASE

cure crime abuse transmitted suffer from corruption diagnose abuse etc. Output sentences from the BNC EG0 275 In the 1930s the words "means test" was a curse, fuelling the resistance against it both among the unemployed and some of its administrators. HL3 1206 [..] he would strive to accelerate progress towards the eco- nomic integration of the Caribbean. HXJ 121 [..] it is likely that some industries will flourish in certain coun- tries as the market widens.

28 / 29 Word representations and modelling ambiguity: A case study of metaphor

slide-29
SLIDE 29

How well does it work?

AGG WordNet HGFC 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

Precision

Multilingual metaphor processing

(Shutova et al. 2017, Computational Linguistics) Cross-cultural differences Spanish: poverty metaphors

POVERTY IS AN ENEMY, PAIN

English: immigration metaphors

IMMIGRATION IS A DISEASE, FIRE

Russian: sporting events / competi- tions associated with WAR

29 / 29 Word representations and modelling ambiguity: A case study of metaphor