Incrementality in Compositional Distributional Semantics M. - - PowerPoint PPT Presentation

▶
incrementality in compositional distributional semantics
SMART_READER_LITE
LIVE PREVIEW

Incrementality in Compositional Distributional Semantics M. - - PowerPoint PPT Presentation

Sep 15, 2022 244 likes •929 views

Incrementality in Compositional Distributional Semantics M. Sadrzadeh, EECS, QMUL SemDial 2018 joint work with M. Purver, J. Hough, R. Kempson SYCO2, Glasgow December 2018 NLP in one slide structure preserving map Semantic Formal

slide-1
SLIDE 1

Incrementality in Compositional Distributional Semantics

SemDial 2018 joint work with M. Purver, J. Hough, R. Kempson

SYCO2, Glasgow December 2018

  • M. Sadrzadeh, EECS, QMUL
slide-2
SLIDE 2

NLP in one slide

Formal Grammar Semantic Calculus

structure preserving map

slide-3
SLIDE 3

NLP in one slide

Formal Grammar Models of First Order Logic

structure preserving map

slide-4
SLIDE 4

NLP in one slide

Formal Grammar Distributions

  • f Linguistic

Data

structure preserving map

slide-5
SLIDE 5
  • sugar, a sliced lemon, a tablespoonful of apricot

preserve or jam, a pinch each of, their enjoyment. Cautiously she sampled her first pineapple and another fruit whose taste she likened well suited to programming on the digital computer. In finding the optimal R-stage policy from for the purpose of gathering data and information necessary for the study authorized in the

Distributional Semantics

computer data pinch result sugar apricot 2.25 2.25 pineapple 2.25 2.25 digital 1.66 information 0.57 0.47 Figure 15.7 The PPMI matrix showing the association between words and context words,

PPMI(w,c) = max(log2 P(w,c) P(w)P(c),0)

Speech and Language Processing, Jurafsky and Martin

slide-6
SLIDE 6

State of the art NLP packages

import spacy nlp = spacy.load('en_core_web_md') tokens = nlp(u'dog cat car') for token1 in tokens: for token2 in tokens: print(token1.text, token2.text, token1.similarity(token2))

dog dog 1.0 dog cat 0.80168545 dog car 0.35629162 cat dog 0.80168545 cat cat 1.0 cat car 0.31907532 car dog 0.35629162 car cat 0.31907532 car car 1.0

slide-7
SLIDE 7

Distributional Semantics

dog cat car dog 1 0.80 0.35 cat 1 0.31 car 1

slide-8
SLIDE 8

Distributional Semantics

blood grave dead vampire zombie butterfly

↵

  • ↵
slide-9
SLIDE 9

NLP in one slide

Formal Grammar Distributions

  • f Linguistic

Data

structure preserving map

??? ???

structure preserving map

slide-10
SLIDE 10

NLP in one slide

Formal Grammar Distributions

  • f Linguistic

Data

structure preserving map

Type Grammars Multilinear Algebra

structure preserving map

slide-11
SLIDE 11

CCG

T ypes Rules

A X/Y X \ Y

X/Y Y =) X Y X \ Y =) X

NP, S NP/NP, S\NP (S\NP)/NP

noun phrase adj, iTv Tv

NP/NP NP => NP NP S\NP => S

slide-12
SLIDE 12

Multilinear Algebraic Semantics

7! C ⌦ B A 7! A

7! A A = {ei}i X

A { } 3 Ti = X

i

Ciei X

Vectors

slide-13
SLIDE 13

A/B 7! A ⌦ B

A 7! A A = {ei}i B = {ej}j X

7! A ⌦ B

X 3 Tij = X

ij

Cij ei ⌦ ej

Matrices

Multilinear Algebraic Semantics

slide-14
SLIDE 14

A/(B/C) 7! A ⌦ (B ⌦ C) 7! A A = {ei}i B = {ej}j C = {ek}k X

X A ⌦ B ⌦ C 3 Tijk = X

ijk

Cijk ei ⌦ ej ⌦ ek

Cubes

Multilinear Algebraic Semantics

slide-15
SLIDE 15

Higher order tensors

A ⌦ B ⌦ · · · ⌦ Z 3 Ti j···w = X

i j···w

Ci j···w ei ⌦ e j ⌦ · · · ⌦ ew

Multilinear Algebraic Semantics

slide-16
SLIDE 16

A/B B =) A 7! (A ⌦ B) B =) A

X

···

Tij T j tensor contract =) Ti

Matrix Multiplication

( X

i j

Ci jei ⌦ e j)( X

i

C je j) = X

i

Ci jC jeihe j | e ji

Multilinear Algebraic Semantics

slide-17
SLIDE 17

· · · 7! A ⌦ B ⌦ · · · ⌦ M M ⌦ N ⌦ P ⌦ · · · ⌦ W tensor contract

· · · 7! A ⌦ B ⌦ · · · ⌦ M M ⌦ N ⌦ P ⌦ · · · ⌦ W Tij···m Tmnp···w tensor contract =) Tij···np···w

Higher order tensor contraction

X X X ( X

ij···m

Cij···mei ⌦ ej ⌦ · · · ⌦ em)( X

mn···w

Cmn···wem ⌦ en ⌦ · · · ⌦ ew) X X

···

X

···

= X

ij···n···w

Cij···mCmn···wei ⌦ ej ⌦ · · · ⌦ en ⌦ · · · ⌦ ewhem | emi

Multilinear Algebraic Semantics

slide-18
SLIDE 18

Dogs Chase White Cats

NP (S \ NP)/NP NP/NP NP

N N NP

S \ NP

S

slide-19
SLIDE 19

Dogs Chase White Cats

2 2 N 2 N N (S ⌦ N) ⌦ N N ⌦ N N 2 N ⌦ N N

S ⌦ N

N S

slide-20
SLIDE 20

Ti Tijk Tkl Tl

Dogs Chase White Cats

Tk

Ti j

T j

slide-21
SLIDE 21

Pregroup Grammars

T ypes … Rules

XYl

YrX NPNPl NPrS N

S NPrS NPl

XYlY  X Y

X YYrX  X

NPNPlNP  NP

NPNPrS  S N

slide-22
SLIDE 22

Catgorical Semantics

Formal Grammar Distributions

  • f Linguistic

Data

structure preserving map

Pregroup Grammars FVect

monoidal functor

slide-23
SLIDE 23

Categorial Grammars + Distributional Semantics

Coecke, Sadrzadeh, Clark, 2010 Grefenstette and Sadrzadeh 2011, 2015 Maillard, Clark, Grefenstette, 2014 Krishnamurti and Mitchell, 2014 Baroni and Zamparelli 2010 Wijnholds (and Moortgat) 2015-16

slide-24
SLIDE 24

Language Processing

Complete Sentences

slide-25
SLIDE 25

Naturally Occurring Dialogue

slide-26
SLIDE 26

Naturally Occurring Dialogue

1) A: Ray destroyed . . . B: . . . the fuchsia. He never liked it. The roses he spared . . . A: . . . this time.

slide-27
SLIDE 27

A: You are going to write the letter? B: Only if you post it! Naturally Occurring Dialogue

Howes et al, 2011, Poesio and Reiser 2010

slide-28
SLIDE 28

Computational Dialogue Systems A: I want to book a ticket … B: … from where? A: London B: … to where? A: to Paris.

Purver and Kempson 2011 Purver, Eshghi, Hough 2017

slide-29
SLIDE 29

Psycholinguistic Analysis A: The footballer dribbled … B (thinking) it means controlling the ball A: … the ball across the pitch A: The baby dribbled … the milk all over the floor.

Pickering and Frisson 2001

slide-30
SLIDE 30

Cognitive Neuroscience Predictive Processing: agents incrementally generate expectations and judge the degree to which they are met.

Frisson and Frith 2001 Clarke 2015

slide-31
SLIDE 31

Dynamic Syntax + Type Theoretic Semantics

  • Incremental Language Processing

Hough 2015, Purver et al 2014. Ruth Kempson, Wilfried Meyer-Viol, and Dov Gabbay. 2001.

slide-32
SLIDE 32

Recent Contribution

Dynamic Syntax + Distributional Semantics

Sadrzadeh, Purver, Hough, Kempson SemDial 2018

slide-33
SLIDE 33

Outline

  • Dynamic Syntax: DS
  • CDS for DS
  • Some Examples
  • Some Experimental Results
slide-34
SLIDE 34

Dynamic Syntax

O(X3, O(X1, X2)) X3 O(X1, X2) X1 X2

Trees decorated with semantic formulae and applications

slide-35
SLIDE 35

Dynamic Syntax

and with …

  • Ty: types of formulae
  • ?: requirements for further development
  • <>: node currently under development
  • links: connect trees of arguments of conjunctives etc
slide-36
SLIDE 36

Dynamic Syntax

  • ?()

(), () ?(⟨, ⟩), ♦

slide-37
SLIDE 37

Dynamic Syntax

  • ?()

(), () ?(⟨, ⟩) ?(), ♦ (⟨, ⟨, ⟩⟩), (λλ.(, ))

slide-38
SLIDE 38

Dynamic Syntax

  • ?()

(), () ?(⟨, ⟩), ♦ (), () (⟨, ⟨, ⟩⟩), (λλ.(, ))

slide-39
SLIDE 39

Dynamic Syntax

  • (), ((, )), ♦

(), () (⟨, ⟩), (λ.(, )) (), () (⟨, ⟨, ⟩⟩), (λλ.(, ))

slide-40
SLIDE 40

“mary, . . . ” “. . . who . . . ”

?S W, T mary

i

, ♦ ?W ⊗ S ?S W, T mary

i

?W ⊗ S ?S W, T mary

i

, ♦

“. . . sleeps, . . . ”

?S W, T mary

i

?W ⊗ S, ♦ S, T mary

i

T sleep

ij

W, T mary

i

W ⊗ S, T sleep

ij

“. . . snores . . . ”

S, µ(T mary

i

T sleep

ij

, T mary

i

T snore

ij

), ♦ W, T mary

i

W ⊗ S, T snore

ij

W, T mary

i

T sleep

ij

W, T mary

i

W ⊗ S, T sleep

ij

Mary who sleeps snores.

slide-41
SLIDE 41

Multilinear Algebraic Semantics for DS

O(X3, O(X1, X2)) X3 O(X1, X2) X1 X2

X1 → Ti1i2···in ∈ V1 ⊗ V2 ⊗ · · · Vn X2 → Tinin+1···in+k ∈ Vn ⊗ Vn+1 ⊗ · · · Vn+k X3 → Tin+kin+k+1···in+k+m ∈ Vn+k ⊗ Vn+k+1 ⊗ · · · Vn+k+m

Simple Nodes

slide-42
SLIDE 42

O(X3, O(X1, X2)) X3 O(X1, X2) X1 X2

Operations Nodes

O(X1, X2) → Ti1i2···inTinin+1···in+k ∈ V1 ⊗ V2 ⊗ · · · ⊗ Vn−1 ⊗ Vn+1 ⊗ · · · ⊗ Vn+k

Multilinear Algebraic Semantics for DS

slide-43
SLIDE 43

Extras

  • Ty(t) —-> S
  • Ty(e) —> W
  • ?X —-> sum or direct sum of the words and phrase

with semantics in X and their probabilities

  • —-> a neutral element such as the identity in X
  • —> a tensor full of 1’s in X

Multilinear Algebraic Semantics for DS

slide-44
SLIDE 44

? ∋

  • ? ⊗ , ♦
slide-45
SLIDE 45

? ∋

  • ? ⊗

?, ♦ ⊗ ⊗ ∋

slide-46
SLIDE 46

∋

  • , ♦

∋

  • ⊗ ∋
  • ∋
  • ⊗ ⊗ ∋
slide-47
SLIDE 47

“Babies …”

?S W, T mary

i

?W ⊗ S, ♦

T babies

i

T +

ij

Incremental Utterances

babies

slide-48
SLIDE 48

“Babies …”

?S W, T mary

i

?W ⊗ S, ♦

T +

ij =

Incremental Utterances

babies

slide-49
SLIDE 49

“Babies …”

?S W, T mary

i

?W ⊗ S, ♦

T +

ij =

T vomit + T score + T dribble + T control baby + T control milk + T control footballer + T control ball

Incremental Utterances

babies

slide-50
SLIDE 50

“Babies …”

?S W, T mary

i

?W ⊗ S, ♦

T babies

i

T +

ij

Incremental Utterances

Babies …

babies

slide-51
SLIDE 51

“Babies vomit”

T babies

i

T vomit

ij

Incremental Utterances

Babies vomit Babies …

slide-52
SLIDE 52

“Babies score”

T babies

i

T score

ij

k> (k ± epsil

Incremental Utterances

Babies … Babies score Babies vomit

slide-53
SLIDE 53

“Footballers …”

T footballersT +

Incremental Utterances

Footballers …

slide-54
SLIDE 54

“Footballers vomit”

T footballersT vomit

ij

Incremental Utterances

Footballers … Footballers vomit

slide-55
SLIDE 55

“Footballers score”

T footballers

i

T score

ij

Incremental Utterances

Footballers … Footballers vomit Footballers score

slide-56
SLIDE 56

Dataset

  • Kartsaklis D., MS, Pulman S.: Separating disambiguation from

composition in compositional distributional semantics.

  • Chose ambiguous verbs and two landmark meanings from

Pickering and Frisson 2001

  • Picked subjects and objects for landmarks using most frequently
  • ccurring ones in the BNC
slide-57
SLIDE 57
  • Pairs of subjects and complete sentences

(footballers … , footballers dribble milk) (footballers … , footballers dribble ball)

  • Pairs of subject+verb and complete sentences:

(footballers dribble … , footballers dribble milk) (footballers dribble … , footballers dribble ball)

  • Pairs of complete sentences:

(footballers dribble milk , footballers dribble ball) (babies dribble milk , babies dribble ball)

Dataset

slide-58
SLIDE 58

Data

Vectors: 300 Dim from Word2Vec, Tensors: the G&S EMNLP 2011 method

slide-59
SLIDE 59

footballers control

0.086

footballers …

0.049

footballers drip

Just subject

Data

slide-60
SLIDE 60

footballers dribble ball

0.0046

footballers …

0.0019

footballers dribble milk

Just Subject

Data

slide-61
SLIDE 61

footballers dribble ball

0.22

footballers dribble …

0.02

footballers dribble milk

Subject + Verb

Data

slide-62
SLIDE 62

footballers drip ball

0.22

footballers dribble ball

0.36

0.22 < 0.36

footballers control ball

Complete Sentences

Data

slide-63
SLIDE 63

babies dribble milk babies drip milk

0.34

babies control milk

0.32

0.34 > 0.32 Complete Sentences

Data

slide-64
SLIDE 64

1 1.5 2 2.5 3 0.4 0.45 0.5 0.55 0.6 0.65 G&S copy-subj copy-obj add

Partiality Accuracy Subj Subj+ Verb Subj+ Verb+Obj copy-obj copy-subj add

Accuracy Results

slide-65
SLIDE 65
slide-66
SLIDE 66

Work in Progress

Implement the plausibilities model of Clark 2013, Polajnar et al 2015 … under way … Extend it to experimental expectation predication … Incremental Understanding of Dialogue Content

slide-67
SLIDE 67

…

Categorical Semantics

CCC FVect

functor

CCC + biproducts FVect

?

slide-68
SLIDE 68

filleri ⌦ role

A: Thank … B: … you!