[PPT] - Competence and Performance Grammar in Incremental Parsing Vincenzo PowerPoint Presentation

SLIDE 1

1

Competence and Performance Grammar in Incremental Parsing

Patrick Sturt

Department of Psychology University of Glasgow

Vincenzo Lombardo

Dipartimento di Informatica Università di Torino

Alessando Mazzei

Dipartimento di Informatica Università di Torino

SLIDE 2

2

Idea

Defining a grammatical formalism that explicitly takes in account of the incrementality of natural language.

SLIDE 3

3

Idea

Defining a grammatical formalism that explicitly takes in account of the incrementality of natural language.

w1 ... wn wn+1wn+2

SLIDE 4

4

Motivations

Psycholinguistics:

Experimental data on the connectivity of partial syntactic structures:

 Connection of the words before the verb in Japanese

[Kamide-et-al03]

 Incremental syntactic interpretation in English [Sturt-

Lombardo04].

SLIDE 5

5

Motivations

Psycholinguistics:

Experimental data on the connectivity of partial syntactic structures:

 Connection of the words before the verb in Japanese

[Kamide-et-al03]

 Incremental syntactic interpretation in English [Sturt-

Lombardo04].

Theoretical syntax:

Deep relation between constituency and

incrementality [Phillips03].

SLIDE 6

6

Motivations

Psycholinguistics:

Experimental data on the connectivity of partial syntactic structures:

 Connection of the words before the verb in Japanese

[Kamide-et-al03]

 Incremental syntactic interpretation in English [Sturt-

Lombardo04].

Theoretical syntax:

Deep relation between constituency and

incrementality [Phillips03].

Practical Motivations

Language modeling [Roark01] Interpretation of prefix sentences [Milward95]

SLIDE 7

7

Outline

Definition of strong connectivity
Dynamic version of LTAG: DVTAG
Left association
Empirical tests on left association
Limitations and open issues

SLIDE 8

8

Strong connectivity

People incorporate each word into a single, totally connected "syntactic structure" before any further words follow [Stabler94].

SLIDE 9

9

Strong connectivity

People incorporate each word into a single, totally connected "syntactic structure" before any further words follow [Stabler94]. In this discussion: Syntactic structure Constituency tree

SLIDE 10

10

Overview of LTAG

S NP↓ VP V

pleases NP↓

NP N

Bill

NP N

Sue

VP ADV

ften

VP* S VP V

pleases

VP ADV

ften

NP N

Bill

NP N

Sue

SLIDE 11

11

Dynamic Syntax with TAG

State
Transition

Wi+1 :

C(_q)

wi+1

C*(_q)

Si+1

C(_k)

wi+1

B(_k) A(wi)

wi

C(_k) D↓(_k) E↓(_j) B(_k) A(wi)

wi

C(_k) D↓(_k) E↓(_j)

Si

SLIDE 12

12

DVTAG lexicon

Left-anchor and

lexical projection

Predicted nodes
Each non terminal

node is augmented with a head-variable B(_k) A(wi)

wi

C(_k) D↓(_k) E↓(_j)

SLIDE 13

13

DVTAG lexicon

Left-anchor and

lexical projection

Predicted nodes
Each non terminal

node is augmented with a head-variable B(_k) A(wi)

wi

C(_k) D↓(_k) E↓(_j)

SLIDE 14

14

DVTAG lexicon

Left-anchor and

lexical projection

Predicted nodes
Each non terminal

node is augmented with a head-variable B(_k) A(wi)

wi

C(_k) D↓(_k) E↓(_j)

SLIDE 15

15

Constraints on DVTAG derivation

B(_k) A(wi)

wi

C(_k) D↓(_k) E↓(_j)

Accessibility Fringe

SLIDE 16

16

Constraints on DVTAG derivation

B(_k) A(wi)

wi

C(_k) D↓(_k) E↓(_j)

Accessibility Fringe

SLIDE 17

17

Constraints on DVTAG derivation

B(_k) A(wi)

wi

C(_k) D↓(_k) E↓(_j) E(wq) wq

Accessibility Fringe

SLIDE 18

18

DVTAG example

Sue often pleases Bill

SLIDE 19

19

DVTAG example

Sue

SLIDE 20

20

DVTAG example

S(_i) NP(Sue)

Sue

N VP(_i) V(_i)

eats likes pleases ...

NP↓(_j) Sue

SLIDE 21

21

DVTAG example

S(_i) NP(Sue)

Sue

N VP(_i) V(_i)

eats likes pleases ...

NP↓(_j) Sue often

SLIDE 22

22

DVTAG example

S(_i) NP(Sue)

Sue

N VP(_i) V(_i)

eats likes pleases ...

NP↓(_j) VP(_k) ADV(often)

ften

VP*(_k) Sue often

SLIDE 23

23

DVTAG example

S(_i) NP(Sue)

Sue

N VP(_i) V(_i)

eats likes pleases ...

NP↓(_j) VP(_k) ADV(often)

ften

VP*(_k)

Adjoining from the left

Sue often

SLIDE 24

24

DVTAG example

S(_i) NP(Sue)

Sue

N VP(_i) V(_i)

eats likes pleases ...

NP↓(_j) VP(_i) ADV(often)

ften

Sue often

SLIDE 25

25

DVTAG example

S(_i) NP(Sue)

Sue

N VP(_i) V(_i)

eats likes pleases ...

NP↓(_j) VP(_i) ADV(often)

ften

Sue often pleases

SLIDE 26

26

DVTAG example

S(_i) NP(Sue)

Sue

N VP(_i) V(_i)

eats likes pleases ...

NP↓(_j) VP(_i) ADV(often)

ften

Shift

Sue often pleases

SLIDE 27

27

DVTAG example

S(pleases) NP(Sue)

Sue

N VP(pleases) V(pleases)

pleases

NP↓(_j) VP(pleases) ADV(often)

ften

Sue often pleases

SLIDE 28

28

DVTAG example

S(pleases) NP(Sue)

Sue

N VP(pleases) V(pleases)

pleases

NP↓(_j) VP(pleases) ADV(often)

ften

Sue often pleases Bill

SLIDE 29

29

DVTAG example

S(pleases) NP(Sue)

Sue

N VP(pleases) V(pleases)

pleases

NP↓(_j) VP(pleases) ADV(often)

ften

NP(Bill) N

Bill

Sue often pleases Bill

SLIDE 30

30

DVTAG example

S(pleases) NP(Sue)

Sue

N VP(pleases) V(pleases)

pleases

NP↓(_j) VP(pleases) ADV(often)

ften

NP(Bill) N

Bill Substitution

Sue often pleases Bill

SLIDE 31

31

DVTAG example

S(pleases) NP(Sue)

Sue

N VP(pleases) V(pleases)

pleases

VP(pleases) ADV(often)

ften

NP(Bill) N

Bill

Sue often pleases Bill

SLIDE 32

32

Left association

Constraints on introduction of predicted nodes
Produce a DVTAG lexicon from a LTAG lexicon

(1) STEP-1: Iteration of off-line Substitution and Adjoining on the left-side of the LTAG trees (2) STEP-2: Template trees

SLIDE 33

33

Left association

AP(_k) AP*(_k) ADV(very)

very

ADVP(very) (1) STEP-1: Iteration of off-line Substitution and Adjoining on the left-side of the LTAG trees

SLIDE 34

34

Left association

AP(_k) AP*(_k) ADV(very)

very

ADVP(very) N'*(_j) N'(_j) ADJ(nice)

nice

AP(nice) (1) STEP-1: Iteration of off-line Substitution and Adjoining on the left-side of the LTAG trees

SLIDE 35

35

Left association

N(cats)

cats

N'(cats) NP(cats) AP(_k) AP*(_k) ADV(very)

very

ADVP(very) N'*(_j) N'(_j) ADJ(nice)

nice

AP(nice) (1) STEP-1: Iteration of off-line Substitution and Adjoining on the left-side of the LTAG trees

SLIDE 36

36

Left association

(1) STEP-1: Iteration of off-line Substitution and Adjoining on the left-side of the LTAG trees N(cats)

cats

N'(cats) NP(cats) N'(cats) AP(nice) ADV(very)

very

ADVP(very) ADJ(nice)

nice

AP(nice)

SLIDE 37

37

Left association

N(cats)

cats N'(cats) NP(cats) N'(cats) AP(nice) ADV(very) very

ADVP(very)

ADJ(nice) nice AP(nice)

(2) STEP-2: Template trees

N(_i)

N'(_i) NP(_i) N'(_i) AP(_j) ADV(very) very

ADVP(very)

ADJ(_j) AP(_j)

cats dogs ... nice small ...

SLIDE 38

38

Empirical tests on left association

Left association on a wide coverage LTAG

– Closure on left association. – Termination condition: no repetitions of the

same root Xi≠Xj

Xn X1

SLIDE 39

39

Test 1

XTAG sample (English), 628 templates

Result: 176,190 templates

– The max number of left associations is 7 – Maximum number of template occurrences

(62.970) with 4 left associations

In previous experiments on Penn treebank

maximum depth 4 [Lombardo-Sturt02b].

SLIDE 40

40

Test 2

Italian treebank grammar, 988 templates

Treebank filter: only left-associated templates

present in the treebank.

Result: 706,866 templates. The max number of

left associations is 3.

SLIDE 41

41

Conclusions

DVTAG account of strong connectivity
Left association to produce wide coverage

DVTAG

Empirical naive tests

SLIDE 42

42

Limitations and open issues

Producing a DVTAG with left association is

computationally intensive.

How does the lexicon size affect the parsing

complexity in DVTAG?

Do we need underspecification technique in a

real context (es. [Roark01]) ?

SLIDE 43

43

Thank you.

SLIDE 44

44

Reference

[Kamide-et-al03] Y. Kamide, G. T.M. Altmann, and S. L.

Haywood. 2003. The time-course of prediction in

incremental sentence processing: Evidence from anticipatory eye movements. In Journal of Memory and Language, 49. [Lombardo-Sturt02] V. Lombardo and P. Sturt. 2002c. Towards a dynamic version of TAG. In TAG+6. [Lombardo-Sturt02b] V. Lombardo and P. Sturt.2002 Incrementality and Lexicalism: a Treebank Study. In The Lexical Basis of Sentence Processing. [Milward1995] D. Milward. 1995. Incremental interpretation of categorial grammar. In Proceedings

f EACL95.

SLIDE 45

45

Reference

[Phillips03] C. Phillips. 2003. Linear order and

constituency. In Linguistic Inquiry, 34.

[Stabler94] E. P. Stabler. 1994. The finite connectivity of linguistic structure. In Perspectives on Sentence Processing. [Sturt-Lombardo04] Sturt, P. and Lombardo, V. (2004). The time-course of processing of coordinate

sentences. Poster presented at the 17th annual CUNY

Sentence Processing Conference. [Roark2001] B. Roark. 2001. Probabilistic top-down parsing and language modeling. In Computational Linguistics, 27(1).

SLIDE 46

46

Lexicalized Tree Adjoining Grammars

Extended domain of locality
Recursion Factorization by adjoining operation
Lexicalization

SLIDE 47

47

LTAG and incrementality

Competence level: For each step of the derivation, several elementary trees are inserted in the same time [Vijay-Shanker87]. S NP↓ VP V

pleases NP↓

NP N

Bill

NP N

Sue

VP ADV

ften

VP* S VP V

pleases

VP ADV

ften

NP N

Bill

NP N

Sue

SLIDE 48

48

LTAG and incrementality

Performance level: Earley parser builds totally connected tree, but bottom-up information from the lexical input is weak. S NP↓ VP V

pleases NP↓

NP N

Bill

NP N

Sue

VP ADV

ften

VP*

3 4 6 5 9 10 11 12 22 21 16 17 19 18 1 2 7 8 13 14 15 20 23 24

SLIDE 49

49

Inverse operations

S(thinks) NP(Sue)

Sue

N VP(thinks) V(thinks)

thinks

S*(_j)

Sue thinks

S(_i) NP(Bill)

Bill

N VP(_i) V(_i)

eats likes pleases ...

NP↓(_j)

Bill ... Inverse adjoining from the left