Grammar Implementation with Lexicalized Tree Adjoining Grammars and - - PowerPoint PPT Presentation

▶
grammar implementation with lexicalized tree adjoining
SMART_READER_LITE
LIVE PREVIEW

Grammar Implementation with Lexicalized Tree Adjoining Grammars and - - PowerPoint PPT Presentation

Jul 11, 2023 111 likes •1.29k views

Grammar Implementation with Lexicalized Tree Adjoining Grammars and Frame Semantics Introduction Laura Kallmeyer, Timm Lichte, Rainer Osswald & Simon Petitjean University of Dsseldorf DGfS CL Fall School, September 11, 2017 SFB 991 What

slide-1
SLIDE 1

Grammar Implementation with Lexicalized Tree Adjoining Grammars and Frame Semantics

Introduction Laura Kallmeyer, Timm Lichte, Rainer Osswald & Simon Petitjean

University of Düsseldorf

DGfS CL Fall School, September 11, 2017

SFB 991

slide-2
SLIDE 2

What this course is about

“Working with Tree-Adjoining Grammar” Who is “working”?

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 2 2

slide-3
SLIDE 3

What this course is about

“Working with Tree-Adjoining Grammar” Who is “working”? The linguist!

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 3 2

slide-4
SLIDE 4

What this course is about

“Working with Tree-Adjoining Grammar” Who is “working”? The linguist! On what?

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 4 2

slide-5
SLIDE 5

What this course is about

“Working with Tree-Adjoining Grammar” Who is “working”? The linguist! On what? Trying to implement1/2 syntactic theories!

implement1: general concepts → mathematical objects implement2: paper & pencil → electronic resource

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 5 2

slide-6
SLIDE 6

What this course is about

“Working with Tree-Adjoining Grammar” Who is “working”? The linguist! On what? Trying to implement1/2 syntactic theories!

implement1: general concepts → mathematical objects implement2: paper & pencil → electronic resource

Why implementation? As is frequently pointed out but cannot be overemphasized, an im- portant goal of formalization in linguistics is to enable subsequent researchers to see the defects of an analysis as clearly as its merits;

  • nly then can progress be made efficiently. (Dowty (1979): 322)

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 6 2

slide-7
SLIDE 7

What this course is about

“Working with Tree-Adjoining Grammar” Who is “working”? The linguist! On what? Trying to implement1/2 syntactic theories!

implement1: general concepts → mathematical objects implement2: paper & pencil → electronic resource

Why implementation? As is frequently pointed out but cannot be overemphasized, an im- portant goal of formalization in linguistics is to enable subsequent researchers to see the defects of an analysis as clearly as its merits;

  • nly then can progress be made efficiently. (Dowty (1979): 322)

incentive for rigor check for consistency application (NLP)

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 7 2

slide-8
SLIDE 8

What this course is about: Observations and theory

Observations Theory

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 8 3

slide-9
SLIDE 9

What this course is about: Observations and theory

Observations set of pairs uterance , meaning Theory

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 9 3

slide-10
SLIDE 10

What this course is about: Observations and theory

Observations set of pairs uterance , meaning Theory “Grammar”, that is formalized efficient (i.e., not NP-hard) comprehensible (i.e., no black-box technology) (maybe characteristic for computational linguistics)

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 10 3

slide-11
SLIDE 11

What this course is about: Observations and theory

Observations (more specific) set of pairs sentence , semantic frame Theory (more specific) “Grammar”, that is formalized efficient (i.e., not NP-hard) comprehensible (i.e., no black-box technology) ⇒ based on Tree-Adjoining Grammar (TAG) and Frame Semantics

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 11 4

slide-12
SLIDE 12

What this course is about: Example 1

A simple example: tree composition in LTAG (1) Adam always ate an apple.

NP ‘Adam’ S VP NP V ‘ate’ NP NP ‘an apple’ VP VP∗ Adv ‘always’

  • S

VP VP NP ‘an apple’ V ‘ate’ Adv ‘always’ NP ‘Adam’

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 12 5

slide-13
SLIDE 13

What this course is about: Example 2

A simple example: tree-frame pairs and composition

(2) Adam ate an apple.

S VP[I=e] NP[I=y] V ‘ate’ NP[I=x] e          eating actor x theme y         

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 13 6

slide-14
SLIDE 14

What this course is about: Example 2

A simple example: tree-frame pairs and composition

(2) Adam ate an apple.

NP[I=u] ‘Adam’ u       person name ‘Adam’       S VP[I=e] NP[I=y] V ‘ate’ NP[I=x] e          eating actor x theme y         

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 14 6

slide-15
SLIDE 15

What this course is about: Example 2

A simple example: tree-frame pairs and composition

(2) Adam ate an apple.

NP[I=u] ‘Adam’ u       person name ‘Adam’       S VP[I=e] NP[I=y] V ‘ate’ NP[I=x] e          eating actor x theme y          x u

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 15 6

slide-16
SLIDE 16

What this course is about: Example 2

A simple example: tree-frame pairs and composition

(2) Adam ate an apple.

NP[I=u] ‘Adam’ u       person name ‘Adam’       S VP[I=e] NP[I=y] V ‘ate’ NP[I=x] e          eating actor x theme y          NP[I=v] ‘an apple’ v

  • apple
  • x u

y v

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 16 6

slide-17
SLIDE 17

What this course is about: Example 2

A simple example: tree-frame pairs and composition

(2) Adam ate an apple.

NP[I=u] ‘Adam’ u       person name ‘Adam’       S VP[I=e] NP[I=y] V ‘ate’ NP[I=x] e          eating actor x theme y          NP[I=v] ‘an apple’ v

  • apple
  • x u

y v S VP[I=e] NP[I=y] ‘an apple’ V ‘ate’ NP[I=x] ‘Adam’ e              eating actor x       person name ‘Adam’       theme y

  • apple

           

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 17 6

slide-18
SLIDE 18

What this course is about: Example 2

A simple example: tree-frame pairs and composition

(2) Adam ate an apple.

NP[I=u] ‘Adam’ u       person name ‘Adam’       S VP[I=e] NP[I=y] V ‘ate’ NP[I=x] e          eating actor x theme y          NP[I=v] ‘an apple’ v

  • apple
  • x u

y v S VP[I=e] NP[I=y] ‘an apple’ V ‘ate’ NP[I=x] ‘Adam’ e              eating actor x       person name ‘Adam’       theme y

  • apple

            e eating x person ‘Adam’ y apple actor name theme

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 18 6

slide-19
SLIDE 19

This week

Mon: introduction to LTAG Tue: syntactic analyses with LTAG I, derivation trees, feature structures Wed: syntactic analyses with LTAG II, introduction to LTAG semantics Thu: introduction to frame semantics Fri: puting things together (will be taught by Laura Kallmeyer and Rainer Osswald)

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 19 7

slide-20
SLIDE 20

Next week

Mon: introduction to grammar engineering and XMG Tue: implementing syntax with XMG Wed: implementing semantics with XMG Thu: parsing implemented grammars with TuLiPA Fri: Conclusion (will be taught by Timm Lichte and Simon Petitjean) Some of the slides are taken from Kata Balogh and Timm Lichte’s ESSLLI 2015 course.

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 20 8

slide-21
SLIDE 21

Outline of today’s course

1

Why “working” with TAG? Formal reasons Linguistic reasons

2

From CFG to TAG Context-Free Grammars Lexicalization Tree Substitution Grammars (TSG) Adding adjunction

3

Further related formalisms

4

Summary & outlook

5

Appendix: NL and the generative capacity of grammar formalisms

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 21 9

slide-22
SLIDE 22

Outline of today’s course

1

Why “working” with TAG? Formal reasons Linguistic reasons

2

From CFG to TAG Context-Free Grammars Lexicalization Tree Substitution Grammars (TSG) Adding adjunction

3

Further related formalisms

4

Summary & outlook

5

Appendix: NL and the generative capacity of grammar formalisms

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 22 10

slide-23
SLIDE 23

Why “working” with TAG? Formal reasons

Hypothesis of the adequacy of expressive power TAG exactly provides the expressive power needed to treat NL. (The complexity of a language is determined by the weakest formal grammar that generates it.)

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 23 11

slide-24
SLIDE 24

Why “working” with TAG? Formal reasons

Hypothesis of the adequacy of expressive power TAG exactly provides the expressive power needed to treat NL. (The complexity of a language is determined by the weakest formal grammar that generates it.) Why is the formal complexity of natural languages interesting? It allows one to gain insights into ⇒ the general structure of natural language ⇒ the general human language capacity ⇒ the adequacy of grammar formalisms ⇒ lower bound of the computational complexity of NLP tasks

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 24 11

slide-25
SLIDE 25

Why “working” with TAG? Formal reasons

Expressive power in terms of a specific generative capacity: Weak generative capacity (WGC) The capacity to generate string languages. Strong generative capacity (SGC) The capacity to generate tree languages. Derivational generative capacity (DGC) The capacity to generate string languages in a certain way. In what follows we will consider the weak generative capacity.

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 25 12

slide-26
SLIDE 26

Why “working” with TAG? Formal reasons

How much expressive power do we need to treat NL?

HPSG, TG, TM a f (n) LFG, LBA a2n, anbncn..., W k CFG, PDA anbmcmdn, WW R FSA anbmckdl type 3: regular type 2: context-free type 1: context-sensitive type 0: recursively enumerable

Chomsky(-Schützenberger) hierarchy

(Chomsky & Schützenberger 1963)

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 26 13

slide-27
SLIDE 27

Why “working” with TAG? Formal reasons

How much expressive power do we need to treat NL?

HPSG, TG, TM a f (n) LFG, LBA a2n, anbncn..., W k CFG, PDA anbmcmdn, WW R FSA anbmckdl type 3: regular type 2: context-free type 1: context-sensitive type 0: recursively enumerable

Chomsky(-Schützenberger) hierarchy

(Chomsky & Schützenberger 1963)

NL is not regular!

(Chomsky 1956; 1957)

center embedding with rela- tive clauses

n1 n2 n3 v3 v2 v1

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 27 13

slide-28
SLIDE 28

Why “working” with TAG? Formal reasons

How much expressive power do we need to treat NL?

HPSG, TG, TM a f (n) LFG, LBA a2n, anbncn..., W k CFG, PDA anbmcmdn, WW R FSA anbmckdl type 3: regular type 2: context-free type 1: context-sensitive type 0: recursively enumerable

Chomsky(-Schützenberger) hierarchy

(Chomsky & Schützenberger 1963)

NL is not context-free!

Shieber (1985)

cross serial dependencies in Dutch and Swiss-German

n1 n2 n3 v1 v2 v3

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 28 13

slide-29
SLIDE 29

Why “working” with TAG? Formal reasons

How much expressive power do we need to treat NL?

HPSG, TG, TM a f (n) LFG, LBA a2n, anbncn..., W k CFG, PDA anbmcmdn, WW R FSA anbmckdl type 3: regular type 2: context-free type 1: context-sensitive type 0: recursively enumerable

Chomsky(-Schützenberger) hierarchy

(Chomsky & Schützenberger 1963)

NL is context-sensitive?

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 29 13

slide-30
SLIDE 30

Why “working” with TAG? Formal reasons

How much expressive power do we need to treat NL?

HPSG, TG, TM af (n) LFG, LBA a2n, anbncn..., W k TAG, EPDA anbmcndm, WW CFG, PDA anbmcmdn, WW R FSA anbmckdl type 3: regular type 2: context-free type 1: context-sensitive mildly context-sensitive type 0: recursively enumerable

Chomsky(-Schützenberger) hierarchy

(Chomsky & Schützenberger 1963)

NL is mildly context- sensitive? (Joshi 1985) ⊃ CFL cross-serial dep. semi-linear in PTIME

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 30 13

slide-31
SLIDE 31

Why “working” with TAG? Formal reasons

How much expressive power do we need to treat NL?

HPSG, TG, TM af (n) LFG, LBA a2n, W (#W)k anbncn... W k TAG, EPDA anbmcndm, W#W WW CFG, PDA anbmcmdn, W#W R WW R FSA anbmckdl type 3: regular type 2: context-free type 1: context-sensitive mildly context-sensitive type 0: recursively enumerable

Chomsky(-Schützenberger) hierarchy

(Chomsky & Schützenberger 1963)

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 31 13

slide-32
SLIDE 32

Why “working” with TAG? Linguistic reasons

extended domain of locality

S NP VP V repaired NP

long-distance dependencies / discontinuous constituents (3) Who did Mary say that Tom claimed ...repaired the fridge? multi-word expressions (4) to kick the bucket (‘to die’) incarnation of Construction Grammar

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 32 14

slide-33
SLIDE 33

Outline of today’s course

1

Why “working” with TAG? Formal reasons Linguistic reasons

2

From CFG to TAG Context-Free Grammars Lexicalization Tree Substitution Grammars (TSG) Adding adjunction

3

Further related formalisms

4

Summary & outlook

5

Appendix: NL and the generative capacity of grammar formalisms

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 33 15

slide-34
SLIDE 34

From CFG to TAG: Context-Free Grammar

string rewriting replace non-terminals by strings of terminals and non-terminals GCFG = N, T, S, P P = { S → NP VP VP → V NP | AP VP NP → N | Det N AP → A N → Peter | fridge Det → the A → easily V → repaired }

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 34 16

slide-35
SLIDE 35

From CFG to TAG: Context-Free Grammar

string rewriting replace non-terminals by strings of terminals and non-terminals GCFG = N, T, S, P P = { S → NP VP VP → V NP | AP VP NP → N | Det N AP → A N → Peter | fridge Det → the A → easily V → repaired } Example derivation : S

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 35 16

slide-36
SLIDE 36

From CFG to TAG: Context-Free Grammar

string rewriting replace non-terminals by strings of terminals and non-terminals GCFG = N, T, S, P P = { S → NP VP VP → V NP | AP VP NP → N | Det N AP → A N → Peter | fridge Det → the A → easily V → repaired } Example derivation : NP VP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 36 16

slide-37
SLIDE 37

From CFG to TAG: Context-Free Grammar

string rewriting replace non-terminals by strings of terminals and non-terminals GCFG = N, T, S, P P = { S → NP VP VP → V NP | AP VP NP → N | Det N AP → A N → Peter | fridge Det → the A → easily V → repaired } Example derivation : N VP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 37 16

slide-38
SLIDE 38

From CFG to TAG: Context-Free Grammar

string rewriting replace non-terminals by strings of terminals and non-terminals GCFG = N, T, S, P P = { S → NP VP VP → V NP | AP VP NP → N | Det N AP → A N → Peter | fridge Det → the A → easily V → repaired } Example derivation : Peter VP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 38 16

slide-39
SLIDE 39

From CFG to TAG: Context-Free Grammar

string rewriting replace non-terminals by strings of terminals and non-terminals GCFG = N, T, S, P P = { S → NP VP VP → V NP | AP VP NP → N | Det N AP → A N → Peter | fridge Det → the A → easily V → repaired } Example derivation : Peter AP VP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 39 16

slide-40
SLIDE 40

From CFG to TAG: Context-Free Grammar

string rewriting replace non-terminals by strings of terminals and non-terminals GCFG = N, T, S, P P = { S → NP VP VP → V NP | AP VP NP → N | Det N AP → A N → Peter | fridge Det → the A → easily V → repaired } Example derivation : Peter A VP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 40 16

slide-41
SLIDE 41

From CFG to TAG: Context-Free Grammar

string rewriting replace non-terminals by strings of terminals and non-terminals GCFG = N, T, S, P P = { S → NP VP VP → V NP | AP VP NP → N | Det N AP → A N → Peter | fridge Det → the A → easily V → repaired } Example derivation : Peter easily VP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 41 16

slide-42
SLIDE 42

From CFG to TAG: Context-Free Grammar

string rewriting replace non-terminals by strings of terminals and non-terminals GCFG = N, T, S, P P = { S → NP VP VP → V NP | AP VP NP → N | Det N AP → A N → Peter | fridge Det → the A → easily V → repaired } Example derivation : Peter easily V NP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 42 16

slide-43
SLIDE 43

From CFG to TAG: Context-Free Grammar

string rewriting replace non-terminals by strings of terminals and non-terminals GCFG = N, T, S, P P = { S → NP VP VP → V NP | AP VP NP → N | Det N AP → A N → Peter | fridge Det → the A → easily V → repaired } Example derivation : Peter easily repaired NP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 43 16

slide-44
SLIDE 44

From CFG to TAG: Context-Free Grammar

string rewriting replace non-terminals by strings of terminals and non-terminals GCFG = N, T, S, P P = { S → NP VP VP → V NP | AP VP NP → N | Det N AP → A N → Peter | fridge Det → the A → easily V → repaired } Example derivation : Peter easily repaired Det N

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 44 16

slide-45
SLIDE 45

From CFG to TAG: Context-Free Grammar

string rewriting replace non-terminals by strings of terminals and non-terminals GCFG = N, T, S, P P = { S → NP VP VP → V NP | AP VP NP → N | Det N AP → A N → Peter | fridge Det → the A → easily V → repaired } Example derivation : Peter easily repaired the N

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 45 16

slide-46
SLIDE 46

From CFG to TAG: Context-Free Grammar

string rewriting replace non-terminals by strings of terminals and non-terminals GCFG = N, T, S, P P = { S → NP VP VP → V NP | AP VP NP → N | Det N AP → A N → Peter | fridge Det → the A → easily V → repaired } Example derivation : Peter easily repaired the fridge

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 46 16

slide-47
SLIDE 47

From CFG to TAG: Context-Free Grammar

string rewriting replace non-terminals by strings of terminals and non-terminals GCFG = N, T, S, P P = { S → NP VP VP → V NP | AP VP NP → N | Det N AP → A N → Peter | fridge Det → the A → easily V → repaired } Example derivation history:

S NP N Peter VP AP A easily VP V repaired NP Det the N fridge

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 47 16

slide-48
SLIDE 48

From CFG to TAG: Context-Free Grammar

Why not stick to CFGs (literally)? low generative capacity: cannot describe all NL phenomena; e.g. cross-serial dependencies (anbmcndm) Swiss German (Shieber 1985) duplication (w#w) Bambara (Culy 1985) multiple agreement (anbncn) Bantu languages

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 48 17

slide-49
SLIDE 49

From CFG to TAG: Context-Free Grammar

Why not stick to CFGs (literally)? low generative capacity: cannot describe all NL phenomena; e.g. cross-serial dependencies (anbmcndm) Swiss German (Shieber 1985) duplication (w#w) Bambara (Culy 1985) multiple agreement (anbncn) Bantu languages poor support of expressing linguistic generalizations Rules have a very limited domain of locality. ( no strong lexicalization) atomic non-terminals ( massive proliferation)

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 49 17

slide-50
SLIDE 50

From CFG to TAG: Context-Free Grammar

Why not stick to CFGs (literally)? low generative capacity: cannot describe all NL phenomena; e.g. cross-serial dependencies (anbmcndm) Swiss German (Shieber 1985) duplication (w#w) Bambara (Culy 1985) multiple agreement (anbncn) Bantu languages poor support of expressing linguistic generalizations Rules have a very limited domain of locality. ( no strong lexicalization) atomic non-terminals ( massive proliferation) First step: turn strings into trees!

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 50 17

slide-51
SLIDE 51

Lexicalization

lexicalization → each structure of the grammar has at least one non-terminal

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 51 18

slide-52
SLIDE 52

Lexicalization

lexicalization → each structure of the grammar has at least one non-terminal Lexicalized grammar A lexicalized grammar consists of: (i) a finite set of structures each associated with a lexical item (anchor); and (ii) operation(s) for composing these structures.

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 52 18

slide-53
SLIDE 53

Lexicalization

lexicalization → each structure of the grammar has at least one non-terminal Lexicalized grammar A lexicalized grammar consists of: (i) a finite set of structures each associated with a lexical item (anchor); and (ii) operation(s) for composing these structures. Lexicalization A formalism F can be lexicalized by another formalism F ′, if for any finitely ambiguous grammar G in F, there is a grammar G′ in F ′, such that (i) G′ is a lexicalized grammar; and (ii) G and G′ generate the same tree set.

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 53 18

slide-54
SLIDE 54

Lexicalization

lexicalization → each structure of the grammar has at least one non-terminal Lexicalized grammar A lexicalized grammar consists of: (i) a finite set of structures each associated with a lexical item (anchor); and (ii) operation(s) for composing these structures. Lexicalization A formalism F can be lexicalized by another formalism F ′, if for any finitely ambiguous grammar G in F, there is a grammar G′ in F ′, such that (i) G′ is a lexicalized grammar; and (ii) G and G′ generate the same tree set. weak vs. strong lexicalization weak lexicalization: preserve the string language strong lexicalization: preserve the tree structure

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 54 18

slide-55
SLIDE 55

Lexicalization

Formally interesting:

a finite lexicalized grammar provides finitely many analyses for each string (finitely ambiguous)

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 55 19

slide-56
SLIDE 56

Lexicalization

Formally interesting:

a finite lexicalized grammar provides finitely many analyses for each string (finitely ambiguous)

Linguistically interesting:

syntactic properties of lexical items can be accounted for more directly each lexical item comes with the possibility of certain partial syntactic constructions

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 56 19

slide-57
SLIDE 57

Lexicalization

Formally interesting:

a finite lexicalized grammar provides finitely many analyses for each string (finitely ambiguous)

Linguistically interesting:

syntactic properties of lexical items can be accounted for more directly each lexical item comes with the possibility of certain partial syntactic constructions

Computationally interesting:

the search space during parsing can be delimited (grammar filtering)

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 57 19

slide-58
SLIDE 58

Lexicalization of CFG’s

Greibach normal-form (Greibach 1965): A → aX or A → a (a ∈ VT; A ∈ VN; X ∈ (VN)∗)

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 58 20

slide-59
SLIDE 59

Lexicalization of CFG’s

Greibach normal-form (Greibach 1965): A → aX or A → a (a ∈ VT; A ∈ VN; X ∈ (VN)∗) example:

a CFG G: S → SS, S → a

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 59 20

slide-60
SLIDE 60

Lexicalization of CFG’s

Greibach normal-form (Greibach 1965): A → aX or A → a (a ∈ VT; A ∈ VN; X ∈ (VN)∗) example:

a CFG G: S → SS, S → a lexicalize G ⇒ G′ (Greibach): S → aS, S → a

same string language, but not the same tree set

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 60 20

slide-61
SLIDE 61

Lexicalization of CFG’s

Greibach normal-form (Greibach 1965): A → aX or A → a (a ∈ VT; A ∈ VN; X ∈ (VN)∗) example:

a CFG G: S → SS, S → a lexicalize G ⇒ G′ (Greibach): S → aS, S → a

same string language, but not the same tree set by G: by G′:

S S S a S a S S a S a S a S a S a S a

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 61 20

slide-62
SLIDE 62

Lexicalization of CFG’s

Greibach normal-form (Greibach 1965): A → aX or A → a (a ∈ VT; A ∈ VN; X ∈ (VN)∗) example:

a CFG G: S → SS, S → a lexicalize G ⇒ G′ (Greibach): S → aS, S → a

same string language, but not the same tree set by G: by G′:

S S S a S a S S a S a S a S a S a S a

  • nly weak lexicalization possible

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 62 20

slide-63
SLIDE 63

From CFG to TAG: Tree Substitution Grammar (TSG)

First step: turn strings into trees! tree rewriting Substitution: replace a non-terminal leaf with a tree GCFG = N, T, S, P P = { S → NP VP VP → V NP | AP VP NP → N | Det N AP → A N → Peter | fridge Det → the A → easily V → repaired }

≈

GTSG = N, T, S, I I = {

S NP VP VP V NP VP AP VP NP N NP Det N AP A A easily N Peter N fridge Det the V repaired

}

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 63 21

slide-64
SLIDE 64

From CFG to TAG: Tree Substitution Grammar (TSG)

First step: turn strings into trees! tree rewriting Substitution: replace a non-terminal leaf with a tree GTSG = N, T, S, I I = {

S NP VP VP V NP VP AP VP NP N NP Det N AP A A easily N Peter N fridge Det the V repaired

} Example derivation:

S NP VP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 64 21

slide-65
SLIDE 65

From CFG to TAG: Tree Substitution Grammar (TSG)

First step: turn strings into trees! tree rewriting Substitution: replace a non-terminal leaf with a tree GTSG = N, T, S, I I = {

S NP VP VP V NP VP AP VP NP N NP Det N AP A A easily N Peter N fridge Det the V repaired

} Example derivation:

S NP N VP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 65 21

slide-66
SLIDE 66

From CFG to TAG: Tree Substitution Grammar (TSG)

First step: turn strings into trees! tree rewriting Substitution: replace a non-terminal leaf with a tree GTSG = N, T, S, I I = {

S NP VP VP V NP VP AP VP NP N NP Det N AP A A easily N Peter N fridge Det the V repaired

} Example derivation:

S NP N Peter VP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 66 21

slide-67
SLIDE 67

From CFG to TAG: Tree Substitution Grammar (TSG)

First step: turn strings into trees! tree rewriting Substitution: replace a non-terminal leaf with a tree GTSG = N, T, S, I I = {

S NP VP VP V NP VP AP VP NP N NP Det N AP A A easily N Peter N fridge Det the V repaired

} Example derivation:

S NP N Peter VP AP VP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 67 21

slide-68
SLIDE 68

From CFG to TAG: Tree Substitution Grammar (TSG)

First step: turn strings into trees! tree rewriting Substitution: replace a non-terminal leaf with a tree GTSG = N, T, S, I I = {

S NP VP VP V NP VP AP VP NP N NP Det N AP A A easily N Peter N fridge Det the V repaired

} Example derivation:

S NP N Peter VP AP A VP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 68 21

slide-69
SLIDE 69

From CFG to TAG: Tree Substitution Grammar (TSG)

First step: turn strings into trees! tree rewriting Substitution: replace a non-terminal leaf with a tree GTSG = N, T, S, I I = {

S NP VP VP V NP VP AP VP NP N NP Det N AP A A easily N Peter N fridge Det the V repaired

} Example derivation:

S NP N Peter VP AP A easily VP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 69 21

slide-70
SLIDE 70

From CFG to TAG: Tree Substitution Grammar (TSG)

First step: turn strings into trees! tree rewriting Substitution: replace a non-terminal leaf with a tree GTSG = N, T, S, I I = {

S NP VP VP V NP VP AP VP NP N NP Det N AP A A easily N Peter N fridge Det the V repaired

} Example derivation:

S NP N Peter VP AP A easily VP V NP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 70 21

slide-71
SLIDE 71

From CFG to TAG: Tree Substitution Grammar (TSG)

First step: turn strings into trees! tree rewriting Substitution: replace a non-terminal leaf with a tree GTSG = N, T, S, I I = {

S NP VP VP V NP VP AP VP NP N NP Det N AP A A easily N Peter N fridge Det the V repaired

} Example derivation:

S NP N Peter VP AP A easily VP V repaired NP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 71 21

slide-72
SLIDE 72

From CFG to TAG: Tree Substitution Grammar (TSG)

First step: turn strings into trees! tree rewriting Substitution: replace a non-terminal leaf with a tree GTSG = N, T, S, I I = {

S NP VP VP V NP VP AP VP NP N NP Det N AP A A easily N Peter N fridge Det the V repaired

} Example derivation:

S NP N Peter VP AP A easily VP V repaired NP Det N

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 72 21

slide-73
SLIDE 73

From CFG to TAG: Tree Substitution Grammar (TSG)

First step: turn strings into trees! tree rewriting Substitution: replace a non-terminal leaf with a tree GTSG = N, T, S, I I = {

S NP VP VP V NP VP AP VP NP N NP Det N AP A A easily N Peter N fridge Det the V repaired

} Example derivation:

S NP N Peter VP AP A easily VP V repaired NP Det the N

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 73 21

slide-74
SLIDE 74

From CFG to TAG: Tree Substitution Grammar (TSG)

First step: turn strings into trees! tree rewriting Substitution: replace a non-terminal leaf with a tree GTSG = N, T, S, I I = {

S NP VP VP V NP VP AP VP NP N NP Det N AP A A easily N Peter N fridge Det the V repaired

} Example derivation:

S NP N Peter VP AP A easily VP V repaired NP Det the N fridge

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 74 21

slide-75
SLIDE 75

From CFG to TAG: Tree Substitution Grammar (TSG)

TSG versus CFG: weakly equivalent (same string languages, but more tree languages)

S NP VP AP VP V repaired NP single recursion!

still no strong lexicalization of CFG, cross-serial dependencies etc. Applications of TSG: Data-Oriented Parsing (DOP, Bod 2009) Second step: add adjunction!

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 75 22

slide-76
SLIDE 76

From CFG to TAG: Adding adjunction

Adjunction: replace a non-terminal node with an “auxiliary” tree put the subtree of the replaced node under the footnode (*)

VP AP A easily VP* S NP VP V repaired NP

⇒

S NP VP AP A easily VP V repaired NP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 76 23

slide-77
SLIDE 77

From CFG to TAG: Adding adjunction

Adjunction: replace a non-terminal node with an “auxiliary” tree put the subtree of the replaced node under the footnode (*)

VP AP A easily VP* VP AP A easily VP*

⇒

VP AP A easily VP AP A easily VP*

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 77 23

slide-78
SLIDE 78

From CFG to TAG: Adding adjunction

Adjunction: replace a non-terminal node with an “auxiliary” tree put the subtree of the replaced node under the footnode (*)

VP AP A easily VP* VP AP A easily VP*

⇒

VP AP A easily VP AP A easily VP*

⇒ Adjunction at footnodes causes spurious ambiguities in derivations. ⇒ Therefore, this is usually forbidden.

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 78 23

slide-79
SLIDE 79

From CFG to TAG: Example with adjunction

tree rewriting Substitution: replace a non-terminal leaf with a tree Adjunction: replace a non-terminal node with an “auxiliary” tree GTSG = N, T, S, I I = {

S NP VP VP V NP VP AP VP NP N NP Det N AP A A easily N Peter N fridge Det the V repaired

}

≈

S NP VP repaired NP VP AP A easily VP* NP N Peter NP Det N fridge Det the

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 79 24

slide-80
SLIDE 80

From CFG to TAG: Example with adjunction

tree rewriting Substitution: replace a non-terminal leaf with a tree Adjunction: replace a non-terminal node with an “auxiliary” tree

S NP VP V repaired NP VP AP A easily VP* NP N Peter NP Det N fridge Det the

Example derivation:

S NP VP V repaired NP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 80 24

slide-81
SLIDE 81

From CFG to TAG: Example with adjunction

tree rewriting Substitution: replace a non-terminal leaf with a tree Adjunction: replace a non-terminal node with an “auxiliary” tree

S NP VP V repaired NP VP AP A easily VP* NP N Peter NP Det N fridge Det the

Example derivation:

S NP N Peter VP V repaired NP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 81 24

slide-82
SLIDE 82

From CFG to TAG: Example with adjunction

tree rewriting Substitution: replace a non-terminal leaf with a tree Adjunction: replace a non-terminal node with an “auxiliary” tree

S NP VP V repaired NP VP AP A easily VP* NP N Peter NP Det N fridge Det the

Example derivation:

S NP N Peter VP AP A easily VP V repaired NP

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 82 24

slide-83
SLIDE 83

From CFG to TAG: Example with adjunction

tree rewriting Substitution: replace a non-terminal leaf with a tree Adjunction: replace a non-terminal node with an “auxiliary” tree

S NP VP V repaired NP VP AP A easily VP* NP N Peter NP Det N fridge Det the

Example derivation:

S NP N Peter VP AP A easily VP V repaired NP Det N fridge

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 83 24

slide-84
SLIDE 84

From CFG to TAG: Example with adjunction

tree rewriting Substitution: replace a non-terminal leaf with a tree Adjunction: replace a non-terminal node with an “auxiliary” tree

S NP VP V repaired NP VP AP A easily VP* NP N Peter NP Det N fridge Det the

Example derivation:

S NP N Peter VP AP A easily VP V repaired NP Det the N fridge

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 84 24

slide-85
SLIDE 85

From CFG to TAG: Restrictions on adjunction (I)

Restrictions on the shape of auxiliary trees: The root node and the footnode must carry the same non-terminal. Specific adjunction constraints on target nodes:

  • bligatory adjunction (OA): true/false

null adjunction (NA): no adjoinable auxiliary tree selective adjunction (SA): a nonempty set of adjoinable auxiliary trees Adjunction constraints are essential in generating non-context-free languages (e.g. the copy language {ww|w ∈ {a, b}∗})!

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 85 25

slide-86
SLIDE 86

From CFG to TAG: Restrictions on adjunction (I)

Example grammar for the copy language {ww|w ∈ {a, b}∗}:

S ε SNA a S S*NA a SNA b S S*NA b

Example derivation of abbabb: ⇒ TAG = TSG + adjunction + adjunction constraints

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 86 26

slide-87
SLIDE 87

From CFG to TAG: Restrictions on adjunction (I)

Example grammar for the copy language {ww|w ∈ {a, b}∗}:

S ε SNA a S S*NA a SNA b S S*NA b

Example derivation of abbabb:

S ε

⇒ TAG = TSG + adjunction + adjunction constraints

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 87 26

slide-88
SLIDE 88

From CFG to TAG: Restrictions on adjunction (I)

Example grammar for the copy language {ww|w ∈ {a, b}∗}:

S ε SNA a S S*NA a SNA b S S*NA b

Example derivation of abbabb:

SNA a S SNA ε a

⇒ TAG = TSG + adjunction + adjunction constraints

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 88 26

slide-89
SLIDE 89

From CFG to TAG: Restrictions on adjunction (I)

Example grammar for the copy language {ww|w ∈ {a, b}∗}:

S ε SNA a S S*NA a SNA b S S*NA b

Example derivation of abbabb:

SNA a S SNA ε a

⇒ TAG = TSG + adjunction + adjunction constraints

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 89 26

slide-90
SLIDE 90

From CFG to TAG: Restrictions on adjunction (I)

Example grammar for the copy language {ww|w ∈ {a, b}∗}:

S ε SNA a S S*NA a SNA b S S*NA b

Example derivation of abbabb:

SNA a SNA b S SNA SNA ε a b

⇒ TAG = TSG + adjunction + adjunction constraints

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 90 26

slide-91
SLIDE 91

From CFG to TAG: Restrictions on adjunction (I)

Example grammar for the copy language {ww|w ∈ {a, b}∗}:

S ε SNA a S S*NA a SNA b S S*NA b

Example derivation of abbabb:

SNA a SNA b S SNA SNA ε a b

⇒ TAG = TSG + adjunction + adjunction constraints

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 91 26

slide-92
SLIDE 92

From CFG to TAG: Restrictions on adjunction (I)

Example grammar for the copy language {ww|w ∈ {a, b}∗}:

S ε SNA a S S*NA a SNA b S S*NA b

Example derivation of abbabb:

SNA a SNA b SNA b S SNA SNA SNA ε a b b

⇒ TAG = TSG + adjunction + adjunction constraints

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 92 26

slide-93
SLIDE 93

From CFG to TAG: Tree-Adjoining Grammar

A Tree Adjoining Grammar (TAG) is a tuple G = N, T, I, A, O, C: T and N are disjoint alphabets, the terminals and nonterminals, I is a finite set of intial trees, and A is a finite set of auxiliary trees. O : {v | v is a node in a tree in I ∪ A} → {1, 0} is a function, and C : {v | v is a node in a tree in I ∪ A} → P(A) is a function. Let v be a node in I ∪ A:

  • bligatory adjunction (OA): O(v) = 1

null adjunction (NA): O(v) = 0 and C(v) = ∅ selective adjunction (SA): O(v) = 0 and C(v) ∅ and C(v) A The trees in I ∪ A are called elementary trees.

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 93 27

slide-94
SLIDE 94

From CFG to TAG: Tree-Adjoining Grammar

TAG is mildly context-sensitive (MCS, Joshi 1985) generates the context-free languages generates cross-serial dependencies (i.e. WW) constant growth (or semi linear, no a2n) polynomial time parsing (O(n6)) (Schabes 1990; Joshi & Schabes 1997; Kallmeyer 2010) TAG can strongly lexicalize finitely ambiguous CFG. (Schabes 1990; Joshi & Schabes 1991) Formally interesting: a finite lexicalized grammar provides finitely many analyses for each string (finitely ambiguous). Linguistically interesting: syntactic properties of lexical items can be accounted for more directly. Computationally interesting: the search space during parsing can be delimited (grammar filtering).

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 94 28

slide-95
SLIDE 95

Outline of today’s course

1

Why “working” with TAG? Formal reasons Linguistic reasons

2

From CFG to TAG Context-Free Grammars Lexicalization Tree Substitution Grammars (TSG) Adding adjunction

3

Further related formalisms

4

Summary & outlook

5

Appendix: NL and the generative capacity of grammar formalisms

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 95 29

slide-96
SLIDE 96

Restricting TAG

Further adjunction constraints: no adjunction at the spine below the root node of auxiliary trees

  • ff-spine TAG (osTAG, Swanson et al. (2013))

⇒ WGC of CFG (O(n3)) ⇒ more compact grammars than CFG or TSG ⇒ strongly lexicalizes CFG? Restrictions on the shape of auxiliary trees: Footnodes are at the lef or right edge of an ET. Tree Insertion Grammar (TIG, Schabes & Waters (1995)) further constraint: no adjunction of lef auxiliary trees to the spine of right auxiliary trees ⇒ WGC of CFG (O(n3)) ⇒ more compact grammars than CFG (or TSG?) ⇒ strongly lexicalizes CFG

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 96 30

slide-97
SLIDE 97

MCS-alternatives to TAG and extensions

Linear Indexed Grammar (LIG Gazdar 1988; Keller & Weir 1995) Head Grammar (HG Pollard 1984) Multicomponent TAG (MCTAG Seki et al. 1991) Minimalist Grammar (MG Stabler 1997) Combinatory Categorial Grammar (CCG Steedman 1984) Linear Context-Free Rewriting Systems (LCFRS Vijay-Shanker et al. 1987) TAG, CCG (but not recent versions of CCG), LIG and HG are weakly

  • equivalent. MCTAG and LCFRS subsume TAG, CCG, LIG and HG.

(Kallmeyer 2010) ⇒ TAG cannot generate all MCSLs!

{anbncndnen | n ≥ 1}, {www | w ∈ {a, b}∗} MIX := {w|w ∈ {a, b, c}∗, |w|a = |w|b = |w|c} (Bach 1988) SCRind := {σ (NP1, . . . , NPm)Vm . . . V1|m ≥ 1 and σ is a permutation} (Becker et al. 1992)

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 97 31

slide-98
SLIDE 98

Outline of today’s course

1

Why “working” with TAG? Formal reasons Linguistic reasons

2

From CFG to TAG Context-Free Grammars Lexicalization Tree Substitution Grammars (TSG) Adding adjunction

3

Further related formalisms

4

Summary & outlook

5

Appendix: NL and the generative capacity of grammar formalisms

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 98 32

slide-99
SLIDE 99

Summary & outlook

Summary motivation CFG → TSG → TSG+adjunction → TSG + adjunction + adjunction constraints = TAG

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 99 33

slide-100
SLIDE 100

Summary & outlook

Summary motivation CFG → TSG → TSG+adjunction → TSG + adjunction + adjunction constraints = TAG Tomorrow linguistic applications using LTAG the derivation tree subcategorization, extraction, modification adding feature structures

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 100 33

slide-101
SLIDE 101

Outline of today’s course

1

Why “working” with TAG? Formal reasons Linguistic reasons

2

From CFG to TAG Context-Free Grammars Lexicalization Tree Substitution Grammars (TSG) Adding adjunction

3

Further related formalisms

4

Summary & outlook

5

Appendix: NL and the generative capacity of grammar formalisms

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 101 34

slide-102
SLIDE 102

Appendix: NL is not regular

Hypothesis All natural languages can be accepted by a finite state automaton (FSA). FSA: finite set of states (including one start state and at least one end state) finite set of transitions between states On every transition, a word is read from the input.

qa qb qc qd qe the|a|one happy boy|girl|dog eats candies

Potential problems: recursion, constituency, long-distance dependencies ...whenever we might need a storage. How to proof the inadequacy of FSA on the level of string languages?

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 102 35

slide-103
SLIDE 103

Appendix: NL is not regular

The case of nested dependency (Chomsky (1957)): (5) a. a woman hired another woman b. a woman whom another woman hired hired another woman c. a woman whom another woman whom another woman hired hired hired another woman d. ... Formal proof by contradiction (using closure properties and the Pumping Lemma): Every regular language satisfies the Pumping Lemma, hence the patern wanz. homomorphism f : f (a woman) = w, f (whom another woman) = a, f (hired) = b, f (hired another woman) = z wa∗b∗z is a regular language; and f(English) ∩ wa∗b∗z = wanbnz should be regular as well. wanbnz contradicts the Pumping Lemma for regular languages. ⇒ English is not regular!

back Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 103 36

slide-104
SLIDE 104

Appendix: NL is not context-free

a long time debate about the context-freeness of natural languages

Chomsky (1957):34 “Of course there are languages (in our general sense) that cannot be described in terms of phrase structure, but I do not know whether or not English is itself literally outside the range of such analysis.”

several wrong arguments (see Pullum & Gazdar 1982), e.g.:

Bresnan (1978):37–38 “in many cases the number of a verb agrees with that of a noun phrase at some distance from it ... this type of syntactic dependency can extend as memory or pa- tience permits ... the distant type of agreement ... cannot be adequately described even by context-sensitive phrase-structure rules, for the possible context is not correctly describable as a finite string of phrases."

right proof techniques: pumping lemma and closure properties What is a non context-free phenomenon in natural languages?

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 104 37

slide-105
SLIDE 105

Appendix: NL is not context-free

A serious try: Syntax of Dutch (Bresnan et al. 1982) (6) dat that Jan Jan Piet Piet de the kinderen children zag saw helpen help zwemmen. swim ‘that Jan saw Piet helping the children to swim.’ Linguistic dependencies are cross-serial: n1 n2 n3 v1 v2 v3 However: No reflection on the surface, i. e. in the string language! ⇒ In principle the string can be generated by a CFG, even though the dependencies will get lost. n1 n2 n3 v1 v2 v3

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 105 38

slide-106
SLIDE 106

Appendix: NL is not context-free

Another try by Culy (1985): Duplication in the morphology of Bambara (7) a. wulu ‘dog’ b. wulu-lela ‘dog watcher’ c. wulu-lela-nyinila ‘dog watcher hunter’ d. wulu-o-wulu ‘whatever dog’ e. wulu-lela-o-wulu-lela ‘whatever dog watcher’ f. wulu-lela-nyinila-o-wulu-lela-nyinila ‘whatever dog watcher hunter’ Patern: anbmanbm or ww (copy language) ⇒ not context-free! However: proof for morphology, not for syntax (the string language)!

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 106 39

slide-107
SLIDE 107

Appendix: NL is not context-free

German: nested dependency (subordinate clauses) (8) (dass) (that) er he die the Kinder children dem the Hans Hans das the Haus house streichen paint helfen help ließ let ‘(that) he let the children help Hans paint the house’ n1 n2 n3 v3 v2 v1 Schwyzerdütsch: cross-serial dependency (9) ...mer ...we d’chind children.acc em the Hans Hans.dat es the huus house.acc lönd let hälfe help aastriiche paint ‘...we let the children help Hans paint the house’ n1 n2 n3 v1 v2 v3

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 107 40

slide-108
SLIDE 108

Appendix: NL is not context-free

Proof by Shieber (1985): (10) Jan säit das mer d’chind em Hans es huus lönd hälfe aastriiche. homomorphism f :

f (d’chind) = a f (em Hans) = b f (lönd) = c f (hälfe) = d f (Jan säit das mer) = w f (es huus) = x f (aastriiche) = y f (s) = z

f (Schwyzerdütsch) ∩ wa∗b∗xc∗d∗y = wambnxcmdny CF languages are closed under intersection with regular lan- guages wa∗b∗xc∗d∗y is regular by Pumping Lemma: wambnxcmdny is not regular ⇒ Schwyzerdütsch is not context-free

back Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 108 41

slide-109
SLIDE 109

Appendix: Mildly context sensitive languages

Appendix: NL is not mildly context-sensitive?

  • 1. A set L of languages is mildly context-sensitive iff
  • a. L contains all context-free languages
  • b. L can describe cross-serial dependencies: there is an n ≥ 2 such that

{wk | w ∈ (VT )∗} ∈ L for all k ≥ n

  • c. the languages in L are polynomially parseable, i.e., L ⊂ PTIME
  • d. the languages in L have the constant growth property
  • 2. A formalism F is mildly context-sensitive iff the set {L | L = L(G) for

some G ∈ F } is mildly context-sensitive.

constant growth property: if we order the words of a language according to their length, then the length grows in a linear way there is a finite figure n, that limits the maximum number of instantiations of cross serial dependencies in a sentence of L

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 109 42

slide-110
SLIDE 110

Appendix: NL is not mildly context-sensitive?

Evidence brought forward against semi-linearity: case stacking (“Suffixaufnahme”) in Old Georgian (Michaelis & Kracht (1997)) Chinese number-names (Radzinski 1991) coordination in Dutch (Groenink 1997) relativized predicates in Yoruba (Kobele 2006)

back Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 110 43

slide-111
SLIDE 111

Appendix: Indexed Grammar (IG)

Indexed Grammar (IG, Aho (1968)): IG = T, N, I, S, R, where I = a set of indices context-free rules extended with indices ⇒ a kind of stack indices can appear only on non-terminals, e.g. A[ijkiij] adding or removing indices on the right-hand side of the rule the index-string can be infinite long two kinds of rules in R: (i) “push and copy”: A[..] → B[i..] e.g. ...A[jk]... ⇒ ...B[ijk]... (ii) “pop and copy”: A[i..] → B[..] e.g. ...A[ijk]... ⇒ ...B[jk]... Linear Indexed Grammar (LIG, Gazdar (1988)): The stack may be copied to at most one non-terminal per rule. A[..] → B[..]C[..]

Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 111 44

slide-112
SLIDE 112

Appendix: Indexed Grammar (IG)

LIG for the language anbncn: G = T, N, I, S, R, where T = {a, b, c}, N = {S, Q}, I = {i} R = { S[..] → aS[i..]c, S[..] → Q[..], Q[i..] → Q[..]b, Q[ ] → ϵ } example derivation: S[ ] ⇒ aS[i]c ⇒ aaS[ii]cc ⇒ aaaS[iii]ccc ⇒ aaaQ[iii]ccc ⇒ aaaQ[ii]bccc ⇒ aaaQ[i]bbccc ⇒ aaaQ[ ]bbbccc ⇒ aaabbbccc S[ ]

∗

⇒ anQ[in]cn

∗

⇒ anbncn LIG for the language {ww|w ∈ {a, b}∗}: Try yourself!

back Kallmeyer, Lichte, Osswald & Petitjean (HHU Düsseldorf) 112 45

slide-113
SLIDE 113

References

Aho, Alfred V. 1968. Indexed Grammars. An extension of Context-Free Grammars. Journal of the ACM 15(4). 647–671. http://doi.acm.org/10.1145/321479.321488. Bach, Emmon. 1988. Categorial grammars as theories of language. In Richard T. Oehrle, Emmon Bach & Deirdre Wheeler (eds.), Categorial grammars and natural language structures (Studies in Linguistics and Philosophy 32), 17–34. Dordrecht, Holland: D. Reidel Publishing Company. Becker, Tilman, Owen Rambow & Michael Niv. 1992. The derivationel generative power of formal systems or scrambling is beyond LCFRS. IRCS Technical Report IRCS-92-38 Institute for Research in Cognitive Science, University of Pennsylvania Philadelphia, PA. Bod, Rens. 2009. From exemplar to grammar: A probabilistic analogy-based model of language learning. Cognitive Science 33(5). 752–793.

http://dx.doi.org/10.1111/j.1551-6709.2009.01031.x.

Bresnan, Joan. 1978. A realistic transformational grammar. In Morris Halle, Joan Bresnan & George A. Miller (eds.), Linguistic theory and psychological reality, 1–59. Cambridge, MA: The MIT Press. Bresnan, Joan, Ronald M. Kaplan, Stanley Peters & Annie Zaenen. 1982. Cross-serial dependencies in dutch. Linguistic Inquiry 13(4). 613–634. Chomsky, Noam. 1956. Three models for the description of language. IRE Transactions on Information Theory 2. 113–124. Chomsky, Noam. 1957. Syntactic structures. Den Haag: Mouton.

slide-114
SLIDE 114

References (cont.)

Chomsky, Noam & Marcel-Paul Schützenberger. 1963. The algebraic theory of context-free

  • languages. In P. Braffort & D. Hirschberg (eds.), Computer programming and formal

systems (Studies in Logic and the Foundations of Mathematics 35), 118–161. Elsevier. Culy, Christopher. 1985. The complexity of the vocabulary of Bambara. Linguistics and Philosophy 8(3). 345–351. http://www.jstor.org/stable/25001211. Dowty, David R. 1979. Word meaning and Montague Grammar. Dordrecht: D. Reidel Publishing Company. Reprinted 1991 by Kluwer Academic Publishers. Gazdar, Gerald. 1988. Applicability of indexed grammars to natural languages. In Uwe Reyle & Christian Rohrer (eds.), Natural language parsing and linguistic theories (Studies in Linguistics and Philosophy 35), 69–94. Dordrecht: D. Reidel Publishing Company. Greibach, Sheila A. 1965. A New Normal-Form Theorem for Context-Free Phrase Structure

  • Grammars. Journal of the ACM .

Groenink, Annius V. 1997. Mild context-sensitivity and tuple-based generalizations of context-grammar. Linguistics and Philosophy 20(6). 607–636. doi:10.1023/A:1005376413354. http://dx.doi.org/10.1023/A%3A1005376413354. Hausser, Roland. 1992. Complexity in Lef-Associative Grammar. Theoretical Computer Science 106(2). 283–308.

http://www.sciencedirect.com/science/article/pii/030439759290253C.

Joshi, Aravind K. 1985. Tree adjoining grammars: How much context-sensitivity is required to provide reasonable structural descriptions. In David Dowty, Lauri Kartunen & Arnold Zwicky (eds.), Natural language parsing, 206–250. Cambridge University Press.

slide-115
SLIDE 115

References (cont.)

Joshi, Aravind K. & Yves Schabes. 1991. Tree-Adjoining Grammars and lexicalized grammars.

  • Tech. Rep. MS-CIS-91-22 Department of Computer and Information Science, University of
  • Pennsylvania. http://repository.upenn.edu/cis_reports/445/.

Joshi, Aravind K. & Yves Schabes. 1997. Tree-Adjoining Grammars. In G. Rozenberg &

  • A. Salomaa (eds.), Handbook of formal languages, vol. 3, 69–124. Berlin, New York:

Springer. Kallmeyer, Laura. 2010. Parsing beyond Context-Free Grammars. Berlin: Springer. Keller, Bill & David J. Weir. 1995. A tractable extension of Linear Indexed Grammars. In Eacl, 75–82. Kobele, Gregory M. 2006. Generating copies: An investigation into structural identity in language and grammar. Los Angeles: University of California Dissertation.

http://home.uchicago.edu/~gkobele/files/Kobele06GeneratingCopies.pdf.

Michaelis, Jens & Marcus Kracht. 1997. Semilinearity as a syntactic invariant. In Christian Retoré (ed.), Logical aspects of computational linguistics (Lecture Notes in Computer Science 1328), 329–345. Berlin: Springer. doi:10.1007/BFb0052165.

http://dx.doi.org/10.1007/BFb0052165.

Pollard, Carl J. 1984. GPSGs, Head Grammar, and natural language. Stanford, CA: Stanford University Dissertation. Pullum, Geoffrey K. & Gerald Gazdar. 1982. Natural languages and context-free languages. Linguistics and Philosophy 4(4). 471–504. http://www.jstor.org/stable/25001071. Radzinski, Daniel. 1991. Chinese number-names, tree adjoining languages, and mild context-sensitivity. Computational Linguistics 17. 277–299.

slide-116
SLIDE 116

References (cont.)

Rosenberg, Arnold L. 1967. Real-time definable languages. Journal of the Association for Computing Machinery 14(4). 645–662. Schabes, Yves. 1990. Mathematical and computational aspects of lexicalized grammars: University of Pennsylvania dissertation. Schabes, Yves & Richard C. Waters. 1995. Tree Insertion Grammar: A cubic-time parsable formalism that lexicalizes Context-Free Grammar without changing the trees produced. Computational Linguistics 21(4). 479–513. Seki, Hiroyuki, Takahashi Matsumura, Mamoru Fujii & Tadao Kasami. 1991. On multiple context-free grammars. Theoretical Computer Science 88(2). 191–229. Shieber, Stuart. 1985. Evidence against the context-freeness of natural language. Linguistics and Philosophy 8. 333–343. Stabler, Edward. 1997. Derivational minimalism. In Christian Retoré (ed.), Logical aspects

  • f computational linguistics (Lecture Notes in Computer Science 1328), 68–95. New

York: Springer. Steedman, Mark. 1984. A categorial theory of intersecting dependencies in Dutch infinitival

  • complements. In Wim de Geest & Yvan Putseys (eds.), Sentential complementation,

215–226. Foris, Dordrecht. Swanson, Ben, Elif Yamangil, Eugene Charniak & Stuart Shieber. 2013. A context free TAG

  • variant. In Proceedings of the 51st annual meeting of the Association for

Computational Linguistics, 302–310. Sofia, Bulgaria.

http://www.aclweb.org/anthology/P13-1030.

slide-117
SLIDE 117

References (cont.)

Vijay-Shanker, K., David J. Weir & Aravind K. Joshi. 1987. Characterizing structural descriptions produced by various grammatical formalisms. In Proceedings of acl, . Wurm, Christian. 2012. Regular Growth Automata: Properties of a class of finitely induced infinite machines. In Makoto Kanazawa, Markus Kracht, Hiroyuki Seki & Andras Kornai (eds.), Proceedings of the 12th conference on the mathematics of language (MOL 2012) (LNCS 6878), 192–208. Berlin: Springer.