Lecture 15: Dependency Parsing Kai-Wei Chang CS @ University of - - PowerPoint PPT Presentation

lecture 15 dependency parsing
SMART_READER_LITE
LIVE PREVIEW

Lecture 15: Dependency Parsing Kai-Wei Chang CS @ University of - - PowerPoint PPT Presentation

Lecture 15: Dependency Parsing Kai-Wei Chang CS @ University of Virginia kw@kwchang.net Couse webpage: http://kwchang.net/teaching/NLP16 CS6501: NLP 1 How to represent the structure CS6501: NLP 2 Dependency trees v Dependency grammar


slide-1
SLIDE 1

Lecture 15: Dependency Parsing

Kai-Wei Chang CS @ University of Virginia kw@kwchang.net Couse webpage: http://kwchang.net/teaching/NLP16

1 CS6501: NLP

slide-2
SLIDE 2

How to represent the structure

CS6501: NLP 2

slide-3
SLIDE 3

Dependency trees

v Dependency grammar describe the structure of sentences as a graph (tree)

vNodes represent words vEdges represent dependencies

v Idea goes back to 4th century BC in ancient India

CS6501: NLP 3

slide-4
SLIDE 4

Phrase structure (constituent parse) trees v Can be modeled by Context-free grammars v We will see how constituent parse and dependency parse are related

CS6501: NLP 4

slide-5
SLIDE 5

Context-free grammars

CS6501: NLP 5

PP → P NP PP → P DT N PP → in the garden Non-terminal: DT, N, P, NP, PP, … Terminal: the, a, ball, garden

slide-6
SLIDE 6

Generate sentences by CFG

CS6501: NLP 6

slide-7
SLIDE 7

Parse tree defined by CFG

CS6501: NLP 7

1 2 3 4 5 6 7

Rule 2 Rule 4 & 1 Rule 6

slide-8
SLIDE 8

Example: noun phrases

CS6501: NLP 8

slide-9
SLIDE 9

Example: verb phrase

CS6501: NLP 9

slide-10
SLIDE 10

Sentences

CS6501: NLP 10

slide-11
SLIDE 11

Constituent Parse

CS6501: NLP 11

From: Kevin Gimpel

slide-12
SLIDE 12

Constituent Parse

CS6501: NLP 12

Non-terminal Terminal S à NP VP NP à DT NN NP à DT PP à IN NP VP à VBD PP VP à NP VBD NP VP à NP VB

slide-13
SLIDE 13

Nonterminal in Penn Treebank

CS6501: NLP 13

slide-14
SLIDE 14

Probabilistic Context-free Grammar

CS6501: NLP 14

Non-terminal Terminal 1.0 S à NP VP 0.6 NP à DT NN 0.4 NP à NP PP 1.0 PP à IN NP 0.5 VP à VBD PP 0.2 VP à NP VBD NP 0.3 VP à NP VB

slide-15
SLIDE 15

Probabilistic Context-free Grammar

v PCFG achieves ~73% on PTB v State-of-the art ~92% v Lexicalized PCFG (Collins 1997)

CS6501: NLP 15

slide-16
SLIDE 16

CS6501: NLP 16

slide-17
SLIDE 17

How to decide head?

v Usually use deterministic head rules (e.g., Collins head rules) v Define heads in CFG

vS → NP VP vVP → VBD NP PP vNP → DT JJ NN

CS6501: NLP 17

From Noah Smith

slide-18
SLIDE 18

Lexical Head Annotation

CS6501: NLP 18

slide-19
SLIDE 19

Constituent parse → Dependency Parse

CS6501: NLP 19

slide-20
SLIDE 20

Constituent parse → Dependency Parse

CS6501: NLP 20

slide-21
SLIDE 21

CS6501: NLP 21

Head rules can be used to extract dependency parse from a CFG

slide-22
SLIDE 22

CS6501: NLP 22

Arrow types show the name of grammatical relations

slide-23
SLIDE 23

Dependency parsing

v Can be more flexible (non-projective) v English are mostly projective

vSome free word order languages (e.g., Czech) are non-projective

CS6501: NLP 23

slide-24
SLIDE 24

How to build a dependency tree?

v There are several approaches v Graph Algorithms

v Consider all word pairs v Create a Maximum Spanning Tree for a sentence

v Transition-base Approaches

v Similar to how we parse a program: Shift-Reduce Parser

v Many other approaches…

CS6501: NLP 24

slide-25
SLIDE 25

Sources of information for DP

v Lexical affinities

v[ issues → the ] v[ issues → I ]

v Distances

vWords usually depend on nearby words

v Valency of heads

v# dependents for a head

CS6501: NLP 25

slide-26
SLIDE 26

Graph-Based Approaches[McDonald et al. 2005] v Consider all word pairs and assign scores v Score of a tree = sum of score of edges v Can be solve as a MST problem

vChu-Liu-Edmonds

CS6501: NLP 26

slide-27
SLIDE 27

Transition-based parser

v MaltParser (Nivre et al. 2008) v Similar to a Shift-Reduce Parser

vBut “reduce” actions can create dependencies

v The parser has:

vA stack 𝜏– starts with a “Root” symbol vA buffer 𝛾– starts with the input sentence vA set of dependency arcs A– starts off empty

v Use a set of actions to parse sentences

vMany possible action sets

CS6501: NLP 27

slide-28
SLIDE 28

Arc-Eager Dependency Parser

v Shift: v Left-Arc:

CS6501: NLP 28

ROOT Joe likes Mary Joe ROOT likes Mary

Joe ROOT likes Mary

ROOT likes Mary

Joe

Precondition: 𝑥% ≠ Root & (𝑥(,𝑥%) ∉ 𝐵

slide-29
SLIDE 29

Arc-Eager Dependency Parser

v Right-Arc v Reduce:

CS6501: NLP 29

like ROOT Mary Mary like ROOT

Joe Joe

Mary like ROOT

Joe

like ROOT

Joe

Precondition: (𝑥(, 𝑥%) ∈ 𝐵

Mary

slide-30
SLIDE 30

Arc-Eager Dependency Parser

v Start: v Conduct a sequence of actions v Terminate with 𝜏, 𝛾 = ∅

CS6501: NLP 30

slide-31
SLIDE 31

CS6501: NLP 31

slide-32
SLIDE 32

CS6501: NLP 32

slide-33
SLIDE 33

CS6501: NLP 33

slide-34
SLIDE 34

CS6501: NLP 34

slide-35
SLIDE 35

CS6501: NLP 35

slide-36
SLIDE 36

CS6501: NLP 36

slide-37
SLIDE 37

CS6501: NLP 37

slide-38
SLIDE 38

CS6501: NLP 38

slide-39
SLIDE 39

CS6501: NLP 39

slide-40
SLIDE 40

CS6501: NLP 40

slide-41
SLIDE 41

CS6501: NLP 41

slide-42
SLIDE 42

CS6501: NLP 42

slide-43
SLIDE 43

CS6501: NLP 43

slide-44
SLIDE 44

CS6501: NLP 44

slide-45
SLIDE 45

CS6501: NLP 45

slide-46
SLIDE 46

CS6501: NLP 46

slide-47
SLIDE 47

CS6501: NLP 47

slide-48
SLIDE 48

It’s your turn

v Happy children like to play with their friend . v Shift → Left-arc → Shift → Left-arc → Right-arc → Shift → Left-arc → Right-arc → Right-arc → Shift → Left-arc → Right-arc → Reduce*3 → Right-arc → Reduce*3

CS6501: NLP 48

slide-49
SLIDE 49

CS6501: NLP 49

From Chris Manning

slide-50
SLIDE 50

CS6501: NLP 50

slide-51
SLIDE 51

Structured Prediction –beyond sequence tagging

Task Input Output

Part-of-speech Tagging They operate ships and banks. Dependency Parsing They operate ships and banks.

51

Pronoun Verb Noun And Noun

Root They operate ships and banks .

Assign values to a set of interdependent output variables