SLIDE 1
Lecture 15: Dependency Parsing
Kai-Wei Chang CS @ University of Virginia kw@kwchang.net Couse webpage: http://kwchang.net/teaching/NLP16
1 CS6501: NLP
How to represent the structure
CS6501: NLP 2
Lecture 15: Dependency Parsing Kai-Wei Chang CS @ University of - - PowerPoint PPT Presentation
Lecture 15: Dependency Parsing Kai-Wei Chang CS @ University of - - PowerPoint PPT Presentation
Lecture 15: Dependency Parsing Kai-Wei Chang CS @ University of Virginia kw@kwchang.net Couse webpage: http://kwchang.net/teaching/NLP16 CS6501: NLP 1 How to represent the structure CS6501: NLP 2 Dependency trees v Dependency grammar
SLIDE 2
SLIDE 3
Dependency trees
v Dependency grammar describe the structure of sentences as a graph (tree)
vNodes represent words vEdges represent dependencies
v Idea goes back to 4th century BC in ancient India
CS6501: NLP 3
SLIDE 4
Phrase structure (constituent parse) trees v Can be modeled by Context-free grammars v We will see how constituent parse and dependency parse are related
CS6501: NLP 4
SLIDE 5
Context-free grammars
CS6501: NLP 5
PP → P NP PP → P DT N PP → in the garden Non-terminal: DT, N, P, NP, PP, … Terminal: the, a, ball, garden
SLIDE 6
Generate sentences by CFG
CS6501: NLP 6
SLIDE 7
Parse tree defined by CFG
CS6501: NLP 7
1 2 3 4 5 6 7
Rule 2 Rule 4 & 1 Rule 6
SLIDE 8
Example: noun phrases
CS6501: NLP 8
SLIDE 9
Example: verb phrase
CS6501: NLP 9
SLIDE 10
Sentences
CS6501: NLP 10
SLIDE 11
Constituent Parse
CS6501: NLP 11
From: Kevin Gimpel
SLIDE 12
Constituent Parse
CS6501: NLP 12
Non-terminal Terminal S à NP VP NP à DT NN NP à DT PP à IN NP VP à VBD PP VP à NP VBD NP VP à NP VB
SLIDE 13
Nonterminal in Penn Treebank
CS6501: NLP 13
SLIDE 14
Probabilistic Context-free Grammar
CS6501: NLP 14
Non-terminal Terminal 1.0 S à NP VP 0.6 NP à DT NN 0.4 NP à NP PP 1.0 PP à IN NP 0.5 VP à VBD PP 0.2 VP à NP VBD NP 0.3 VP à NP VB
SLIDE 15
Probabilistic Context-free Grammar
v PCFG achieves ~73% on PTB v State-of-the art ~92% v Lexicalized PCFG (Collins 1997)
CS6501: NLP 15
SLIDE 16
CS6501: NLP 16
SLIDE 17
How to decide head?
v Usually use deterministic head rules (e.g., Collins head rules) v Define heads in CFG
vS → NP VP vVP → VBD NP PP vNP → DT JJ NN
CS6501: NLP 17
From Noah Smith
SLIDE 18
Lexical Head Annotation
CS6501: NLP 18
SLIDE 19
Constituent parse → Dependency Parse
CS6501: NLP 19
SLIDE 20
Constituent parse → Dependency Parse
CS6501: NLP 20
SLIDE 21
CS6501: NLP 21
Head rules can be used to extract dependency parse from a CFG
SLIDE 22
CS6501: NLP 22
Arrow types show the name of grammatical relations
SLIDE 23
Dependency parsing
v Can be more flexible (non-projective) v English are mostly projective
vSome free word order languages (e.g., Czech) are non-projective
CS6501: NLP 23
SLIDE 24
How to build a dependency tree?
v There are several approaches v Graph Algorithms
v Consider all word pairs v Create a Maximum Spanning Tree for a sentence
v Transition-base Approaches
v Similar to how we parse a program: Shift-Reduce Parser
v Many other approaches…
CS6501: NLP 24
SLIDE 25
Sources of information for DP
v Lexical affinities
v[ issues → the ] v[ issues → I ]
v Distances
vWords usually depend on nearby words
v Valency of heads
v# dependents for a head
CS6501: NLP 25
SLIDE 26
Graph-Based Approaches[McDonald et al. 2005] v Consider all word pairs and assign scores v Score of a tree = sum of score of edges v Can be solve as a MST problem
vChu-Liu-Edmonds
CS6501: NLP 26
SLIDE 27
Transition-based parser
v MaltParser (Nivre et al. 2008) v Similar to a Shift-Reduce Parser
vBut “reduce” actions can create dependencies
v The parser has:
vA stack 𝜏– starts with a “Root” symbol vA buffer 𝛾– starts with the input sentence vA set of dependency arcs A– starts off empty
v Use a set of actions to parse sentences
vMany possible action sets
CS6501: NLP 27
SLIDE 28
Arc-Eager Dependency Parser
v Shift: v Left-Arc:
CS6501: NLP 28
ROOT Joe likes Mary Joe ROOT likes Mary
→
Joe ROOT likes Mary
→
ROOT likes Mary
Joe
Precondition: 𝑥% ≠ Root & (𝑥(,𝑥%) ∉ 𝐵
SLIDE 29
Arc-Eager Dependency Parser
v Right-Arc v Reduce:
CS6501: NLP 29
like ROOT Mary Mary like ROOT
→
Joe Joe
Mary like ROOT
Joe
→
like ROOT
Joe
Precondition: (𝑥(, 𝑥%) ∈ 𝐵
Mary
SLIDE 30
Arc-Eager Dependency Parser
v Start: v Conduct a sequence of actions v Terminate with 𝜏, 𝛾 = ∅
CS6501: NLP 30
SLIDE 31
CS6501: NLP 31
SLIDE 32
CS6501: NLP 32
SLIDE 33
CS6501: NLP 33
SLIDE 34
CS6501: NLP 34
SLIDE 35
CS6501: NLP 35
SLIDE 36
CS6501: NLP 36
SLIDE 37
CS6501: NLP 37
SLIDE 38
CS6501: NLP 38
SLIDE 39
CS6501: NLP 39
SLIDE 40
CS6501: NLP 40
SLIDE 41
CS6501: NLP 41
SLIDE 42
CS6501: NLP 42
SLIDE 43
CS6501: NLP 43
SLIDE 44
CS6501: NLP 44
SLIDE 45
CS6501: NLP 45
SLIDE 46
CS6501: NLP 46
SLIDE 47
CS6501: NLP 47
SLIDE 48
It’s your turn
v Happy children like to play with their friend . v Shift → Left-arc → Shift → Left-arc → Right-arc → Shift → Left-arc → Right-arc → Right-arc → Shift → Left-arc → Right-arc → Reduce*3 → Right-arc → Reduce*3
CS6501: NLP 48
SLIDE 49
CS6501: NLP 49
From Chris Manning
SLIDE 50
CS6501: NLP 50
SLIDE 51
Structured Prediction –beyond sequence tagging
Task Input Output
Part-of-speech Tagging They operate ships and banks. Dependency Parsing They operate ships and banks.
51
Pronoun Verb Noun And Noun
Root They operate ships and banks .