SLIDE 1
Emma Strubell
Algorithms for NLP
CS 11-711 · Fall 2020
Announcements
2
Algorithms for NLP CS 11-711 Fall 2020 Lecture 11: Syntactic - - PowerPoint PPT Presentation
Algorithms for NLP CS 11-711 Fall 2020 Lecture 11: Syntactic - - PowerPoint PPT Presentation
Algorithms for NLP CS 11-711 Fall 2020 Lecture 11: Syntactic parsing and context-free grammars Emma Strubell Announcements Fridays recitation will be a P2 Q&A + questions to work through. 2 Ambiguity I saw the woman with the
SLIDE 2
SLIDE 3
3
I saw the woman with the telescope wrapped in paper.
Ambiguity
SLIDE 4
3
■
Who has the telescope?
I saw the woman with the telescope wrapped in paper.
Ambiguity
SLIDE 5
3
■
Who has the telescope?
■
Who or what is wrapped in paper?
I saw the woman with the telescope wrapped in paper.
Ambiguity
SLIDE 6
3
■
Who has the telescope?
■
Who or what is wrapped in paper?
■
Event of perception or assault?
I saw the woman with the telescope wrapped in paper.
Ambiguity
SLIDE 7
Syntactic parsing
4
■ Input: ■ Output:
The move followed a round of similar increases by other lenders, reflecting a continuing decline in that market.
SLIDE 8
Parsing as supervised ML
5
Canadian Utilities had 1988 revenue of $ 1.16 billion , mainly from its natural gas and electric utility businesses in Alberta , where the company serves about 800,000 customers .
SLIDE 9
Parsing as supervised ML
5
■ Data for parsing experiments:
Canadian Utilities had 1988 revenue of $ 1.16 billion , mainly from its natural gas and electric utility businesses in Alberta , where the company serves about 800,000 customers .
SLIDE 10
Parsing as supervised ML
5
■ Data for parsing experiments: ■ WSJ portion of the Penn Treebank = 50k sentences annotated with trees ■ Usual train/test split: 40k training, 1700 development, 2400 test
Canadian Utilities had 1988 revenue of $ 1.16 billion , mainly from its natural gas and electric utility businesses in Alberta , where the company serves about 800,000 customers .
SLIDE 11
Syntax
6
■ The study of the patterns of formation of sentences and phrases from words ■ my dog Pron N ■ the dog Det N ■ the cat Det N ■ and Conj ■ the large cat Det Adj N ■ the black cat Det Adj N ■ ate a sausage V Det N
SLIDE 12
Parsing
7
SLIDE 13
Parsing
7
■ The process of predicting syntactic representations
SLIDE 14
Parsing
7
■ The process of predicting syntactic representations ■ Different types of syntactic representations are possible, for example:
SLIDE 15
Parsing
7
■ The process of predicting syntactic representations ■ Different types of syntactic representations are possible, for example:
constituency (aka phrase-structure) tree
SLIDE 16
Constituency trees
8
SLIDE 17
Constituency trees
8
■ Internal nodes correspond to phrases.
SLIDE 18
Constituency trees
8
■ Internal nodes correspond to phrases.
■ S: a sentence
SLIDE 19
Constituency trees
8
■ Internal nodes correspond to phrases.
■ S: a sentence ■ NP (noun phrase): My dog, a sandwich, lakes, …
SLIDE 20
Constituency trees
8
■ Internal nodes correspond to phrases.
■ S: a sentence ■ NP (noun phrase): My dog, a sandwich, lakes, … ■ VP (verb phrase): ate a sausage, barked, …
SLIDE 21
Constituency trees
8
■ Internal nodes correspond to phrases.
■ S: a sentence ■ NP (noun phrase): My dog, a sandwich, lakes, … ■ VP (verb phrase): ate a sausage, barked, … ■ PP (prepositional phrases): with a friend, in a car, …
SLIDE 22
Constituency trees
8
■ Internal nodes correspond to phrases.
■ S: a sentence ■ NP (noun phrase): My dog, a sandwich, lakes, … ■ VP (verb phrase): ate a sausage, barked, … ■ PP (prepositional phrases): with a friend, in a car, …
■ Nodes immediately above words are part-of-speech tags (or preterminals).
SLIDE 23
Constituency trees
8
■ Internal nodes correspond to phrases.
■ S: a sentence ■ NP (noun phrase): My dog, a sandwich, lakes, … ■ VP (verb phrase): ate a sausage, barked, … ■ PP (prepositional phrases): with a friend, in a car, …
■ Nodes immediately above words are part-of-speech tags (or preterminals).
■ PN: pronoun
SLIDE 24
Constituency trees
8
■ Internal nodes correspond to phrases.
■ S: a sentence ■ NP (noun phrase): My dog, a sandwich, lakes, … ■ VP (verb phrase): ate a sausage, barked, … ■ PP (prepositional phrases): with a friend, in a car, …
■ Nodes immediately above words are part-of-speech tags (or preterminals).
■ PN: pronoun ■ D: determiner
SLIDE 25
Constituency trees
8
■ Internal nodes correspond to phrases.
■ S: a sentence ■ NP (noun phrase): My dog, a sandwich, lakes, … ■ VP (verb phrase): ate a sausage, barked, … ■ PP (prepositional phrases): with a friend, in a car, …
■ Nodes immediately above words are part-of-speech tags (or preterminals).
■ PN: pronoun ■ D: determiner ■ V: verb
SLIDE 26
Constituency trees
8
■ Internal nodes correspond to phrases.
■ S: a sentence ■ NP (noun phrase): My dog, a sandwich, lakes, … ■ VP (verb phrase): ate a sausage, barked, … ■ PP (prepositional phrases): with a friend, in a car, …
■ Nodes immediately above words are part-of-speech tags (or preterminals).
■ PN: pronoun ■ D: determiner ■ V: verb ■ N: noun
SLIDE 27
Constituency trees
8
■ Internal nodes correspond to phrases.
■ S: a sentence ■ NP (noun phrase): My dog, a sandwich, lakes, … ■ VP (verb phrase): ate a sausage, barked, … ■ PP (prepositional phrases): with a friend, in a car, …
■ Nodes immediately above words are part-of-speech tags (or preterminals).
■ PN: pronoun ■ D: determiner ■ V: verb ■ N: noun ■ P: preposition
SLIDE 28
Bracketing notation
9
■ Often convenient to represent a tree as a bracketed sequence:
(S (NP (PN My) (N dog) ) (VP (V ate) (NP (D a) (N sausage) ) ) )
SLIDE 29
Parsing
10
■ The process of predicting syntactic representations ■ Different types of syntactic representations are possible, for example:
constituency (aka phrase-structure) tree
SLIDE 30
Parsing
10
■ The process of predicting syntactic representations ■ Different types of syntactic representations are possible, for example:
constituency (aka phrase-structure) tree dependency tree My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 31
Dependency trees
11
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 32
Dependency trees
11
■ Nodes are words (along with part-of-speech tags)
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 33
Dependency trees
11
■ Nodes are words (along with part-of-speech tags) ■ Directed arcs encode syntactic dependencies between words
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 34
Dependency trees
11
■ Nodes are words (along with part-of-speech tags) ■ Directed arcs encode syntactic dependencies between words ■ Labels are types of relations between words
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 35
Dependency trees
11
■ Nodes are words (along with part-of-speech tags) ■ Directed arcs encode syntactic dependencies between words ■ Labels are types of relations between words ■ poss: possessive
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 36
Dependency trees
11
■ Nodes are words (along with part-of-speech tags) ■ Directed arcs encode syntactic dependencies between words ■ Labels are types of relations between words ■ poss: possessive ■ dobj: direct object
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 37
Dependency trees
11
■ Nodes are words (along with part-of-speech tags) ■ Directed arcs encode syntactic dependencies between words ■ Labels are types of relations between words ■ poss: possessive ■ dobj: direct object ■ nsubj: (noun) subject
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 38
Dependency trees
11
■ Nodes are words (along with part-of-speech tags) ■ Directed arcs encode syntactic dependencies between words ■ Labels are types of relations between words ■ poss: possessive ■ dobj: direct object ■ nsubj: (noun) subject ■ det: determiner
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 39
Dependency parsing
12
Recovering shallow semantics
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 40
Dependency parsing
12
■ Some semantic information can be (approximately) derived from syntactic information
Recovering shallow semantics
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 41
Dependency parsing
12
■ Some semantic information can be (approximately) derived from syntactic information ■ Subjects (nsubj) are (often) agents: initiators / doers of an action
Recovering shallow semantics
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 42
Dependency parsing
12
■ Some semantic information can be (approximately) derived from syntactic information ■ Subjects (nsubj) are (often) agents: initiators / doers of an action ■ Direct objects (dobj) are (often) patients: affected entities
Recovering shallow semantics
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 43
Dependency parsing
12
■ Some semantic information can be (approximately) derived from syntactic information ■ Subjects (nsubj) are (often) agents: initiators / doers of an action ■ Direct objects (dobj) are (often) patients: affected entities ■ Even for agents and patients, consider:
Recovering shallow semantics
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 44
Dependency parsing
12
■ Some semantic information can be (approximately) derived from syntactic information ■ Subjects (nsubj) are (often) agents: initiators / doers of an action ■ Direct objects (dobj) are (often) patients: affected entities ■ Even for agents and patients, consider: ■ Mary is baking a cake in the oven
Recovering shallow semantics
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 45
Dependency parsing
12
■ Some semantic information can be (approximately) derived from syntactic information ■ Subjects (nsubj) are (often) agents: initiators / doers of an action ■ Direct objects (dobj) are (often) patients: affected entities ■ Even for agents and patients, consider: ■ Mary is baking a cake in the oven ■ A cake is baking in the oven
Recovering shallow semantics
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 46
Dependency parsing
12
■ Some semantic information can be (approximately) derived from syntactic information ■ Subjects (nsubj) are (often) agents: initiators / doers of an action ■ Direct objects (dobj) are (often) patients: affected entities ■ Even for agents and patients, consider: ■ Mary is baking a cake in the oven ■ A cake is baking in the oven ■ In general, it is not trivial even for the most shallow forms of semantics
Recovering shallow semantics
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 47
Dependency parsing
12
■ Some semantic information can be (approximately) derived from syntactic information ■ Subjects (nsubj) are (often) agents: initiators / doers of an action ■ Direct objects (dobj) are (often) patients: affected entities ■ Even for agents and patients, consider: ■ Mary is baking a cake in the oven ■ A cake is baking in the oven ■ In general, it is not trivial even for the most shallow forms of semantics ■ e.g. prepositions: in can encode direction, position, temporal information, …
Recovering shallow semantics
My dog ate a sausage
root
nsubj poss root dobj det
PN N V D N
SLIDE 48
Constituency and dependency representations
13
SLIDE 49
Constituency and dependency representations
13
■ Constituency trees can (potentially) be converted to dependency trees.
SLIDE 50
Constituency and dependency representations
13
■ Constituency trees can (potentially) be converted to dependency trees. ■ Dependency trees can (potentially) be converted to constituency trees.
SLIDE 51
Constituency trees
14
■ Internal nodes correspond to phrases.
■ S: a sentence ■ NP (noun phrase): My dog, a sandwich, lakes, … ■ VP (verb phrase): ate a sausage, barked, … ■ PP (prepositional phrases): with a friend, in a car, …
■ Nodes immediately above words are part-of-speech tags (or preterminals).
■ PN: pronoun ■ D: determiner ■ V: verb ■ N: noun ■ P: preposition
SLIDE 52
Constituency tests
15
SLIDE 53
Constituency tests
15
■ How do we know what nodes go in the tree?
SLIDE 54
Constituency tests
15
■ How do we know what nodes go in the tree? ■ Classic constituency tests:
SLIDE 55
Constituency tests
15
■ How do we know what nodes go in the tree? ■ Classic constituency tests:
■ Replacement
SLIDE 56
Constituency tests
15
■ How do we know what nodes go in the tree? ■ Classic constituency tests:
■ Replacement ■ Substitution by proform
SLIDE 57
Constituency tests
15
■ How do we know what nodes go in the tree? ■ Classic constituency tests:
■ Replacement ■ Substitution by proform ■ Movement: Clefting, preposing, passive
SLIDE 58
Constituency tests
15
■ How do we know what nodes go in the tree? ■ Classic constituency tests:
■ Replacement ■ Substitution by proform ■ Movement: Clefting, preposing, passive ■ Modification
SLIDE 59
Constituency tests
15
■ How do we know what nodes go in the tree? ■ Classic constituency tests:
■ Replacement ■ Substitution by proform ■ Movement: Clefting, preposing, passive ■ Modification ■ Coordination / conjunction
SLIDE 60
Constituency tests
15
■ How do we know what nodes go in the tree? ■ Classic constituency tests:
■ Replacement ■ Substitution by proform ■ Movement: Clefting, preposing, passive ■ Modification ■ Coordination / conjunction ■ Ellipsis / deletion
SLIDE 61
Conflicting tests
16
SLIDE 62
Conflicting tests
16
■ Constituency is not always clear.
SLIDE 63
Conflicting tests
16
■ Constituency is not always clear. ■ Coordination:
He went to and came from the store.
SLIDE 64
Conflicting tests
16
■ Constituency is not always clear. ■ Coordination: ■ Phonological reduction:
He went to and came from the store. I will go → I’ll go I want to go → I wanna go a le centre → au centre La velocité des ondes sismiques
SLIDE 65
Morphology + syntax + semantics
17
SLIDE 66
Morphology + syntax + semantics
17
■ Syntax: The study of the patterns of formation of sentences and phrases
from a word.
SLIDE 67
Morphology + syntax + semantics
17
■ Syntax: The study of the patterns of formation of sentences and phrases
from a word.
■ Borders with semantics and morphology are sometimes blurred.
SLIDE 68
Morphology + syntax + semantics
17
■ Syntax: The study of the patterns of formation of sentences and phrases
from a word.
■ Borders with semantics and morphology are sometimes blurred.
Afyonkarahisarlılaştırabildiklerimizdenmişsinizcesinee
SLIDE 69
Morphology + syntax + semantics
17
■ Syntax: The study of the patterns of formation of sentences and phrases
from a word.
■ Borders with semantics and morphology are sometimes blurred.
Afyonkarahisarlılaştırabildiklerimizdenmişsinizcesinee as if you are one of the people that we thought to be originating from Afyonkarahisar
SLIDE 70
Context-free grammars (CFGs)
18
SLIDE 71
Context-free grammars (CFGs)
18
■ Context-free grammars (CFGs): a formalism for parsing.
SLIDE 72
Context-free grammars (CFGs)
18
■ Context-free grammars (CFGs): a formalism for parsing.
Grammar (CFG)
ROOT → S NP → NP PP S → NP VP VP → VBP NP NP → DT NN VP → VBP NP PP NP → NN NNS PP → IN NP
SLIDE 73
Context-free grammars (CFGs)
18
■ Context-free grammars (CFGs): a formalism for parsing.
Grammar (CFG) Lexicon
NN → interest NNS → raises VBP → interest VBP → raises … ROOT → S NP → NP PP S → NP VP VP → VBP NP NP → DT NN VP → VBP NP PP NP → NN NNS PP → IN NP
SLIDE 74
Context-free grammars (CFGs)
18
■ Context-free grammars (CFGs): a formalism for parsing. ■ Other grammar formalisms: LFG, HPSG, TAG, CCG, …
Grammar (CFG) Lexicon
NN → interest NNS → raises VBP → interest VBP → raises … ROOT → S NP → NP PP S → NP VP VP → VBP NP NP → DT NN VP → VBP NP PP NP → NN NNS PP → IN NP
SLIDE 75
Context-free grammars (CFGs)
19
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 76
Context-free grammars (CFGs)
20
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 77
Context-free grammars (CFGs)
20
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 78
Context-free grammars (CFGs)
21
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 79
Context-free grammars (CFGs)
21
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 80
Context-free grammars (CFGs)
22
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 81
Context-free grammars (CFGs)
22
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 82
Context-free grammars (CFGs)
23
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 83
Context-free grammars (CFGs)
23
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 84
Context-free grammars (CFGs)
24
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 85
Context-free grammars (CFGs)
24
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 86
Context-free grammars (CFGs)
25
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 87
Context-free grammars (CFGs)
25
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 88
Context-free grammars (CFGs)
26
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 89
Context-free grammars (CFGs)
26
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 90
Context-free grammars (CFGs)
27
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 91
Context-free grammars (CFGs)
28
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 92
Context-free grammars (CFGs)
28
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 93
Context-free grammars (CFGs)
29
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 94
Context-free grammars (CFGs)
29
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 95
Context-free grammars (CFGs)
30
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 96
Context-free grammars (CFGs)
30
Grammar (CFG) Lexicon
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP VP → V VP → V NP VP → VP PP NP → NP PP NP → D N NP → PN PP → P NP
SLIDE 97
Context-free grammars (CFGs)
31
■ CFG: Formal definition. A 4-tuple (N, Σ, R, S):
N a set of non-terminal symbols (or variables) Σ a set of terminal symbols (disjoint from N) R a set of rules or productions, each of the form A → β , where A is a non-terminal, β is a string of symbols from the infinite set of strings (Σ∪N)∗ S a designated start symbol and a member of N
SLIDE 98
Context-free grammars (CFGs)
31
■ CFG: Formal definition. A 4-tuple (N, Σ, R, S):
N a set of non-terminal symbols (or variables) Σ a set of terminal symbols (disjoint from N) R a set of rules or productions, each of the form A → β , where A is a non-terminal, β is a string of symbols from the infinite set of strings (Σ∪N)∗ S a designated start symbol and a member of N
VP , NP , S, PP , … V, N, P…
SLIDE 99
Context-free grammars (CFGs)
31
■ CFG: Formal definition. A 4-tuple (N, Σ, R, S):
N a set of non-terminal symbols (or variables) Σ a set of terminal symbols (disjoint from N) R a set of rules or productions, each of the form A → β , where A is a non-terminal, β is a string of symbols from the infinite set of strings (Σ∪N)∗ S a designated start symbol and a member of N
VP , NP , S, PP , … V, N, P… saw, telescope, the, girl, …
SLIDE 100
Context-free grammars (CFGs)
31
■ CFG: Formal definition. A 4-tuple (N, Σ, R, S):
N a set of non-terminal symbols (or variables) Σ a set of terminal symbols (disjoint from N) R a set of rules or productions, each of the form A → β , where A is a non-terminal, β is a string of symbols from the infinite set of strings (Σ∪N)∗ S a designated start symbol and a member of N
VP , NP , S, PP , … V, N, P… saw, telescope, the, girl, … NP → NP PP , …
SLIDE 101
Context-free grammars (CFGs)
31
■ CFG: Formal definition. A 4-tuple (N, Σ, R, S):
N a set of non-terminal symbols (or variables) Σ a set of terminal symbols (disjoint from N) R a set of rules or productions, each of the form A → β , where A is a non-terminal, β is a string of symbols from the infinite set of strings (Σ∪N)∗ S a designated start symbol and a member of N
VP , NP , S, PP , … V, N, P… saw, telescope, the, girl, … NP → NP PP , … ROOT, TOP
SLIDE 102
An example grammar
32
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP (NP a girl) (VP ate a sandwich) VP → V VP → V NP (V ate) (NP a sandwich) VP → VP PP (VP saw a girl) (PP with a telescope) NP → NP PP (NP a girl) (PP with a sandwich) NP → D N (D a) (N sandwich) NP → PN PP → P NP (P with) (NP a sandwich)
SLIDE 103
An example grammar
32
■ N = {S, VP
, NP , PP , N, V, PN, P}
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP (NP a girl) (VP ate a sandwich) VP → V VP → V NP (V ate) (NP a sandwich) VP → VP PP (VP saw a girl) (PP with a telescope) NP → NP PP (NP a girl) (PP with a sandwich) NP → D N (D a) (N sandwich) NP → PN PP → P NP (P with) (NP a sandwich)
SLIDE 104
An example grammar
32
■ N = {S, VP
, NP , PP , N, V, PN, P}
■ Σ = {girl, telescope, sandwich, I, saw, ate, with, in, a, the}
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP (NP a girl) (VP ate a sandwich) VP → V VP → V NP (V ate) (NP a sandwich) VP → VP PP (VP saw a girl) (PP with a telescope) NP → NP PP (NP a girl) (PP with a sandwich) NP → D N (D a) (N sandwich) NP → PN PP → P NP (P with) (NP a sandwich)
SLIDE 105
An example grammar
32
■ N = {S, VP
, NP , PP , N, V, PN, P}
■ Σ = {girl, telescope, sandwich, I, saw, ate, with, in, a, the} ■ S = {S}
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP (NP a girl) (VP ate a sandwich) VP → V VP → V NP (V ate) (NP a sandwich) VP → VP PP (VP saw a girl) (PP with a telescope) NP → NP PP (NP a girl) (PP with a sandwich) NP → D N (D a) (N sandwich) NP → PN PP → P NP (P with) (NP a sandwich)
SLIDE 106
An example grammar
32
■ N = {S, VP
, NP , PP , N, V, PN, P}
■ Σ = {girl, telescope, sandwich, I, saw, ate, with, in, a, the} ■ S = {S} ■ R =
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP (NP a girl) (VP ate a sandwich) VP → V VP → V NP (V ate) (NP a sandwich) VP → VP PP (VP saw a girl) (PP with a telescope) NP → NP PP (NP a girl) (PP with a sandwich) NP → D N (D a) (N sandwich) NP → PN PP → P NP (P with) (NP a sandwich)
SLIDE 107
An example grammar
32
■ N = {S, VP
, NP , PP , N, V, PN, P}
■ Σ = {girl, telescope, sandwich, I, saw, ate, with, in, a, the} ■ S = {S} ■ R =
N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP (NP a girl) (VP ate a sandwich) VP → V VP → V NP (V ate) (NP a sandwich) VP → VP PP (VP saw a girl) (PP with a telescope) NP → NP PP (NP a girl) (PP with a sandwich) NP → D N (D a) (N sandwich) NP → PN PP → P NP (P with) (NP a sandwich) inner rules
SLIDE 108
An example grammar
32
■ N = {S, VP
, NP , PP , N, V, PN, P}
■ Σ = {girl, telescope, sandwich, I, saw, ate, with, in, a, the} ■ S = {S} ■ R =
preterminal rules N → girl N → telescope N → sandwich PN → I V → saw V → ate P → with P → in D → a D → the S → NP VP (NP a girl) (VP ate a sandwich) VP → V VP → V NP (V ate) (NP a sandwich) VP → VP PP (VP saw a girl) (PP with a telescope) NP → NP PP (NP a girl) (PP with a sandwich) NP → D N (D a) (N sandwich) NP → PN PP → P NP (P with) (NP a sandwich) inner rules
SLIDE 109
Why context-free?
33
What can be a valid subtree is only effected by the phrase type (VP) but not the context.
SLIDE 110
Why context-free?
33
Example contexts: What can be a valid subtree is only effected by the phrase type (VP) but not the context.
SLIDE 111
Why context-free?
33
Example contexts: What can be a valid subtree is only effected by the phrase type (VP) but not the context. not grammatical
SLIDE 112
Ambiguity
34
■ Ambiguity makes parsing hard. ■ Example: coordination ambiguity ■ For example: coarse VP and NP categories can’t enforce subject-verb
agreement in number, resulting in this coordination ambiguity.
SLIDE 113
Ambiguity
34
■ Ambiguity makes parsing hard. ■ Example: coordination ambiguity ■ For example: coarse VP and NP categories can’t enforce subject-verb
agreement in number, resulting in this coordination ambiguity.
coordination
SLIDE 114
Ambiguity
34
■ Ambiguity makes parsing hard. ■ Example: coordination ambiguity ■ For example: coarse VP and NP categories can’t enforce subject-verb
agreement in number, resulting in this coordination ambiguity.
coordination bark may be a noun or a verb
SLIDE 115
Ambiguity
34
■ Ambiguity makes parsing hard. ■ Example: coordination ambiguity ■ For example: coarse VP and NP categories can’t enforce subject-verb
agreement in number, resulting in this coordination ambiguity.
coordination bark may be a noun or a verb this tree would be ruled out if the context could be captured (subject-verb agreement)
SLIDE 116
Ambiguity
35
■ Ambiguity makes parsing hard. ■ Example: prepositional phrase attachment ambiguity
SLIDE 117
Ambiguity
35
■ Ambiguity makes parsing hard. ■ Example: prepositional phrase attachment ambiguity
SLIDE 118
Ambiguity
35
■ Ambiguity makes parsing hard. ■ Example: prepositional phrase attachment ambiguity
SLIDE 119
Prepositional phrase ambiguity
36
SLIDE 120
Prepositional phrase ambiguity
36
“Put the block in the box on the table in the kitchen.”
SLIDE 121
Prepositional phrase ambiguity
36
■ 3 prepositional phrases, 5 interpretations: ■ Put the block ((in the box on the table) in the kitchen.) ■ Put the block (in the box (on the table in the kitchen.) ■ Put ((the block in the box) on the table) in the kitchen. ■ Put (the block (in the box on the table)) in the kitchen. ■ Put (the block in the box) (on the table in the kitchen.)
“Put the block in the box on the table in the kitchen.”
SLIDE 122
Prepositional phrase ambiguity
36
■ 3 prepositional phrases, 5 interpretations: ■ Put the block ((in the box on the table) in the kitchen.) ■ Put the block (in the box (on the table in the kitchen.) ■ Put ((the block in the box) on the table) in the kitchen. ■ Put (the block (in the box on the table)) in the kitchen. ■ Put (the block in the box) (on the table in the kitchen.)
“Put the block in the box on the table in the kitchen.”
■ General case:
SLIDE 123
Prepositional phrase ambiguity
36
■ 3 prepositional phrases, 5 interpretations: ■ Put the block ((in the box on the table) in the kitchen.) ■ Put the block (in the box (on the table in the kitchen.) ■ Put ((the block in the box) on the table) in the kitchen. ■ Put (the block (in the box on the table)) in the kitchen. ■ Put (the block in the box) (on the table in the kitchen.)
“Put the block in the box on the table in the kitchen.”
■ General case: ■ ((())) ()(()) ()()() (())() (()())
SLIDE 124
Prepositional phrase ambiguity
36
■ 3 prepositional phrases, 5 interpretations: ■ Put the block ((in the box on the table) in the kitchen.) ■ Put the block (in the box (on the table in the kitchen.) ■ Put ((the block in the box) on the table) in the kitchen. ■ Put (the block (in the box on the table)) in the kitchen. ■ Put (the block in the box) (on the table in the kitchen.)
“Put the block in the box on the table in the kitchen.”
■ General case: ■ ((())) ()(()) ()()() (())() (()())
Catalan numbers:
SLIDE 125
Typical tree
37
Canadian Utilities had 1988 revenue of $ 1.16 billion , mainly from its natural gas and electric utility businesses in Alberta , where the company serves about 800,000 customers .
SLIDE 126
More syntactic ambiguities
38
SLIDE 127
More syntactic ambiguities
38
■ Prepositional phrases:
They cooked the beans in the pot on the stove with handles.
SLIDE 128
More syntactic ambiguities
38
■ Prepositional phrases:
They cooked the beans in the pot on the stove with handles.
■ Particle vs. preposition:
The puppy tore up the staircase
SLIDE 129
More syntactic ambiguities
38
■ Prepositional phrases:
They cooked the beans in the pot on the stove with handles.
■ Particle vs. preposition:
The puppy tore up the staircase
■ Complement structures:
The tourists objected to the guide that they couldn’t hear. She knows you like the back of her hand.
SLIDE 130
More syntactic ambiguities
38
■ Prepositional phrases:
They cooked the beans in the pot on the stove with handles.
■ Particle vs. preposition:
The puppy tore up the staircase
■ Complement structures:
The tourists objected to the guide that they couldn’t hear. She knows you like the back of her hand.
■ Gerund vs. participal adjective:
Visiting relatives can be boring. Changing schedules frequently confused passengers.
SLIDE 131
More syntactic ambiguities
39
SLIDE 132
More syntactic ambiguities
39
■ Modifier scope within NPs:
impractical design requirements plastic cup holder
SLIDE 133
More syntactic ambiguities
39
■ Modifier scope within NPs:
impractical design requirements plastic cup holder
■ Multiple gap constructions:
The chicken is ready to eat. The contractors are rich enough to sue.
SLIDE 134
More syntactic ambiguities
39
■ Modifier scope within NPs:
impractical design requirements plastic cup holder
■ Multiple gap constructions:
The chicken is ready to eat. The contractors are rich enough to sue.
■ Coordination scope:
Small rats and mice and squeeze into holes or cracks in the wall.
SLIDE 135
Dark ambiguities
40
SLIDE 136
Dark ambiguities
40
■ Dark ambiguities: most analyses are shockingly bad (meaning, they don’t
have an interpretation you can get your mind around.)
SLIDE 137
Dark ambiguities
40
■ Dark ambiguities: most analyses are shockingly bad (meaning, they don’t
have an interpretation you can get your mind around.)
This analysis corresponds to the correct parse of: “This is panic buying!”
SLIDE 138
Dark ambiguities
40
■ Dark ambiguities: most analyses are shockingly bad (meaning, they don’t
have an interpretation you can get your mind around.)
■ Unknown words and new usages
This analysis corresponds to the correct parse of: “This is panic buying!”
SLIDE 139
Dark ambiguities
40
■ Dark ambiguities: most analyses are shockingly bad (meaning, they don’t
have an interpretation you can get your mind around.)
■ Unknown words and new usages ■ Solution: need mechanisms to focus attention on the best ones…
probabilistic techniques do this.
This analysis corresponds to the correct parse of: “This is panic buying!”
SLIDE 140
How to deal with ambiguity?
41
Put the block in the box on the table in the kitchen.
SLIDE 141
How to deal with ambiguity?
41
■ Want to score all derivations to encode how plausible they are.
Put the block in the box on the table in the kitchen.
SLIDE 142
Probabilistic context-free grammars (PCFGs)
42
N a set of non-terminal symbols (or variables) Σ a set of terminal symbols (disjoint from N) R a set of rules or productions, each of the form A → β , where A is a non-terminal, β is a string of symbols from the infinite set of strings (Σ∪N)∗ S a designated start symbol and a member of N
■ CFG: A 4-tuple (N, Σ, R, S):
SLIDE 143
Probabilistic context-free grammars (PCFGs)
42
■ A PCFG adds: a top-down production probability per rule.
N a set of non-terminal symbols (or variables) Σ a set of terminal symbols (disjoint from N) R a set of rules or productions, each of the form A → β , where A is a non-terminal, β is a string of symbols from the infinite set of strings (Σ∪N)∗ S a designated start symbol and a member of N
■ CFG: A 4-tuple (N, Σ, R, S):
SLIDE 144
Probabilistic context-free grammars (PCFGs)
42
■ A PCFG adds: a top-down production probability per rule. ■ If each rule is of the form X → Y1Y2…Yk
N a set of non-terminal symbols (or variables) Σ a set of terminal symbols (disjoint from N) R a set of rules or productions, each of the form A → β , where A is a non-terminal, β is a string of symbols from the infinite set of strings (Σ∪N)∗ S a designated start symbol and a member of N
■ CFG: A 4-tuple (N, Σ, R, S):
SLIDE 145
Probabilistic context-free grammars (PCFGs)
42
■ A PCFG adds: a top-down production probability per rule. ■ If each rule is of the form X → Y1Y2…Yk ■ Model its probability: P(Y1Y2…Yk | X)
N a set of non-terminal symbols (or variables) Σ a set of terminal symbols (disjoint from N) R a set of rules or productions, each of the form A → β , where A is a non-terminal, β is a string of symbols from the infinite set of strings (Σ∪N)∗ S a designated start symbol and a member of N
■ CFG: A 4-tuple (N, Σ, R, S):
SLIDE 146
An example PCFG
43
■ Associate probabilities with the rules:
N → girl 0.2 N → telescope 0.7 N → sandwich 0.1 PN → I 1.0 V → saw 0.5 V → ate 0.5 P → with 0.6 P → in 0.4 D → a 0.3 D → the 0.7 S → NP VP 1.0 (NP a girl) (VP ate a sandwich) VP → V 0.2 VP → V NP 0.4 (V ate) (NP a sandwich) VP → VP PP 0.4 (VP saw a girl) (PP with a telescope) NP → NP PP 0.3 (NP a girl) (PP with a sandwich) NP → D N 0.5 (D a) (N sandwich) NP → PN 0.2 PP → P NP 1.0 (P with) (NP a sandwich)
SLIDE 147
An example PCFG
43
■ Associate probabilities with the rules:
N → girl 0.2 N → telescope 0.7 N → sandwich 0.1 PN → I 1.0 V → saw 0.5 V → ate 0.5 P → with 0.6 P → in 0.4 D → a 0.3 D → the 0.7 S → NP VP 1.0 (NP a girl) (VP ate a sandwich) VP → V 0.2 VP → V NP 0.4 (V ate) (NP a sandwich) VP → VP PP 0.4 (VP saw a girl) (PP with a telescope) NP → NP PP 0.3 (NP a girl) (PP with a sandwich) NP → D N 0.5 (D a) (N sandwich) NP → PN 0.2 PP → P NP 1.0 (P with) (NP a sandwich)
P(X → α)
<latexit sha1_base64="Ga0C9SNYktqgHtEKe+P0Rd8YeI=">ADmXicfVLbtNAEN3GXEq4NC2PfTFElQpCUVKQ6GO5PFRIiCBIGykbReP12Fl1L9buOjSy/CW8wkfxN6ydUNVNxUheH82e2TlziTLBrev3/2y1gjt3793ftB+Ojxk53O7t6Z1blhOGJaDOwKLgCkeO4HjzCDISOB5dPGhuj9foLFcq+9umeFUQqp4whk475p1doaH45A6HVIQ2RxezDrdfq9fW7gJBmvQJWsbznZbhsa5RKVYwKsnQz6mZsWYBxnAs2zS1mwC4gxYmHCiTaVErL8MD74nDRBv/KRfW3usRBUhrlzLyTAlubm/eVc7b7ia5S46nBVdZ7lCxVaIkF6GvtGpDGHODzImlB8AM91pDNgcDzPlmtQ+up5mjWKBrFsLktLBJnb0hKZJlM/hyVWibGlT4g2kpQcUvC5qA5GIZYwK5cGVBbfIP39avV/GCZ3bduqsnBTqDU+5AiEwcbQ6m7/mztan236Ef2ADH72qr9kaMBp45WASVcln5gKX1GK/g/JldXTA+bZRW1AF9M1RedoSrKGjKhLdIoNTrPGoI34muh/gFI/BhWfGyGrRh+Swc3d3ITnB31Bq97R1/fdE/er/d1m+yT5+SQDMhbckJOyZCMCM5+Ul+kd/BfvAuOA0+raitrXMU9Kw4NtfQAE3AQ=</latexit> SLIDE 148
An example PCFG
43
■ Associate probabilities with the rules:
N → girl 0.2 N → telescope 0.7 N → sandwich 0.1 PN → I 1.0 V → saw 0.5 V → ate 0.5 P → with 0.6 P → in 0.4 D → a 0.3 D → the 0.7 S → NP VP 1.0 (NP a girl) (VP ate a sandwich) VP → V 0.2 VP → V NP 0.4 (V ate) (NP a sandwich) VP → VP PP 0.4 (VP saw a girl) (PP with a telescope) NP → NP PP 0.3 (NP a girl) (PP with a sandwich) NP → D N 0.5 (D a) (N sandwich) NP → PN 0.2 PP → P NP 1.0 (P with) (NP a sandwich)
P(X → α)
<latexit sha1_base64="Ga0C9SNYktqgHtEKe+P0Rd8YeI=">ADmXicfVLbtNAEN3GXEq4NC2PfTFElQpCUVKQ6GO5PFRIiCBIGykbReP12Fl1L9buOjSy/CW8wkfxN6ydUNVNxUheH82e2TlziTLBrev3/2y1gjt3793ftB+Ojxk53O7t6Z1blhOGJaDOwKLgCkeO4HjzCDISOB5dPGhuj9foLFcq+9umeFUQqp4whk475p1doaH45A6HVIQ2RxezDrdfq9fW7gJBmvQJWsbznZbhsa5RKVYwKsnQz6mZsWYBxnAs2zS1mwC4gxYmHCiTaVErL8MD74nDRBv/KRfW3usRBUhrlzLyTAlubm/eVc7b7ia5S46nBVdZ7lCxVaIkF6GvtGpDGHODzImlB8AM91pDNgcDzPlmtQ+up5mjWKBrFsLktLBJnb0hKZJlM/hyVWibGlT4g2kpQcUvC5qA5GIZYwK5cGVBbfIP39avV/GCZ3bduqsnBTqDU+5AiEwcbQ6m7/mztan236Ef2ADH72qr9kaMBp45WASVcln5gKX1GK/g/JldXTA+bZRW1AF9M1RedoSrKGjKhLdIoNTrPGoI34muh/gFI/BhWfGyGrRh+Swc3d3ITnB31Bq97R1/fdE/er/d1m+yT5+SQDMhbckJOyZCMCM5+Ul+kd/BfvAuOA0+raitrXMU9Kw4NtfQAE3AQ=</latexit>∀X → α ∈ R :
<latexit sha1_base64="Uut5BbxAe/gvdIr0Ktg9Ulcvms=">ADtnicfVLbhMxEHWzXMpyS+GRF0MUqaCqSgoSFU8V8MALIiDSRopD5HVmN1Z9k+0NjVb7E3wNr/AX/A3eTaiyTcVI6z0az3jOmZnECO58r/dnpxXduHnr9u6d+O69+w8etvcenTqdWwZDpoW2o4Q6EFzB0HMvYGQsUJkIOEvO31X3Zwuwjmv1S8NTCTNFE85oz64pu2D7mB/hInXmFBh5vR5TFJtqRB404sJV/jLm2m70zvs1Ya3QX8NOmhtg+ley5KZrkE5Zmgzo37PeMnBbWeMwFlTHIHhrJzmsE4QEUluElRypxN3hmONAJn/K49m5mFQ6t5RJiJTUz93Vu8p53d049+nxpODK5B4UWxVKc4GD3qpHeMYtMC+WAVBmeCK2ZxaynzoZNzdLDMHsQDfFMLkpHBpXb1BKZFlM/liJTQmFhR8Z1pKqmYvCpJSycVyBinNhS8L4tJ/+Lp+HcwW3Lh16y6fFOCJtjzjKgwTUk+qo+kOv7kn9RmT9xAGZOFjYP3JgKVe28CE2kzSizIMLCNPSQX/F8nVZWSATVlFTSCIqfqiDairCET2gFJMqtz0yC8lV8TDQ/QNIxhFQ/NtFVE2NL+1Z3cBqdHh/2Xh0efX3VO3q73dRc9Qc/QPuqj1+gEfUADNEQM/UA/0S/0OzqOvkUQZavQ1s465zFqWGT+AjmAQY=</latexit>0 ≤ P(X → α) ≤ 1
<latexit sha1_base64="Fpl2uK2s9UVUIUwQuE2xnXW1p+4=">AD1HicfVJbixMxGM2XtZ62a4+hIthVWkzKyC4tOiPogVrHdQlNKJvPNGwuY5KpW8Y+ia/+Fn+N4JP+FNOZunS2i4HJHE6+k5zvEmWCWxcEv3YazUuXr1zdvda6fuPmrb32/u2h1blhMGBaDOKqAXBFQwcdwJGmQEqIwH0cnL1fnxHIzlWn10iwmkqaKJ5xR56lp+023fzDCxGlMqMhm9EGrSxJtqB4k8aEK/zheSvARMAn7DUbkoLp+1O0AvKhbdBuAYdtF796X7DkFizXIJyTFBrx2GQuUlBjeNMwLJFcgsZSc0hbGHikqwk6JMeom7nomx9+o/5XDJbioKq1dyMhHSupm9vzZirzobJy75Nmk4CrLHShWPZTkAvuEVxXEMTfAnFh4QJnh3itmM2oc7Ovnwbz8xAzMHVE2FyUtikfL1mKZLuvi0SrRFDCj4zLSUVMUPC5JQycUihoTmwi0LYpN/+KJ6PYrnPLPr0p1dKcARbXjKle80JI6stjrtfzNHyr1FXoFvkIG3vW7DAx12ngn1KSni59w1Jyj6zg/yK5Oov0sJ5WURrwyazqojNQxbKETGgLJEqNzrOa4S19adRfQBPfhioe6rIqwk9peH4mt8HwsBc+7h2+f9I5erGe10F91HByhET9EReo36aIAY+oF+ot/oT3PY/NL82vxWhTZ21po7qLa3/8CUZFLuA=</latexit> SLIDE 149
An example PCFG
43
■ Associate probabilities with the rules:
N → girl 0.2 N → telescope 0.7 N → sandwich 0.1 PN → I 1.0 V → saw 0.5 V → ate 0.5 P → with 0.6 P → in 0.4 D → a 0.3 D → the 0.7 S → NP VP 1.0 (NP a girl) (VP ate a sandwich) VP → V 0.2 VP → V NP 0.4 (V ate) (NP a sandwich) VP → VP PP 0.4 (VP saw a girl) (PP with a telescope) NP → NP PP 0.3 (NP a girl) (PP with a sandwich) NP → D N 0.5 (D a) (N sandwich) NP → PN 0.2 PP → P NP 1.0 (P with) (NP a sandwich)
P(X → α)
<latexit sha1_base64="Ga0C9SNYktqgHtEKe+P0Rd8YeI=">ADmXicfVLbtNAEN3GXEq4NC2PfTFElQpCUVKQ6GO5PFRIiCBIGykbReP12Fl1L9buOjSy/CW8wkfxN6ydUNVNxUheH82e2TlziTLBrev3/2y1gjt3793ftB+Ojxk53O7t6Z1blhOGJaDOwKLgCkeO4HjzCDISOB5dPGhuj9foLFcq+9umeFUQqp4whk475p1doaH45A6HVIQ2RxezDrdfq9fW7gJBmvQJWsbznZbhsa5RKVYwKsnQz6mZsWYBxnAs2zS1mwC4gxYmHCiTaVErL8MD74nDRBv/KRfW3usRBUhrlzLyTAlubm/eVc7b7ia5S46nBVdZ7lCxVaIkF6GvtGpDGHODzImlB8AM91pDNgcDzPlmtQ+up5mjWKBrFsLktLBJnb0hKZJlM/hyVWibGlT4g2kpQcUvC5qA5GIZYwK5cGVBbfIP39avV/GCZ3bduqsnBTqDU+5AiEwcbQ6m7/mztan236Ef2ADH72qr9kaMBp45WASVcln5gKX1GK/g/JldXTA+bZRW1AF9M1RedoSrKGjKhLdIoNTrPGoI34muh/gFI/BhWfGyGrRh+Swc3d3ITnB31Bq97R1/fdE/er/d1m+yT5+SQDMhbckJOyZCMCM5+Ul+kd/BfvAuOA0+raitrXMU9Kw4NtfQAE3AQ=</latexit>∀X → α ∈ R :
<latexit sha1_base64="Uut5BbxAe/gvdIr0Ktg9Ulcvms=">ADtnicfVLbhMxEHWzXMpyS+GRF0MUqaCqSgoSFU8V8MALIiDSRopD5HVmN1Z9k+0NjVb7E3wNr/AX/A3eTaiyTcVI6z0az3jOmZnECO58r/dnpxXduHnr9u6d+O69+w8etvcenTqdWwZDpoW2o4Q6EFzB0HMvYGQsUJkIOEvO31X3Zwuwjmv1S8NTCTNFE85oz64pu2D7mB/hInXmFBh5vR5TFJtqRB404sJV/jLm2m70zvs1Ya3QX8NOmhtg+ley5KZrkE5Zmgzo37PeMnBbWeMwFlTHIHhrJzmsE4QEUluElRypxN3hmONAJn/K49m5mFQ6t5RJiJTUz93Vu8p53d049+nxpODK5B4UWxVKc4GD3qpHeMYtMC+WAVBmeCK2ZxaynzoZNzdLDMHsQDfFMLkpHBpXb1BKZFlM/liJTQmFhR8Z1pKqmYvCpJSycVyBinNhS8L4tJ/+Lp+HcwW3Lh16y6fFOCJtjzjKgwTUk+qo+kOv7kn9RmT9xAGZOFjYP3JgKVe28CE2kzSizIMLCNPSQX/F8nVZWSATVlFTSCIqfqiDairCET2gFJMqtz0yC8lV8TDQ/QNIxhFQ/NtFVE2NL+1Z3cBqdHh/2Xh0efX3VO3q73dRc9Qc/QPuqj1+gEfUADNEQM/UA/0S/0OzqOvkUQZavQ1s465zFqWGT+AjmAQY=</latexit>0 ≤ P(X → α) ≤ 1
<latexit sha1_base64="Fpl2uK2s9UVUIUwQuE2xnXW1p+4=">AD1HicfVJbixMxGM2XtZ62a4+hIthVWkzKyC4tOiPogVrHdQlNKJvPNGwuY5KpW8Y+ia/+Fn+N4JP+FNOZunS2i4HJHE6+k5zvEmWCWxcEv3YazUuXr1zdvda6fuPmrb32/u2h1blhMGBaDOKqAXBFQwcdwJGmQEqIwH0cnL1fnxHIzlWn10iwmkqaKJ5xR56lp+023fzDCxGlMqMhm9EGrSxJtqB4k8aEK/zheSvARMAn7DUbkoLp+1O0AvKhbdBuAYdtF796X7DkFizXIJyTFBrx2GQuUlBjeNMwLJFcgsZSc0hbGHikqwk6JMeom7nomx9+o/5XDJbioKq1dyMhHSupm9vzZirzobJy75Nmk4CrLHShWPZTkAvuEVxXEMTfAnFh4QJnh3itmM2oc7Ovnwbz8xAzMHVE2FyUtikfL1mKZLuvi0SrRFDCj4zLSUVMUPC5JQycUihoTmwi0LYpN/+KJ6PYrnPLPr0p1dKcARbXjKle80JI6stjrtfzNHyr1FXoFvkIG3vW7DAx12ngn1KSni59w1Jyj6zg/yK5Oov0sJ5WURrwyazqojNQxbKETGgLJEqNzrOa4S19adRfQBPfhioe6rIqwk9peH4mt8HwsBc+7h2+f9I5erGe10F91HByhET9EReo36aIAY+oF+ot/oT3PY/NL82vxWhTZ21po7qLa3/8CUZFLuA=</latexit>∀X ∈ N :
<latexit sha1_base64="49MoRPeQT+rHJt1OmDWQ1ZXC1o=">AD5nicfVLbhMxFHUTHmV4pbBkY4giFYSqmYIE6qoCFmyAgEgbKY4ij+fOjFU/BtsTGo3yC+wQW76Fv+AP2MIX4ExClWkqLI3n6Poe+9xzb1wIbl0Y/txqtS9dvnJ1+1pw/cbNW7c7O3eOrC4NgwHTQpthTC0IrmDguBMwLAxQGQs4jk9eLs6Pp2As1+qjmxUwljRTPOWMOh+adEivzvExGlMqChy+jDokVQbKgReD2PCFf5wEPRCTAR8wp60xlnGomCN6dPfHkw63XAvrBfeBNEKdNFq9Sc7LUMSzUoJyjFBrR1FYeHGFTWOMwHzgJQWCspOaAYjDxWVYMdVbcMc93wkwV6D/5TDdXSdUVFp7UzGPlNSl9vzZ4vgRWej0qXPxVXRelAseVDaSmwd2DhKU64AebEzAPKDPdaMcupocx572ha8/kIKbgmoUwOa5sWr/ekBTLeZN8uiw0IAYUfGZaSqSRxVJqeRilkBKS+HmFbHpP3yRX4+TKS/syrqzKwU4og3PuPIdhNSRxdYM+1/uSL0H5BX4Bhl41W/K8BQp41XQk0m6encNywj98kC/i+Tq7NMD5tlVbUAX8zCF12AquY1ZEJbIHFmdFk0BG/wa6H+Apr6NizoUlbZvgpjc7P5CY42t+Lnuztv3/aPXyxmtdtdA89QLsoQs/QIXqN+miAGPqBfqHf6E87b39pf21/W6a2tlacu6ix2t/AiehUeo=</latexit>X
α:X→α∈R
P(X → α) = 1
<latexit sha1_base64="t0fKPrJku4jtGJsFL4Ue2p0Q0w=">AEHicfVJbixMxFE47XtbxtquPgkSXYhUpM6ugFIRFfBFreJeYFNKJj3Ths1lTDLrlmH+hD/AX+Ob+Cr4T3w0nalLp10MTObj5PtyvnNOkxw6Lod6sdXLh46fLGlfDqtes3bm5u3dq3OjcM9pgW2hwm1ILgCvYcdwIOMwNUJgIOkuNX8/ODEzCWa/XJzTIYSjpRPOWMOh8abX7tDLqHmDiNCRXZlD4MOyTVhgqBl8OYcIU/9sNOhImAz9iLljR1LG5IPf9dPyQ2l6OipvXLyzxSnb8Asejze2oF1ULr4N4AbRYg1GW21DxprlEpRjglp7FEeZGxbUOM4ElCHJLWSUHdMJHmoqAQ7LKrmlbjI2PsjftPOVxFlxUFldbOZOKZkrqpXT2bB87O8pd+nxYcJXlDhSrE6W5wL7a+STwmBtgTsw8oMxw7xWzKTWUOT8v38ulNFMQJ+CahTA5LGxaZW9YSmTZFJ/WhYbEgIvTEtJ1fhRQVIquZiNIaW5cGVBbPoPn9evx+MTntlF686uFOCINnzClR87pI7Mt2bY/6aOVHtIXoMfkIG3vX7DAx12ngn1EwkPS39wCbkHpnD/zG5OmN62CyrqAz4YuZ90RmoqwgE9oCSZG51nD8Jq+MuovoKkfQ82Hpqxm+Fcar7JdbC/04uf9HY+PN3efbl4rxvoDrqPuihGz9AueoMGaA8x9Kd1t/Wg1Q2+Bd+DH8HPmtpuLTS3UWMFv/4C5yZig=</latexit> SLIDE 150
An example PCFG
43
■ Associate probabilities with the rules:
N → girl 0.2 N → telescope 0.7 N → sandwich 0.1 PN → I 1.0 V → saw 0.5 V → ate 0.5 P → with 0.6 P → in 0.4 D → a 0.3 D → the 0.7 S → NP VP 1.0 (NP a girl) (VP ate a sandwich) VP → V 0.2 VP → V NP 0.4 (V ate) (NP a sandwich) VP → VP PP 0.4 (VP saw a girl) (PP with a telescope) NP → NP PP 0.3 (NP a girl) (PP with a sandwich) NP → D N 0.5 (D a) (N sandwich) NP → PN 0.2 PP → P NP 1.0 (P with) (NP a sandwich) Now we can score a tree as a product
- f probabilities
corresponding to the used rules!
P(X → α)
<latexit sha1_base64="Ga0C9SNYktqgHtEKe+P0Rd8YeI=">ADmXicfVLbtNAEN3GXEq4NC2PfTFElQpCUVKQ6GO5PFRIiCBIGykbReP12Fl1L9buOjSy/CW8wkfxN6ydUNVNxUheH82e2TlziTLBrev3/2y1gjt3793ftB+Ojxk53O7t6Z1blhOGJaDOwKLgCkeO4HjzCDISOB5dPGhuj9foLFcq+9umeFUQqp4whk475p1doaH45A6HVIQ2RxezDrdfq9fW7gJBmvQJWsbznZbhsa5RKVYwKsnQz6mZsWYBxnAs2zS1mwC4gxYmHCiTaVErL8MD74nDRBv/KRfW3usRBUhrlzLyTAlubm/eVc7b7ia5S46nBVdZ7lCxVaIkF6GvtGpDGHODzImlB8AM91pDNgcDzPlmtQ+up5mjWKBrFsLktLBJnb0hKZJlM/hyVWibGlT4g2kpQcUvC5qA5GIZYwK5cGVBbfIP39avV/GCZ3bduqsnBTqDU+5AiEwcbQ6m7/mztan236Ef2ADH72qr9kaMBp45WASVcln5gKX1GK/g/JldXTA+bZRW1AF9M1RedoSrKGjKhLdIoNTrPGoI34muh/gFI/BhWfGyGrRh+Swc3d3ITnB31Bq97R1/fdE/er/d1m+yT5+SQDMhbckJOyZCMCM5+Ul+kd/BfvAuOA0+raitrXMU9Kw4NtfQAE3AQ=</latexit>∀X → α ∈ R :
<latexit sha1_base64="Uut5BbxAe/gvdIr0Ktg9Ulcvms=">ADtnicfVLbhMxEHWzXMpyS+GRF0MUqaCqSgoSFU8V8MALIiDSRopD5HVmN1Z9k+0NjVb7E3wNr/AX/A3eTaiyTcVI6z0az3jOmZnECO58r/dnpxXduHnr9u6d+O69+w8etvcenTqdWwZDpoW2o4Q6EFzB0HMvYGQsUJkIOEvO31X3Zwuwjmv1S8NTCTNFE85oz64pu2D7mB/hInXmFBh5vR5TFJtqRB404sJV/jLm2m70zvs1Ya3QX8NOmhtg+ley5KZrkE5Zmgzo37PeMnBbWeMwFlTHIHhrJzmsE4QEUluElRypxN3hmONAJn/K49m5mFQ6t5RJiJTUz93Vu8p53d049+nxpODK5B4UWxVKc4GD3qpHeMYtMC+WAVBmeCK2ZxaynzoZNzdLDMHsQDfFMLkpHBpXb1BKZFlM/liJTQmFhR8Z1pKqmYvCpJSycVyBinNhS8L4tJ/+Lp+HcwW3Lh16y6fFOCJtjzjKgwTUk+qo+kOv7kn9RmT9xAGZOFjYP3JgKVe28CE2kzSizIMLCNPSQX/F8nVZWSATVlFTSCIqfqiDairCET2gFJMqtz0yC8lV8TDQ/QNIxhFQ/NtFVE2NL+1Z3cBqdHh/2Xh0efX3VO3q73dRc9Qc/QPuqj1+gEfUADNEQM/UA/0S/0OzqOvkUQZavQ1s465zFqWGT+AjmAQY=</latexit>0 ≤ P(X → α) ≤ 1
<latexit sha1_base64="Fpl2uK2s9UVUIUwQuE2xnXW1p+4=">AD1HicfVJbixMxGM2XtZ62a4+hIthVWkzKyC4tOiPogVrHdQlNKJvPNGwuY5KpW8Y+ia/+Fn+N4JP+FNOZunS2i4HJHE6+k5zvEmWCWxcEv3YazUuXr1zdvda6fuPmrb32/u2h1blhMGBaDOKqAXBFQwcdwJGmQEqIwH0cnL1fnxHIzlWn10iwmkqaKJ5xR56lp+023fzDCxGlMqMhm9EGrSxJtqB4k8aEK/zheSvARMAn7DUbkoLp+1O0AvKhbdBuAYdtF796X7DkFizXIJyTFBrx2GQuUlBjeNMwLJFcgsZSc0hbGHikqwk6JMeom7nomx9+o/5XDJbioKq1dyMhHSupm9vzZirzobJy75Nmk4CrLHShWPZTkAvuEVxXEMTfAnFh4QJnh3itmM2oc7Ovnwbz8xAzMHVE2FyUtikfL1mKZLuvi0SrRFDCj4zLSUVMUPC5JQycUihoTmwi0LYpN/+KJ6PYrnPLPr0p1dKcARbXjKle80JI6stjrtfzNHyr1FXoFvkIG3vW7DAx12ngn1KSni59w1Jyj6zg/yK5Oov0sJ5WURrwyazqojNQxbKETGgLJEqNzrOa4S19adRfQBPfhioe6rIqwk9peH4mt8HwsBc+7h2+f9I5erGe10F91HByhET9EReo36aIAY+oF+ot/oT3PY/NL82vxWhTZ21po7qLa3/8CUZFLuA=</latexit>∀X ∈ N :
<latexit sha1_base64="49MoRPeQT+rHJt1OmDWQ1ZXC1o=">AD5nicfVLbhMxFHUTHmV4pbBkY4giFYSqmYIE6qoCFmyAgEgbKY4ij+fOjFU/BtsTGo3yC+wQW76Fv+AP2MIX4ExClWkqLI3n6Poe+9xzb1wIbl0Y/txqtS9dvnJ1+1pw/cbNW7c7O3eOrC4NgwHTQpthTC0IrmDguBMwLAxQGQs4jk9eLs6Pp2As1+qjmxUwljRTPOWMOh+adEivzvExGlMqChy+jDokVQbKgReD2PCFf5wEPRCTAR8wp60xlnGomCN6dPfHkw63XAvrBfeBNEKdNFq9Sc7LUMSzUoJyjFBrR1FYeHGFTWOMwHzgJQWCspOaAYjDxWVYMdVbcMc93wkwV6D/5TDdXSdUVFp7UzGPlNSl9vzZ4vgRWej0qXPxVXRelAseVDaSmwd2DhKU64AebEzAPKDPdaMcupocx572ha8/kIKbgmoUwOa5sWr/ekBTLeZN8uiw0IAYUfGZaSqSRxVJqeRilkBKS+HmFbHpP3yRX4+TKS/syrqzKwU4og3PuPIdhNSRxdYM+1/uSL0H5BX4Bhl41W/K8BQp41XQk0m6encNywj98kC/i+Tq7NMD5tlVbUAX8zCF12AquY1ZEJbIHFmdFk0BG/wa6H+Apr6NizoUlbZvgpjc7P5CY42t+Lnuztv3/aPXyxmtdtdA89QLsoQs/QIXqN+miAGPqBfqHf6E87b39pf21/W6a2tlacu6ix2t/AiehUeo=</latexit>X
α:X→α∈R
P(X → α) = 1
<latexit sha1_base64="t0fKPrJku4jtGJsFL4Ue2p0Q0w=">AEHicfVJbixMxFE47XtbxtquPgkSXYhUpM6ugFIRFfBFreJeYFNKJj3Ths1lTDLrlmH+hD/AX+Ob+Cr4T3w0nalLp10MTObj5PtyvnNOkxw6Lod6sdXLh46fLGlfDqtes3bm5u3dq3OjcM9pgW2hwm1ILgCvYcdwIOMwNUJgIOkuNX8/ODEzCWa/XJzTIYSjpRPOWMOh8abX7tDLqHmDiNCRXZlD4MOyTVhgqBl8OYcIU/9sNOhImAz9iLljR1LG5IPf9dPyQ2l6OipvXLyzxSnb8Asejze2oF1ULr4N4AbRYg1GW21DxprlEpRjglp7FEeZGxbUOM4ElCHJLWSUHdMJHmoqAQ7LKrmlbjI2PsjftPOVxFlxUFldbOZOKZkrqpXT2bB87O8pd+nxYcJXlDhSrE6W5wL7a+STwmBtgTsw8oMxw7xWzKTWUOT8v38ulNFMQJ+CahTA5LGxaZW9YSmTZFJ/WhYbEgIvTEtJ1fhRQVIquZiNIaW5cGVBbPoPn9evx+MTntlF686uFOCINnzClR87pI7Mt2bY/6aOVHtIXoMfkIG3vX7DAx12ngn1EwkPS39wCbkHpnD/zG5OmN62CyrqAz4YuZ90RmoqwgE9oCSZG51nD8Jq+MuovoKkfQ82Hpqxm+Fcar7JdbC/04uf9HY+PN3efbl4rxvoDrqPuihGz9AueoMGaA8x9Kd1t/Wg1Q2+Bd+DH8HPmtpuLTS3UWMFv/4C5yZig=</latexit> SLIDE 151
PCFGs
44
S → NP VP 1.0 VP → V 0.2 VP → V NP 0.4 VP → VP PP 0.4 NP → NP PP 0.3 NP → D N 0.5 NP → PN 0.2 PP → P NP 1.0 N → girl 0.2 N → telescope 0.7 N → sandwich 0.1 PN → I 1.0 V → saw 0.5 V → ate 0.5 P → with 0.6 P → in 0.4 D → a 0.3 D → the 0.7
SLIDE 152
PCFGs
44
S → NP VP 1.0 VP → V 0.2 VP → V NP 0.4 VP → VP PP 0.4 NP → NP PP 0.3 NP → D N 0.5 NP → PN 0.2 PP → P NP 1.0 N → girl 0.2 N → telescope 0.7 N → sandwich 0.1 PN → I 1.0 V → saw 0.5 V → ate 0.5 P → with 0.6 P → in 0.4 D → a 0.3 D → the 0.7 P(T) = 1.0 * 0.2 * 1.0 * 0.4 * 0.5 * 0.3 * 0.5 * 0.3 * 0.2 * 1.0 * 0.6 * 0.5 * 0.3 * 0.7 = 2.26e-5
SLIDE 153
PCFGs
45
S → NP VP 1.0 VP → V 0.2 VP → V NP 0.4 VP → VP PP 0.4 NP → NP PP 0.3 NP → D N 0.5 NP → PN 0.2 PP → P NP 1.0 N → girl 0.2 N → telescope 0.7 N → sandwich 0.1 PN → I 1.0 V → saw 0.5 V → ate 0.5 P → with 0.6 P → in 0.4 D → a 0.3 D → the 0.7 P(T) = 1.0 * 0.2 * 1.0 * 0.4 * 0.5 * 0.3 * 0.5 * 0.3 * 0.2 * 1.0 * 0.6 * 0.5 * 0.3 * 0.7 = 2.26e-5 1.0
SLIDE 154
PCFGs
46
S → NP VP 1.0 VP → V 0.2 VP → V NP 0.4 VP → VP PP 0.4 NP → NP PP 0.3 NP → D N 0.5 NP → PN 0.2 PP → P NP 1.0 N → girl 0.2 N → telescope 0.7 N → sandwich 0.1 PN → I 1.0 V → saw 0.5 V → ate 0.5 P → with 0.6 P → in 0.4 D → a 0.3 D → the 0.7 P(T) = 1.0 * 0.2 * 1.0 * 0.4 * 0.5 * 0.3 * 0.5 * 0.3 * 0.2 * 1.0 * 0.6 * 0.5 * 0.3 * 0.7 = 2.26e-5 1.0 0.2
SLIDE 155
PCFGs
47
S → NP VP 1.0 VP → V 0.2 VP → V NP 0.4 VP → VP PP 0.4 NP → NP PP 0.3 NP → D N 0.5 NP → PN 0.2 PP → P NP 1.0 N → girl 0.2 N → telescope 0.7 N → sandwich 0.1 PN → I 1.0 V → saw 0.5 V → ate 0.5 P → with 0.6 P → in 0.4 D → a 0.3 D → the 0.7 P(T) = 1.0 * 0.2 * 1.0 * 0.4 * 0.5 * 0.3 * 0.5 * 0.3 * 0.2 * 1.0 * 0.6 * 0.5 * 0.3 * 0.7 = 2.26e-5 1.0 0.2 1.0
SLIDE 156
PCFGs
48
S → NP VP 1.0 VP → V 0.2 VP → V NP 0.4 VP → VP PP 0.4 NP → NP PP 0.3 NP → D N 0.5 NP → PN 0.2 PP → P NP 1.0 N → girl 0.2 N → telescope 0.7 N → sandwich 0.1 PN → I 1.0 V → saw 0.5 V → ate 0.5 P → with 0.6 P → in 0.4 D → a 0.3 D → the 0.7 P(T) = 1.0 * 0.2 * 1.0 * 0.4 * 0.5 * 0.3 * 0.5 * 0.3 * 0.2 * 1.0 * 0.6 * 0.5 * 0.3 * 0.7 = 2.26e-5 1.0 0.2 1.0 0.4
SLIDE 157
PCFGs
49
S → NP VP 1.0 VP → V 0.2 VP → V NP 0.4 VP → VP PP 0.4 NP → NP PP 0.3 NP → D N 0.5 NP → PN 0.2 PP → P NP 1.0 N → girl 0.2 N → telescope 0.7 N → sandwich 0.1 PN → I 1.0 V → saw 0.5 V → ate 0.5 P → with 0.6 P → in 0.4 D → a 0.3 D → the 0.7 P(T) = 1.0 * 0.2 * 1.0 * 0.4 * 0.5 * 0.3 * 0.5 * 0.3 * 0.2 * 1.0 * 0.6 * 0.5 * 0.3 * 0.7 = 2.26e-5 1.0 0.2 1.0 0.4 0.5
SLIDE 158
PCFGs
50
S → NP VP 1.0 VP → V 0.2 VP → V NP 0.4 VP → VP PP 0.4 NP → NP PP 0.3 NP → D N 0.5 NP → PN 0.2 PP → P NP 1.0 N → girl 0.2 N → telescope 0.7 N → sandwich 0.1 PN → I 1.0 V → saw 0.5 V → ate 0.5 P → with 0.6 P → in 0.4 D → a 0.3 D → the 0.7 P(T) = 1.0 * 0.2 * 1.0 * 0.4 * 0.5 * 0.3 * 0.5 * 0.3 * 0.2 * 1.0 * 0.6 * 0.5 * 0.3 * 0.7 = 2.26e-5 1.0 0.2 1.0 0.4 0.5 0.3
SLIDE 159
PCFGs
51
S → NP VP 1.0 VP → V 0.2 VP → V NP 0.4 VP → VP PP 0.4 NP → NP PP 0.3 NP → D N 0.5 NP → PN 0.2 PP → P NP 1.0 N → girl 0.2 N → telescope 0.7 N → sandwich 0.1 PN → I 1.0 V → saw 0.5 V → ate 0.5 P → with 0.6 P → in 0.4 D → a 0.3 D → the 0.7 P(T) = 1.0 * 0.2 * 1.0 * 0.4 * 0.5 * 0.3 * 0.5 * 0.3 * 0.2 * 1.0 * 0.6 * 0.5 * 0.3 * 0.7 = 2.26e-5 1.0 0.2 1.0 0.4 0.5 0.3 1.0 0.5 0.7 0.3 0.6 0.2 0.3 0.5
SLIDE 160
Context-free grammars (CFGs)
52
S → NP VP 1.0 VP → V 0.2 VP → V NP 0.4 VP → VP PP 0.4 NP → NP PP 0.3 NP → D N 0.5 NP → PN 0.2 PP → P NP 1.0 N → girl 0.2 N → telescope 0.7 N → sandwich 0.1 PN → I 1.0 V → saw 0.5 V → ate 0.5 P → with 0.6 P → in 0.4 D → a 0.3 D → the 0.7 P(T) = 1.0 * 0.2 * 1.0 * 0.4 * 0.5 * 0.3 * 0.5 * 0.3 * 0.2 * 1.0 * 0.6 * 0.5 * 0.3 * 0.7 = 2.26e-5 1.0 0.2 1.0 0.4 0.5 0.3 1.0 0.5 0.7 0.3 0.6 0.2 0.3 0.5
SLIDE 161
PCFG estimation
53
SLIDE 162
■ A treebank: a collection of sentences annotated with constituency trees
PCFG estimation
53
…
SLIDE 163
■ A treebank: a collection of sentences annotated with constituency trees ■ Estimated probability of a rule (maximum likelihood estimate):
PCFG estimation
53
P(X → α) = C(X → α) C(X)
<latexit sha1_base64="4eVtiSWvoIouNRYJWQy/6AoClfM=">AES3icfVLbtNAFHWaFkp4pbBkMxBFtAhFSUECVUKqKAs2hYBIW6kTRePJdTLqPMzMODSy/HF8AB/Ad7BDLi2QxUnFSPZPr5zn2HsRTOd7s/axv1za0bN7dvNW7fuXvfnPnwYkzieUw4EYaexYyB1JoGHjhJZzFpgKJZyGF0f5/ekMrBNGf/HzGIaKTbSIBGceTaPm93Z/94xQbwhlMp6yvUabRsYyKcmymVChyeDRrtLqISvBEVLmtLWq0iR/wH51CVqlJa8g3WPGVkJT96gn3UbjSzj6VElapb/72WjZqvb6RaHrIPeArSCxemPdjYsHRueKNCeS+bcea8b+2HKrBdcQtagiYOY8Qs2gXOEmilw7TodUbaBkTrBMf7UlhXVakTDk3VyEyFfNTt3qXG6+7O0989HqYCh0nHjQvA0WJFhwPjgyFha4l3MEjFuBuRI+ZdgWj+PFRi+FmYKcga8WwtUwdVERvZJSqLKq+LIstEtaPjGjVJMj5+lNGJKyPkYIpZIn6XURf/wdf16Pp6J2C1ad+VSgqfGionQuCUQeZq/qmb8TD0t3g36DnBAFo4x648xWOaNxUyYnSh2meHAJvQxzeH/mEJfMRFWy0qLBLCYvC8mBp1mBeTSOKDhxJokriS8pi8SRQcswjGUfKjKSgZuaW91J9fByX6n96Kz/+l6/DtYl+3g0fBk2A36AWvgsPgfdAPBgGvPa0d105qp/Uf9V/13/U/JXWjtA8DCpnc+svC6xYQ=</latexit>…
SLIDE 164
■ A treebank: a collection of sentences annotated with constituency trees ■ Estimated probability of a rule (maximum likelihood estimate):
PCFG estimation
53
P(X → α) = C(X → α) C(X)
<latexit sha1_base64="4eVtiSWvoIouNRYJWQy/6AoClfM=">AES3icfVLbtNAFHWaFkp4pbBkMxBFtAhFSUECVUKqKAs2hYBIW6kTRePJdTLqPMzMODSy/HF8AB/Ad7BDLi2QxUnFSPZPr5zn2HsRTOd7s/axv1za0bN7dvNW7fuXvfnPnwYkzieUw4EYaexYyB1JoGHjhJZzFpgKJZyGF0f5/ekMrBNGf/HzGIaKTbSIBGceTaPm93Z/94xQbwhlMp6yvUabRsYyKcmymVChyeDRrtLqISvBEVLmtLWq0iR/wH51CVqlJa8g3WPGVkJT96gn3UbjSzj6VElapb/72WjZqvb6RaHrIPeArSCxemPdjYsHRueKNCeS+bcea8b+2HKrBdcQtagiYOY8Qs2gXOEmilw7TodUbaBkTrBMf7UlhXVakTDk3VyEyFfNTt3qXG6+7O0989HqYCh0nHjQvA0WJFhwPjgyFha4l3MEjFuBuRI+ZdgWj+PFRi+FmYKcga8WwtUwdVERvZJSqLKq+LIstEtaPjGjVJMj5+lNGJKyPkYIpZIn6XURf/wdf16Pp6J2C1ad+VSgqfGionQuCUQeZq/qmb8TD0t3g36DnBAFo4x648xWOaNxUyYnSh2meHAJvQxzeH/mEJfMRFWy0qLBLCYvC8mBp1mBeTSOKDhxJokriS8pi8SRQcswjGUfKjKSgZuaW91J9fByX6n96Kz/+l6/DtYl+3g0fBk2A36AWvgsPgfdAPBgGvPa0d105qp/Uf9V/13/U/JXWjtA8DCpnc+svC6xYQ=</latexit>…
# times the rule was used in the corpus
SLIDE 165
■ A treebank: a collection of sentences annotated with constituency trees ■ Estimated probability of a rule (maximum likelihood estimate):
PCFG estimation
53
P(X → α) = C(X → α) C(X)
<latexit sha1_base64="4eVtiSWvoIouNRYJWQy/6AoClfM=">AES3icfVLbtNAFHWaFkp4pbBkMxBFtAhFSUECVUKqKAs2hYBIW6kTRePJdTLqPMzMODSy/HF8AB/Ad7BDLi2QxUnFSPZPr5zn2HsRTOd7s/axv1za0bN7dvNW7fuXvfnPnwYkzieUw4EYaexYyB1JoGHjhJZzFpgKJZyGF0f5/ekMrBNGf/HzGIaKTbSIBGceTaPm93Z/94xQbwhlMp6yvUabRsYyKcmymVChyeDRrtLqISvBEVLmtLWq0iR/wH51CVqlJa8g3WPGVkJT96gn3UbjSzj6VElapb/72WjZqvb6RaHrIPeArSCxemPdjYsHRueKNCeS+bcea8b+2HKrBdcQtagiYOY8Qs2gXOEmilw7TodUbaBkTrBMf7UlhXVakTDk3VyEyFfNTt3qXG6+7O0989HqYCh0nHjQvA0WJFhwPjgyFha4l3MEjFuBuRI+ZdgWj+PFRi+FmYKcga8WwtUwdVERvZJSqLKq+LIstEtaPjGjVJMj5+lNGJKyPkYIpZIn6XURf/wdf16Pp6J2C1ad+VSgqfGionQuCUQeZq/qmb8TD0t3g36DnBAFo4x648xWOaNxUyYnSh2meHAJvQxzeH/mEJfMRFWy0qLBLCYvC8mBp1mBeTSOKDhxJokriS8pi8SRQcswjGUfKjKSgZuaW91J9fByX6n96Kz/+l6/DtYl+3g0fBk2A36AWvgsPgfdAPBgGvPa0d105qp/Uf9V/13/U/JXWjtA8DCpnc+svC6xYQ=</latexit>…
# times the rule was used in the corpus # times nonterminal X appeared in the treebank
SLIDE 166
■ A treebank: a collection of sentences annotated with constituency trees ■ Estimated probability of a rule (maximum likelihood estimate): ■ Smoothing is helpful (especially for preterminal rules).
PCFG estimation
53
P(X → α) = C(X → α) C(X)
<latexit sha1_base64="4eVtiSWvoIouNRYJWQy/6AoClfM=">AES3icfVLbtNAFHWaFkp4pbBkMxBFtAhFSUECVUKqKAs2hYBIW6kTRePJdTLqPMzMODSy/HF8AB/Ad7BDLi2QxUnFSPZPr5zn2HsRTOd7s/axv1za0bN7dvNW7fuXvfnPnwYkzieUw4EYaexYyB1JoGHjhJZzFpgKJZyGF0f5/ekMrBNGf/HzGIaKTbSIBGceTaPm93Z/94xQbwhlMp6yvUabRsYyKcmymVChyeDRrtLqISvBEVLmtLWq0iR/wH51CVqlJa8g3WPGVkJT96gn3UbjSzj6VElapb/72WjZqvb6RaHrIPeArSCxemPdjYsHRueKNCeS+bcea8b+2HKrBdcQtagiYOY8Qs2gXOEmilw7TodUbaBkTrBMf7UlhXVakTDk3VyEyFfNTt3qXG6+7O0989HqYCh0nHjQvA0WJFhwPjgyFha4l3MEjFuBuRI+ZdgWj+PFRi+FmYKcga8WwtUwdVERvZJSqLKq+LIstEtaPjGjVJMj5+lNGJKyPkYIpZIn6XURf/wdf16Pp6J2C1ad+VSgqfGionQuCUQeZq/qmb8TD0t3g36DnBAFo4x648xWOaNxUyYnSh2meHAJvQxzeH/mEJfMRFWy0qLBLCYvC8mBp1mBeTSOKDhxJokriS8pi8SRQcswjGUfKjKSgZuaW91J9fByX6n96Kz/+l6/DtYl+3g0fBk2A36AWvgsPgfdAPBgGvPa0d105qp/Uf9V/13/U/JXWjtA8DCpnc+svC6xYQ=</latexit>…
# times the rule was used in the corpus # times nonterminal X appeared in the treebank
SLIDE 167
Distribution over trees
54
SLIDE 168
■ We defined a distribution over production rules for each nonterminal.
Distribution over trees
54
SLIDE 169
■ We defined a distribution over production rules for each nonterminal. ■ Our goal was to define a distribution over parse trees.
Distribution over trees
54
SLIDE 170
■ We defined a distribution over production rules for each nonterminal. ■ Our goal was to define a distribution over parse trees. ■ Unfortunately, not all PCFGs result in a proper distribution over trees, i.e. the
sum over probabilities of all trees in the grammar may be less than 1.
Distribution over trees
54
SLIDE 171
■ We defined a distribution over production rules for each nonterminal. ■ Our goal was to define a distribution over parse trees. ■ Unfortunately, not all PCFGs result in a proper distribution over trees, i.e. the
sum over probabilities of all trees in the grammar may be less than 1.
■ Fortunately: any PCFG estimated by maximum likelihood is always proper [Chi
and Geman, 1998].
Distribution over trees
54
SLIDE 172
Announcements
55