Grammar: Features and Unification Plan for the Talk Problems with - - PowerPoint PPT Presentation

▶
grammar features and unification plan for the talk
SMART_READER_LITE
LIVE PREVIEW

Grammar: Features and Unification Plan for the Talk Problems with - - PowerPoint PPT Presentation

May 04, 2023 24 likes •509 views

Grammar: Features and Unification Plan for the Talk Problems with CFG (PCFG) Features Structure Attribute-value Matrix (AVM) Unification Grammar formalisms based on unification Agreement Constraints that hold among various

slide-1
SLIDE 1

Grammar: Features and Unification

slide-2
SLIDE 2

Plan for the Talk

 Problems with CFG (PCFG)  Features Structure

 Attribute-value Matrix (AVM)

 Unification  Grammar formalisms based on unification

slide-3
SLIDE 3

Agreement

 Constraints that hold among various constituents.  For example, in English, determiners and the head nouns

in NPs have to agree in their number.

 Which of the following cannot be parsed by the rule

NP  Det Nominal ?

(O) This flight (O) Those flights (X) This flights (X) Those flight

slide-4
SLIDE 4

Agreement

 Constraints that hold among various constituents.  For example, in English, determiners and the head nouns

in NPs have to agree in their number.

 Which of the following cannot be parsed by the rule

NP  Det Nominal ?  This rule does not handle agreement! (The rule does not detect whether the agreement is correct or not.)

(O) This flight (O) Those flights (X) This flights (X) Those flight

slide-5
SLIDE 5

Problem with CFG/PCFG

 Our earlier NP rules are clearly deficient since they

don’t capture the agreement constraint

 NP  Det Nominal

 Accepts, and assigns correct structures, to grammatical examples

(this flight)

 But its also happy with incorrect examples (*these flight)

 Such a rule is said to overgenerate.  We’ll come back to this in a bit

slide-6
SLIDE 6

Verb Phrases

 English VPs consist of a head verb along with 0 or more

following constituents which we’ll call arguments.

slide-7
SLIDE 7

Subcategorization

 *John sneezed the book  *I prefer United has a flight  *Give with a flight  As with agreement phenomena, we need a way to

formally express the constraints!

slide-8
SLIDE 8

Subcategorization

 Sneeze: John sneezed  Find: Please find [a flight to NY]NP  Give: Give [me]NP[a cheaper fare]NP  Help: Can you help [me]NP[with a flight]PP  Prefer: I prefer [to leave earlier]TO-VP  Told: I was told [United has a flight]S  …

slide-9
SLIDE 9

Subcategorization

 But, even though there are many valid VP rules in

English, not all verbs are allowed to participate in all those VP rules.

 We can subcategorize the verbs in a language

according to the sets of VP rules that they participate in.

 This is a modern take on the traditional notion of

transitive/intransitive.

 Modern grammars may have 100s or such classes.

slide-10
SLIDE 10

Problem with CFG/PCFG

 Right now, the various rules for VPs overgenerate.

 They permit the presence of strings containing verbs and

arguments that don’t go together

 For example  VP -> V NP therefore

Sneezed the book is a VP since “sneeze” is a verb and “the book” is a valid NP

slide-11
SLIDE 11

Possible CFG Solution

 Possible solution for

agreement.

 Can use the same trick for

all the verb/VP classes.

 SgS -> SgNP SgVP  PlS -> PlNp PlVP  SgNP -> SgDet SgNom  PlNP -> PlDet PlNom  PlVP -> PlV NP  SgVP ->SgV Np  …

slide-12
SLIDE 12

CFG Solution for Agreement

 Pro:

 It works and stays within the power of CFGs

 Con:

 loss of generalization – “apple” and “apples” are treated

as if they are two separate words

 And it doesn’t scale all that well because of the

interaction among the various constraints explodes the number of rules in our grammar.

slide-13
SLIDE 13

Non-CFG Solution for Agreement

 Add “constraints” to each

rule

 S -> NP VP

constraint: only if the number of NP is equal to the number of the VP

 Instead of replicating rules…

 SgS -> SgNP SgVP  PlS -> PlNp PlVP  SgNP -> SgDet SgNom  PlNP -> PlDet PlNom  PlVP -> PlV NP  SgVP ->SgV Np  …

slide-14
SLIDE 14

Plan for the Talk

 Problems with CFG (PCFG)  Features Structure

 Attribute-value Matrix (AVM)

 Unification  Grammar formalisms based on unification

slide-15
SLIDE 15

Feature Structure

 “Features” in formal grammar  “Features” in machine learning  Attribute-value Matrix (AVM)

 Feature Path  Reentrant structure

slide-16
SLIDE 16

Feature Structure

This feature structure is used in many grammar formalism that goes beyond CFG, such as

 Head-Driven Phrase Structure Grammar (HPSG)

(Pollard and Sag, 1987, 1994)

 Lexical Functional Grammar (LFG) (Bresnan, 1982)  Construction Grammar (Kay and Fillmore, 1999)  Unification Categorial Grammar (Uszkoreit, 1986)

slide-17
SLIDE 17

Attribute-value matrix (AVM)

Definition: FEATURE_1 value_1 FEATURE_2 value_2 …. FEATURE_n value_n For example: NUMBER sg

slide-18
SLIDE 18

Attribute-value matrix (AVM)

More Examples: CAT NP NUMBER sg PERSON 3rd

slide-19
SLIDE 19

Attribute-value matrix (AVM)

Hierarchical Structure: “value” can be another AVM object CAT NP NUMBER sg PERSON 3rd CAT NP AGREEMENT NUMBER sg PERSON 3rd

slide-20
SLIDE 20

Feature Path

Feature Path: a sequence of features in the feature structure (AVM) leading to a particular value CAT NP AGREEMENT NUMBER sg PERSON 3rd

slide-21
SLIDE 21

Feature Path

Feature Path: a sequence of features in the feature structure (AVM) leading to a particular value CAT NP AGREEMENT NUMBER sg PERSON 3rd

slide-22
SLIDE 22

Attribute-value matrix (AVM)

Reentrant Structure:

CAT S HEAD AGREEMENT [1] NUMBER sg PERSON 3rd SUBJECT AGREEMENT [1]

slide-23
SLIDE 23

Reentrant Structure:

CAT S HEAD AGREEMENT [1] NUMBER sg PERSON 3rd SUBJECT AGREEMENT [1]

Feature Path:

slide-24
SLIDE 24

Feature Structure

 “Features” in formal grammar  “Features” in machine learning  Attribute-value Matrix (AVM)

 Feature Path  Reentrant structure

 This feature structure is used in many grammar

formalism that goes beyond CFG, such as HPSG, LFG

slide-25
SLIDE 25

Plan for the Talk

 Problems with CFG (PCFG)  Features Structure

 Attribute-value Matrix (AVM)

 Unification  Grammar formalisms based on unification

slide-26
SLIDE 26

Unification of Feature Structure

 Unification of two feature structure (AVM) finds the

most general feature structure that is compatible with the two given AVMs.

 [ NUMBER sg ] U [ NUMBER sg ] =  [ NUMBER sg ] U [ NUMBER pl ] =  [ NUMBER sg ] U [ NUMBER [ ] ] =

slide-27
SLIDE 27

Unification of Feature Structure

 Unification of two feature structure (AVM) finds the

most general feature structure that is compatible with the two given AVMs.

 [ NUMBER sg ] U [ NUMBER sg ] = [ NUMBER sg ]  [ NUMBER sg ] U [ NUMBER pl ]  Fails !  [ NUMBER sg ] U [ NUMBER [ ] ] = [ NUMBER sg ]

slide-28
SLIDE 28

Unification of Feature Structure

 Unification of two feature structure (AVM) finds the

most general feature structure that is compatible with the two given AVMs.

 [ NUMBER sg ] U [ PERSON 3rd ] =

slide-29
SLIDE 29

Unification of Feature Structure

 Unification of two feature structure (AVM) finds the

most general feature structure that is compatible with the two given AVMs.

 [ NUMBER sg ] U [ PERSON 3rd ] = NUMBER sg ?

PERSON 3rd CATEGORY NP

slide-30
SLIDE 30

Unification of Feature Structure

 Unification of two feature structure (AVM) finds the

most general feature structure that is compatible with the two given AVMs.

 [ NUMBER sg ] U [ PERSON 3rd ] = NUMBER sg ?

PERSON 3rd CATEGORY NP

slide-31
SLIDE 31

Unification of Feature Structure

 Unification of two feature structure (AVM) finds the

most general feature structure that is compatible with the two given AVMs.

 [ NUMBER sg ] U [ PERSON 3rd ] = NUMBER sg

PERSON 3rd

slide-32
SLIDE 32

Unification of Feature Structure

AGREEMENT [1] NUMBER sg PERSON 3rd SUBJECT AGREEMENT [1] U SUBJECT AGREEMENT PERSON 3rd NUMBER sg =

slide-33
SLIDE 33

Unification of Feature Structure

AGREEMENT [1] NUMBER sg PERSON 3rd SUBJECT AGREEMENT [1] U SUBJECT AGREEMENT PERSON 3rd NUMBER sg = AGREEMENT [1] NUMBER sg PERSON 3rd SUBJECT AGREEMENT [1]

slide-34
SLIDE 34

Unification of Feature Structure

AGREEMENT [1] SUBJECT AGREEMENT [1] U SUBJECT AGREEMENT PERSON 3rd NUMBER sg =

slide-35
SLIDE 35

Unification of Feature Structure

AGREEMENT [1] SUBJECT AGREEMENT [1] U SUBJECT AGREEMENT PERSON 3rd NUMBER sg = AGREEMENT [1] SUBJECT AGREEMENT [1] PERSON 3rd NUMBER sg

slide-36
SLIDE 36

Unification of Feature Structure

AGREEMENT [1] NUMBER sg PERSON 3rd SUBJECT AGREEMENT [1] U AGREEMENT NUMBER sg PERSON 3rd SUBJECT AGREEMENT PERSON 3rd NUMBER pl =

slide-37
SLIDE 37

Unification of Feature Structure

AGREEMENT [1] NUMBER sg PERSON 3rd SUBJECT AGREEMENT [1] U AGREEMENT NUMBER sg PERSON 3rd SUBJECT AGREEMENT PERSON 3rd NUMBER pl Fails!

slide-38
SLIDE 38

Unification of Feature Structure

AGREEMENT NUMBER sg SUBJECT AGREEMENT NUMBER sg U SUBJECT AGREEMENT PERSON 3rd NUMBER sg

slide-39
SLIDE 39

Unification of Feature Structure

AGREEMENT NUMBER sg SUBJECT AGREEMENT NUMBER sg U SUBJECT AGREEMENT PERSON 3rd NUMBER sg = AGREEMENT NUMBER sg SUBJECT AGREEMENT PERSON 3rd NUMBER sg

slide-40
SLIDE 40

Plan for the Talk

 Problems with CFG (PCFG)  Features Structure

 Attribute-value Matrix (AVM)

 Unification  Grammar formalisms based on unification

slide-41
SLIDE 41

Grammar Theories based on Unification

 Head-Driven Phrase Structure Grammar (HPSG)

(Pollard and Sag, 1987, 1994)

 Lexical Functional Grammar (LFG) (Bresnan, 1982)  Construction Grammar (Kay and Fillmore, 1999)  Unification Categorial Grammar (Uszkoreit, 1986)  Note that these grammar formalisms tend to focus on

illuminating syntactic analysis, rather than providing computational implementations. (computationally very expensive)

slide-42
SLIDE 42

Example of AVM used in HPSG

 Head-Driven Phrase Structure Grammar (HPSG)

(Pollard and Sag, 1987, 1994)

 Non-derivational  Constraint-based  Highly lexicalized

 Each word is fully described with

 morpho-syntactic features  semantic features

slide-43
SLIDE 43

Example of AVM used in HPSG

 “put” --- e.g., “John put a book on the table”

slide-44
SLIDE 44

Example of AVM used in HPSG

 Each word can have many different AVM descriptions

(due to polysemy, or multiple possible syntactic relations with other words/phrases)

 each lexical_entry corresponds to an AVM description

such as shown below: word  lexical_entry_1

V lexical_entry_2 V … V lexical_entry_n

slide-45
SLIDE 45
slide-46
SLIDE 46

The Chomsky Hierarchy

slide-47
SLIDE 47

The Chomsky Hierarchy

 Head-Driven Phrase Structure Grammar (HPSG) (Pollard and Sag,

1987, 1994)

 Lexical Functional Grammar (LFG) (Bresnan, 1982)  Minimalist Grammar (Stabler, 1997)  Tree-Adjoining Grammars (TAG) (Joshi, 1985)  Combinatory Categorial Grammars (CCG) (Steedman, 1996, 2000)

slide-48
SLIDE 48

Turing Test

 Turing Test: Interrogator

‘c’ engages in a natural language conversation with ‘a’ and ‘b’ to determine which is a computer and which is a human.