CS344: Introduction to CS344: Introduction to Artificial - - PowerPoint PPT Presentation

CS344: Introduction to CS344: Introduction to Artificial Intelligence g

Pushpak Bhattacharyya Pushpak Bhattacharyya CSE Dept., IIT Bombay IIT Bombay Lecture 20-21– Natural Language Parsing Parsing

Parsing of Sentences Parsing of Sentences

Are sentences flat linear structures? Why tree?

Is there a principle in branching When should the constituent give rise When should the constituent give rise

to children? What is the hierarchy building principle?

What is the hierarchy building principle?

Structure Dependency: A Case Study

• I nterrogative I nversion
• I nterrogative I nversion

(1) John will solve the problem. Will John solve the problem? Will John solve the problem? Declarative Interrogative (2) a. Susan must leave. Must Susan leave? (2) a. Susan must leave. Must Susan leave?

• b. Harry can swim.

Can Harry swim? c. Mary has read the book. Has Mary read the book? c a y as ead e boo as a y ead e boo

d.

Bill is sleeping. Is Bill sleeping?

……………………………………………………….

The section, “Structure dependency a case study” here is adopted from a talk given by Howard Lasnik (2003) in Delhi university. g y ( ) y

Interrogative inversion St ct e Independent (1st attempt) Structure Independent (1st attempt)

(3)I nterrogative inversion process Beginning with a declarative, invert the first and second words to construct an interrogative. Declarative Interrogative Declarative Interrogative (4) a. The woman must leave. * Woman the must leave?

• b. A sailor can swim.

* Sailor a can swim?

• b. A sailor can swim.

Sailor a can swim?

• c. No boy has read the book.

* Boy no has read the book?

• d. My friend is sleeping.

* Friend my is sleeping?

Interrogative inversion correct pairings correct pairings

• Compare the incorrect pairings in (4) with the

t i i i (5) correct pairings in (5):

Declarative I nterrogative

(5)

Th t l M t th l ?

(5) a. The woman must leave.

Must the woman leave?

• b. A sailor can swim.

Can a sailor swim? c No boy has read the book Has no boy read the book?

• c. No boy has read the book. Has no boy read the book?
• d. My friend is sleeping.

Is my friend sleeping?

Interrogative inversion Structure Independent (2nd attempt) p ( p )

(6) I nterrogative inversion process:

B i i ith d l ti th ili

Beginning with a declarative, move the auxiliary

verb to the front to construct an interrogative.

Declarative Interrogative Declarative Interrogative (7) a. Bill could be sleeping. * Be Bill could sleeping? Could Bill be sleeping? Could Bill be sleeping?

• b. Mary has been reading.

* Been Mary has reading? Has Mary been reading? y g

• c. Susan should have left.

* Have Susan should left? Should Susan have left?

Structure independent (3rd attempt):

(8) Interrogative inversion process

B i i ith d l ti th fi t

• Beginning with a declarative, move the first

auxiliary verb to the front to construct an interrogative interrogative.

Declarative Interrogative

(9) Th h i h i * I th h h i ? (9) a. The man who is here can swim. * Is the man who here can swim?

• b. The boy who will play has left.

* Will the boy who play has left?

Structure Dependent Correct Pairings

For

the above examples, fronting the second

For

the above examples, fronting the second auxiliary verb gives the correct form:

Declarative Interrogative Declarative Interrogative (10) a.The man who is here can swim. Can the man who is here swim?

b.The boy who will play has left. Has the boy who will play left?

Natural transformations are structure dependent

(11) Does the child acquiring English learn these properties? (11) Does the child acquiring English learn these properties?

(12) We are not dealing with a peculiarity of English. No known human language has a transformational process g g p that would produce pairings like those in (4), (7) and (9), repeated below:

(4) a. The woman must leave. * Woman the must leave? (7) a. Bill could be sleeping. * Be Bill could sleeping?

(9) a The man who is here can swim * Is the man who here can swim? (9) a. The man who is here can swim. * Is the man who here can swim?

Deeper trees needed for capturing sentence structure

This wont do! NP Flat structure! PP PP AP The with the blue cover book big

• f poems

[The big book of poems with the [The big book of poems with the Blue cover] is on the table.

Other languages

English NP English PP PP AP The with the blue cover book big

• f poems

NP Hi di PP AP kitaab Hindi PP [niil jilda vaalii kavita kii kitaab] niil jilda vaalii kavita kii badii

Other languages: contd

English NP English PP PP AP The with the blue cover book big

• f poems

NP B li PP AP Bengali PP [niil malaat deovaa kavitar bai ti] niil malaat deovaa kavitar bai motaa ti

PPs are at the same level: flat with respect to the head word “book”

No distinction in terms of dominance or c command NP dominance or c-command PP PP AP The with the blue cover book big

• f poems

[The big book of poems with the [The big book of poems with the Blue cover] is on the table.

“Constituency test of Replacement” runs Constituency test of Replacement runs into problems

One-replacement:

I bought the big [book of poems with the I bought the big [book of poems with the

blue cover] not the small [one]

One-replacement targets book of poems

One replacement targets book of poems with the blue cover

Another one-replacement: Another one replacement:

I bought the big [book of poems] with the

blue cover not the small [one] with the red blue cover not the small [one] with the red cover

One-replacement targets book of poems

More deeply embedded structure

NP AP The N’1 big PP N’2 N’3 big with the blue cover N PP

3

PP with the blue cover N book

• f poems

To target N1’

I want [ NPthis [ N’big book of poems with

the red cover] and not [ Nthat [ None]] the red cover] and not [ Nthat [ None]]

Bar-level projections

Add intermediate structures Add intermediate structures

NP (D) N’

N’ (AP) N’ | N’ (PP) | N (PP)

N (AP) N | N (PP) | N (PP)

() indicates optionality

New rules produce this tree

NP

N-bar

AP The N’1 big PP N’2 N’3 big with the blue cover N PP

3

PP with the blue cover N book

• f poems

As opposed to this tree

NP PP PP AP The with the blue cover book big

• f poems

V-bar

What is the element in verbs

corresponding to one-replacement for corresponding to one replacement for nouns

do-so or did-so do-so or did-so

As opposed to this tree

NP PP PP AP The with the blue cover book big

• f poems

I [eat beans with a fork]

VP PP NP eat with a fork beans No constituent that groups together V and NP and excludes No constituent that groups together V and NP and excludes PP

Need for intermediate constituents

I [eat beans] with a fork but Ram [does

so] with a spoon

VP

so] with a spoon

V2’ V1’ VPV’ V’ V’ (PP) PP V V’ V’ (PP) V’ V (NP) NP eat with a fork V beans

How to target V1’

I [eat beans with a fork], and Ram

[does so] too.

VP

[does so] too.

V2’ V1’ VPV’ V’ V’ (PP) PP V V’ V’ (PP) V’ V (NP) NP eat with a fork V beans

Parsing Algorithms Parsing Algorithms

A simplified grammar

S → NP VP

NP → DT N | N VP → V ADV | V

A segment of English Grammar

S’(C) S S{ NP/S’} VP S{ NP/S} VP VP(AP+ ) (VAUX) V (AP+ )

({ NP/S’} ) (AP+ ) (PP+ ) (AP+ ) ({ NP/S’} ) (AP+ ) (PP+ ) (AP+ )

NP(D) (AP+ ) N (PP+ ) PPP NP AP(AP) A

( )

Example Sentence Example Sentence

People laugh eop e aug

1

2 3

These are positions

Lexicon: Lexicon: People - N, V Laugh N V

This indicate that both Noun and Verb is possible for the word “People”

Laugh - N, V

People

Top-Down Parsing Top-Down Parsing

State Backup State Action

• 1.

((S) 1) -

• 2. ((NP VP)1) -
• 3a. ((DT N VP)1) ((N VP) 1) -

3b ((N VP)1)

Position of input pointer

• 3b. ((N VP)1) -
• 4. ((VP)2) -

Consume “People”

• 5a. ((V ADV)2) ((V)2) -
• 6. ((ADV)3) ((V)2) Consume “laugh”
• 5b. ((V)2) -
• 6. ((.)3) -

Consume “laugh” Termination Condition : All inputs over. No symbols remaining. Note: Input symbols can be pushed back.

Discussion for Top-Down Parsing

This kind of searching is goal driven This kind of searching is goal driven. Gives importance to textual precedence (rule

precedence).

No regard for data, a priori (useless expansions

made).

Bottom-Up Parsing Bottom-Up Parsing

Some conventions: N12

Represents positions

S1? -> NP12 ° VP2?

End position unknown Work on the LHS done, while the work on RHS remaining

Bottom-Up Parsing (pictorial representation)

S -> NP 2 VP23 ° S > NP12 VP23

People Laugh 1 2 3 1 2 3

N12 N23

12 23

V12 V23 NP12 -> N12 ° NP23 -> N23 ° VP12 -> V12 ° VP23 -> V23 °

12 12 23 23

S1? -> NP12 ° VP2?

Problem with Top-Down Parsing

• Left Recursion
• Suppose you have A-> AB rule
• Suppose you have A > AB rule.

Then we will have the expansion as follows: follows:

• ((A)K) -> ((AB)K) -> ((ABB)K) ……..

C bi i t d d Combining top-down and bottom-up strategies bottom up strategies

Top-Down Bottom-Up Chart p p Parsing

Combines advantages of top-down & bottom-

up parsing. p p g

Does not work in case of left recursion.

e g – “People laugh” e.g.

People laugh

People – noun, verb Laugh – noun, verb

Grammar –

S → NP VP

NP → DT N | N VP → V ADV | V

Transitive Closure

People laugh 1 2 3 1 2 3

S →•NP VP NP →N• VP → V • NP →•DT N S → NP•VP S → NP VP • NP →•N VP →•V ADV success NP → N VP → V ADV success VP →•V

Arcs in Parsing

Each arc represents a chart which

records records

Completed work (left of •) Expected work (right of •) Expected work (right of •)

Example

People laugh loudly 1 2 3 4

S →• NP VP NP → N• VP → V• VP → V ADV• NP →• DT N S → NP•VP VP → V•ADV S → NP VP• NP →• DT N S → NP•VP VP → V•ADV S → NP VP• NP →• N VP → •V ADV S → NP VP• VP → •V

Dealing With Structural Ambiguity

Multiple parses for a sentence

The man saw the boy with a telescope The man saw the boy with a telescope. The man saw the mountain with a

telescope. telescope.

The man saw the boy with the ponytail.

At the level of syntax all these sentences At the level of syntax, all these sentences are ambiguous. But semantics can disambiguate 2nd & 3rd sentence disambiguate 2 & 3 sentence.

Prepositional Phrase (PP) p ( ) Attachment Problem

V – NP1 – P – NP2 (Here P means preposition) (Here P means preposition) NP2 attaches to NP1 ? h

• r NP2 attaches to V ?

Parse Trees for a Structurally y Ambiguous Sentence

Let the grammar be – S → NP VP NP → DT N | DT N PP PP → P NP PP → P NP VP → V NP PP | V NP For the sentence For the sentence, “I saw a boy with a telescope”

Parse Tree - 1

S NP VP N V NP N V NP Det N PP

I saw Det N

PP P NP

I saw a boy

Det N

with a telescope

Parse Tree -2

S NP VP N V NP PP N V NP Det N PP P NP

I saw Det N

Det N

I saw a boy with a telescope