Lecture 14: Formal Grammars Kai-Wei Chang CS @ University of - - PowerPoint PPT Presentation

▶

Mar 31, 2023 633 likes •964 views

Lecture 14: Formal Grammars Kai-Wei Chang CS @ University of Virginia kw@kwchang.net Couse webpage: http://kwchang.net/teaching/NLP16 CS6501: NLP 1 Critical review report (due 10/20) v 1 page maximum v Pick one paper from the suggested list

SLIDE 1

Lecture 14: Formal Grammars

Kai-Wei Chang CS @ University of Virginia kw@kwchang.net Couse webpage: http://kwchang.net/teaching/NLP16

1 CS6501: NLP

SLIDE 2

Critical review report (due 10/20)

v 1 page maximum v Pick one paper from the suggested list v Summarize the paper (use you own words) v Provide detailed comments

vWhat can be improved vPotential future directions vOther related work

v Example (see reviewer 3 at: https://goo.gl/yrBXXo)

CS6501– Natural Language Processing 2

SLIDE 3

How to model language?

v So far, all the models we saw formulate sentence as a sequence

vLanguage models vPOS-tagging vMorphological analysis

CS6501: NLP 3

SLIDE 4

Next key concepts

v In the following few weeks, we will go beyond sequence models v Syntactic parsing – model language as a recursive generating process

vOften use a tree structure

CS6501: NLP 4

SLIDE 5

What is grammar?

v A compact way to define and describe the structure of sentences v Why we need grammar?

vNumber of C++ programs?

CS6501: NLP 5

976 pages. C++ standard (2014) ISO/IEC 14882:2014 1358 pages

SLIDE 6

Can we define a program that generates all English sentences?

CS6501: NLP 6

From Julia Hockenmaier, Intro to NLP

SLIDE 7

Basic sentence structure

CS6501: NLP 7

SLIDE 8

A Markov Model

v I eat shshi; I eat meat; you eat banana… v Great, it covers many sentences

CS6501: NLP 8

SLIDE 9

Words take different arguments

v [Good] I eat sushi v [Bad] I sleep sushi v [Bad] I give sushi v Intransitive verbs (sleep): no object v Transitive verbs (eat): take one direct object v Ditransitive verbs (give): take an additional indirect object.

CS6501: NLP 9

SLIDE 10

A better model

CS6501: NLP 10

SLIDE 11

Language is recursive

CS6501: NLP 11

Adjectives can modify nouns. We can have unlimited modifiers (in theory)

SLIDE 12

We know how to model the simple one

CS6501: NLP 12

SLIDE 13

Recursion can be more complex

CS6501: NLP 13

We can have another noun phrase in preposition

SLIDE 14

What is sentence structure

v Sentence structure is hierarchical

vA sentence consists of phrases (or constituents)

CS6501: NLP 14

SLIDE 15

Can have complex constituents

CS6501: NLP 15

SLIDE 16

Can have complex constituents

v Syntactically, constituents behave like simple ones

CS6501: NLP 16

SLIDE 17

Constituency

v Groups of words that behave as a single unit or phrase

vE.g., Noun phrases: the man, a girl with glasses vPrepositional phrases: with classes, on a table vVerb phrase: eat sushi, sleep, sleep soundly

v Phrases has a head:

vOther parts called dependents vE.g, the man, a girl with glasses

CS6501: NLP 17

SLIDE 18

Properties of constituents

v Substitution

vHe talks [in class] ⇒ He talks [there]

v It can move around in a sentence

vHe talks [in class] ⇒ [In class], he talks.

v Can be used as an answer:

vWhere does he talk? – [In class]

CS6501: NLP 18

SLIDE 19

Types of dependencies

v Phrases has a head:

vOther parts called dependents vE.g, the man, a girl with glasses

v Dependents can be arguments or adjuncts v Arguments are obligatory

vE.g., [John] likes [Mary]

v Adjuncts are optional

vE.g., John runs [fast] vAdverbs, PPs, Adjectives…

CS6501: NLP 19

All arguments have to be present and cannot be

ccupied multiple times

Can be an arbitrary number

f adjuncts

SLIDE 20

How to represent the structure

CS6501: NLP 20

SLIDE 21

Structure (syntax) corresponds to meaning

CS6501: NLP 21

SLIDE 22

Dependency Trees

v Dependency grammar describe the structure of sentences as a graph (tree)

vNodes represent words vEdges represent dependencies

CS6501: NLP 22

SLIDE 23

Phrases structure trees

v Can be modeled by Context-free grammars

CS6501: NLP 23

SLIDE 24

Context-free grammars

CS6501: NLP 24

SLIDE 25

Parse tree defined by CFG

CS6501: NLP 25

SLIDE 26

Generate sentences by CFG

CS6501: NLP 26

SLIDE 27

Example: Noun Phrases

CS6501: NLP 27

SLIDE 28

Example: verb phrase

CS6501: NLP 28

SLIDE 29

Sentences

CS6501: NLP 29

SLIDE 30

Structured Prediction –beyond sequence tagging

Task Input Output

Part-of-speech Tagging They operate ships and banks. Dependency Parsing They operate ships and banks. Segmentation

Pronoun Verb Noun And Noun This image cannot currently be displayed. This image cannot currently be displayed.

Root They operate ships and banks .

Assign values to a set of interdependent output variables

SLIDE 31

Next few lectures

v Extend HMM to deal with the tree structure v Discriminative methods for tree structure

Kai-Wei Chang 31