[PPT] - Introduction to Natural Language Processing PARSING: Earley, PowerPoint Presentation

SLIDE 1

✬ ✫ ✩ ✪ Introduction to Natural Language Processing

PARSING: Earley, Bottom-Up Chart Parsing

SLIDE 2

✬ ✫ ✩ ✪ Objectives of this lecture ➥ After CYK algorithm, present two other algorithms used for syntactic parsing

SLIDE 3

✬ ✫ ✩ ✪

Earley Parsing Top-down algorithm(predictive) Bottom-up = inference Top-down = search 3 advantages:

✌ best known worst-case complexity (as CYK) ✌ adaptive complexity for least complex languages (e.g. regular languages) ✌ No need for a special form of the CF grammar

2 drawbacks :

➷ No way to correct/reconstruct non-parsable sentences (”early error detection”) ➷ not very intuitive

SLIDE 4

✬ ✫ ✩ ✪

Earley Parsing (2) Idea: on-line (i.e during parsing) binarization of the grammar

☞ doted rules and ”Earley items”

doted rules: X → X1...Xk • Xk+1...Xm with X → X1...Xm a rule of the grammar Earley item: one doted rule with one integer i (0 ≤ i ≤ n, n: size of the input string)

☞ the part before the dot (•) represents the subpart of the rule that derives a substring of

the input string starting at position i + 1 Example: (VP → V • NP, 2) is an Earley item for input string the cat ate a mouse 1 2 3 4 5

SLIDE 5

✬ ✫ ✩ ✪

Earley Parsing (3) Principle: Starting from all possible (S → • X1 ... Xm, 0), parallel construction of all the dotted rules deriving (larger and larger) substrings of the input string, up to a point where the whole input sentence is derived

☞ construction of sets of items (Ej) such that: (X → α • β, i) ∈ Ej ⇐ ⇒ ∃γ, δ : S ⇒∗ γ X δ

and

γ ⇒∗ w1 ... wi

and

α ⇒∗ wi+1 ... wj

Example: in the former example (VP → V • NP, 2) ∈ E3 The input string (length n) is syntactically correct (accepted) iff at least one

(S → X1 ... Xm •, 0) is in En

SLIDE 6

✬ ✫ ✩ ✪

Earley Parsing (4)

➊ Initialization: construction of E0

to E0

to E0

SLIDE 7

✬ ✫ ✩ ✪

Earley Parsing: Interpretation

➋ Iterations: building of derivations of w1...wj (Ej)

”eat” wj (i.e. ”there is a • before wj”)

starting at wi+1 and if there exists one subderivation ending in wi which can ”eat” X, do it!

introduce all the rules that might (later on) produce Y

SLIDE 8

✬ ✫ ✩ ✪

Earley Parsing (next)

➋ Iterations: construction of the Ej sets (1 ≤ j ≤ n)

add (Y → α X • β, k) to Ej

add (X → • γ, j) to Ej

SLIDE 9

✬ ✫ ✩ ✪

Earley Parsing: Full Example Example for ”I think”, and the grammar:

S → NP VP Pron → I NP → Pron V → think NP → Det N VP → V VP → V S VP → V NP

SLIDE 10

✬ ✫ ✩ ✪ E0: (S → •NP VP, 0) (NP → • Pron, 0) (NP → • Det N, 0) (Pron → • I, 0) E1: (Pron → I •, 0) (NP → Pron •, 0) (S → NP • VP, 0) (VP → • V, 1) (VP → • V P, 1) (VP → • V NP, 1) (V → • think, 1) E2: (V → think •, 1) (VP → V •, 1) (VP → V • S, 1) (VP → V • NP, 1) (S → NP VP •, 0) (S → • NP VP, 2) (NP → • Pron, 2) (NP → • Det N, 2) (Pron → • I, 2)

SLIDE 11

✬ ✫ ✩ ✪

Link between CYK and Earley

(X → α • β, i) ∈ Ej ⇐ ⇒ (X → α • β) ∈ cellj−i, i+1

SLIDE 12

✬ ✫ ✩ ✪

Bottom-up Chart Parsing Idea: keep the best of both CYK and Earley

☞ on-line binarization ”`

a la” Earley (and even better) within a bottom-up CYK-like algorithm Mainly:

☞ α • ...

This is possible when processing bottom-up

This is possible when processing bottom-up (and without lookahead)

SLIDE 13

✬ ✫ ✩ ✪

Bottom-up Chart Parsing: Example N The the cat crocodile ate

N Det S S ... Det ... V ... ... Det ... NP NP VP VP NP NP Det

SLIDE 14

✬ ✫ ✩ ✪

Bottom-up Chart Parsing (3) More formally, a CYK algorithm in which: If cell contents are denoted by [α • ..., i, j] and [X, i, j] respectively Then initialization is wij ⇒ [X, i, j] for X → wij ∈ R and the completion phase becomes: (association of two cells)

[α • ..., i, j] ⊕ [X, k, j + i] ⇒    [α X • ..., i + k, j]

if Y → α Xβ ∈ R

[Y, i + k, j]

if Y → α X ∈ R (”self-filling”)

[X, i, j] ⇒    [X • ..., i, j]

if Y → Xβ ∈ R

[Y, i, j]

if Y → X ∈ R

SLIDE 15

✬ ✫ ✩ ✪

Bottom-up Chart Parsing: Example Initialization:

Completion:

SLIDE 16

✬ ✫ ✩ ✪

N N

...

...

... cat the The crocodile ate

Det Det ... ... S S NP NP NP NP NP VP

SLIDE 17

✬ ✫ ✩ ✪

Dealing with compounds Example on how to deal with compouds during initialization phase: N N credit card N V

SLIDE 18

✬ ✫ ✩ ✪

Complexity still in O(n3) What coefficient for n3? (with respect to grammar parameters)

m(R′) · |NT | · n3

where m(R′) the number of internal nodes of the trie of the right-hand sides of the non-lexical grammar rules

NT : the set of non-terminals R′: the set of non-lexical grammar rules

SLIDE 19

✬ ✫ ✩ ✪

Keypoints

➟ The way algorithms work (Earley items, linking, stepping, prediction, link CYK-Earley) ➟ worst-case complexity O(n3) ➟ Advantages and drawbacks of algorithms

SLIDE 20

✬ ✫ ✩ ✪

References [1] D. Jurafsky & J. H. Martin, Speech and Language Processing, pp. 377-385, Prentice Hall, 2000. [2] R. Dale, H. Moisi, H. Somers, Handbook of Natural Language Processing, pp. 69-73, Dekker, 2000.