Chart Parsing: The Earley Algorithm 2 The Earley Algorithm - - PowerPoint PPT Presentation

Chart Parsing: The Earley Algorithm

Informatics 2A: Lecture 18 Bonnie Webber (revised by Frank Keller)

26 October 2007

Prediction Dotted Rules

Parsing Operations Details of the Algorithm Visualizing the Chart Comparing Earley and CYK Reading: J&M (1st ed), ch. 10 (pp. 377–385) or J&M (2nd ed),

• ch. 13 (pp. 10-25);

NLTK Book, http://nltk.org/doc/en/advanced-parsing.pdf, pp. 8-19

Prediction

As we saw in the last lecture, the CYK algorithm avoids redundant work by storing sub-trees in a chart. We can avoid even more work by adding prediction to the chart. We need a new data structure: A dotted rule stands for a partially constructed constituent, with the dot indicating how much has already been found and how much is still predicted. Dotted rules are generated from ordinary grammar rules. The grammar rule NP → V NP yields the following dotted rules: VP → . V NP incomplete edge VP → V . NP incomplete edge VP → V NP . complete edge

Dotted Rules

With dotted rules, an arc/edge in the chart records: which rule has been used in the analysis; which part of the rule has already been found (left of the dot), and which part is still predicted to be found (right of the dot); the start and end position of the material left of the dot. For example, the input . . . 1 with 2 the 3 telescope 4 . . . could lead to the following dotted rule: NP → Det . N, [2, 3] This means the word from input position 2 to 3 is spanned by a Det, and an N is predicted to come next; if found, it will yield an NP.

Dotted Rules

Example for a chart which contains dotted rules (in graph representation): with the telescope 1 2 4 3 Det −> the . NP −> Det . N NP −> Det N . Prep −> with . N −> telescope . NP −> . Det N

Parsing Operations

The Earley algorithm comprises three main operations: Predictor: an incomplete edge looks for a symbol to the right

• f its dot; if there is no matching symbol in the chart, one is

predicted by adding all matching rules with an initial dot. Scanner: an incomplete edge looks for a POS to the right of its dot; this POS prediction is compared to the input, and a complete edge is added to the chart if it matches. Completer: a complete edge is combined with an incomplete edge that is looking for it to form another complete edge.

Properties of the Algorithm

The Earley algorithm is a bottom-up chart parser with top-down prediction. Top-down prediction: the algorithm builds up parse trees bottom-up, but the incomplete edges (the predictions) are generated top-down, starting with the start symbol of the grammar. Also, edges are added in left-to-right order: if A → X . B, [i, j] is added before C → Y . D, [i′, j′] then j ≤ j′.

Adding Entries to the Chart

The main recursive step is: Enqueue(state, chart entry), where chart entry is a set of states: Enqueue(state, chart entry) 1 if state is not already in chart entry 2 then Push(state, chart entry)

Main Parsing Function

Earley Parse(words, grammar) 1 Enqueue((γ → . S, [0, 0]), chart[0]) 2 for i ← 0 to length(words) 3 do for each state in chart[i] 4 do if Incomplete(state) and 5 Next Cat(state) is not a part of speech 6 then Predictor(state) 7 else if Incomplete(state) and 8 Next Cat(state) is a part of speech 9 then Scanner(state) 10 else Completer(state) 11 return chart

Predictor, Scanner, Completer

Predictor((A → α . B β, [i, j])) 1 for each (B → γ) in Grammar Rules For(B, grammar) 2 do Enqueue((B → . γ, [j, j]), chart[j]) Scanner((A → α . B β, [i, j])) 1 if B ⊂ Parts Of Speech(word[j]) 2 then Enqueue((B → word[j], [j, j + 1]), chart[j + 1]) Completer((B → γ . , [j, k])) 1 for each (A → α . B β, [i, j]) in chart[j] 2 do Enqueue((A → α B . β, [i, k]), chart[k])

Visualizing the Chart

Grammatical rules Lexical rules S → NP VP Det → a | the (determiner) NP → Det Nom N → fish | frogs | soup (noun) NP → Nom Prep → in | for (preposition) Nom → N SRel TV → saw | ate (transitive verb) Nom → N IV → fish | swim (intransitive verb) VP → TV NP Relpro → that (relative pronoun) VP → IV PP VP → IV PP → Prep NP SRel → Relpro VP

Step 0

j = 0 label begin end reason γ → . S earley parse 1 S → . NP VP enter 2 NP → . Det Nom predict from 1 3 NP → . Nom predict from 1 4 Nom → . N SRel predict from 3 5 Nom → . N predict from 3

step 1

j = 1 label begin end reason 6 N → fish . 1 scan 7 IV → fish . 1 scan 8 Nom → N . SRel 1 complete from 4 using 6 9 Nom → N . 1 complete from 5 using 6 10 NP → Nom . 1 complete from 3 using 9 11 SRel → . Relpro VP 1 1 predict from 8 12 S → NP . VP 1 complete from 1 using 10 13 VP → . TV NP 1 1 predict from 12 14 VP → . IV 1 1 predict from 12 15 VP → . IV PP 1 1 predict from 12

Step 2

j = 2 label begin end reason 16 IV → swim . 1 2 scan 17 VP → IV . 1 2 complete from 14 using 16 18 S → NP VP . 2 complete from 12 using 17 19 VP → IV . PP 1 2 complete from 15 using 16 20 PP → . Prep NP 2 2 predict from 19

Step 3

j = 3 label begin end reason 21 Prep → in . 2 3 scan 22 PP → Prep . NP 2 3 complete from 20 using 21 23 NP → . Det Nom 3 3 predict from 22 24 NP → . Nom 3 3 predict from 22 25 Nom → . N SRel 3 3 predict from 24 25 Nom → . N 3 3 predict from 24

Step 4

j = 4 label begin end reason 26 Det → the . 3 4 scan 27 NP → Det . Nom 3 4 complete from 23 using 26 28 Nom → . N SRel 4 4 predict from 27 29 Nom → . N 4 4 predict from 27

Step 5

j = 5 label begin end reason 30 N → soup . 4 5 scan 31 Nom → N . SRel 4 5 complete from 28 using 30 32 SRel → . Relpro NP 5 5 predict from 31 33 Nom → N . 4 5 complete from 29 using 30 34 NP → Det Nom . 3 5 complete from 27 using 33 35 PP → Prep NP . 2 5 complete from 22 using 34 36 VP → IV PP . 1 5 complete from 19 using 35 37 S → NP VP . 5 complete from 12 using 36

Variations on Earley Algorithm Control Structure

The Earley algorithm can be run with other control structures: left-corner: we predict incomplete edges bottom-up instead of top-down (for details on left-corner parsing, see lecture 30); agenda-based, which uses an ordered list of “pending” edges; allows us to prioritize edges that are likely to be correct. There are also probabilistic variants of the Earley algorithm that can deal with Probabilistic CFGs (introduced in lectures 19 and 20).

Comparing Earley and CKY

Both Earley and CYK are bottom-up chart parsing algorithms, but Earley also uses top-down prediction to avoid building edges that will not lead to a valid parse. To illustrate a difference between Earley and CKY, consider an NP with an object relative clause: The milk that we drank Nom → N Relpro NP TV The milk which we drank Nom → N NP TV The milk we drank yesterday Relpro → that | which

Comparing Earley and CKY

An object relative clause can lead, in English, to a form of local ambiguity called a garden path sentence: The horse raced past the barn fell which comes from lexical ambiguity: raced ⊂ TV, IV. Object relative clauses also allow a kind of center embedding that is particularly difficult to understand: The cheese the rat the cat chased ate was moldy We will return to garden paths at the end of this course, when we deal with human sentence processing.

Comparing Earley and CKY

Consider the string:

where ride ⊂ N, TV, IV. What happens when CKY gets to horse? What happens when Earley gets to horse? With Earley, edges are then only added when they can serve to extend a possible parse tree. So the string ride the horse won’t be analyzed as a VP.

Summary

The Earley algorithm adds prediction to chart parsing. This avoids building redundant structure. Dotted rules indicate whether a constituent is complete or incomplete. The algorithm consists of three steps: predictor: an incomplete edge specifies what it needs to be complete; scanner: for each word, the POS is added to chart; completer: a complete edge combines with an incomplete one that’s looking for it. In contrast to CYK, Earley only adds edges to the chart if they can extend a possible parse tree (top-down prediction).