[PPT] - Parsing beyond context-free grammar: adjunction: replacing an PowerPoint Presentation

SLIDE 1

Kallmeyer/Maier ESSLLI 2008

Parsing beyond context-free grammar: Tree Adjoining Grammar Parsing

Laura Kallmeyer, Wolfgang Maier University of T¨ ubingen ESSLLI Course 2008

Parsing beyond CFG 1 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Overview

1. Tree Adjoining Grammars
2. An Earley parser for TAG

(a) Introduction (b) Items (c) Inference Rules

3. LR Parsing

(a) Introduction (b) Construction of the automaton (c) The recognizer

Parsing beyond CFG 2 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Tree Adjoining Grammars (1) A Tree Adjoining Grammars (TAG) (Joshi & Schabes 1997) is a tree-rewriting system, i.e., a set of elementary trees with two

perations:
adjunction: replacing an internal node with a new tree.

The new tree is an auxiliary tree and has a special leaf, the foot node.

substitution: replacing a leaf with a new tree.

The new tree is an initial tree Notation: γ[p, γ′] is the tree one obtains from replacing the node at position p in γ with the tree γ′ (by substitution or adjunction).

Parsing beyond CFG 3 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Tree Adjoining Grammars (2) (1) John sometimes laughs

NP John S NP VP VP ADV VP∗ V sometimes laughs derived tree laugh[1, john][2, sometimes]: S NP VP John ADV VP sometimes V laughs Parsing beyond CFG 4 TAG Parsing

SLIDE 2

Kallmeyer/Maier ESSLLI 2008

Tree Adjoining Grammars (3) A Tree Adjoining Grammar (TAG) is a quadruple G = N, T, I, A such that

T and N are disjoint alphabets of terminals and nonterminals,
I is a finite set of initial trees, and
A is a finite set of auxiliary trees.

The trees in I ∪ A are called elementary trees. G is lexicalized iff each elementary tree has at least one leaf with a terminal label. TAG allows to specify for each node

1. whether adjunction is mandatory and
2. which trees can be adjoined.

Parsing beyond CFG 5 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Tree Adjoining Grammars (4) A derivation starts with an initial tree. In a final derived tree, all leaves must have terminal labels: Let G = I, A, N, T be a TAG. Let γ and γ′ be finite trees.

γ ⇒ γ′ in G iff there is a node position p and an instance γ′

0 of a

tree (possibly derived from some) γ0 ∈ I ∪ A such that γ′ = γ[p, γ0].

∗

⇒ is the reflexive transitive closure of ⇒.

The tree language of G is LT(G) := {γ | there is an α ∈ I such

that α

∗

⇒ γ, all leaves in γ have terminal labels and there are no OA nodes in γ}.

Parsing beyond CFG 6 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Tree Adjoining Grammars (5) Languages TAG can generate:

{ww | w ∈ {a, b}∗}
L4 := {anbncndn | n ≥ 0}

Languages TAG cannot generate:

{wn | w ∈ {a, b}∗} for any n > 2.

⇒ TAG generate only a limited amount of cross-serial dependencies

Lk := {an

1an 2an 3 . . . an k | n ≥ 0} for any k > 4.

⇒ TAG can “count up to 4, not further”.

L := {a2n | n ≥ 0}.

⇒ TAG cannot generate languages whose word lengths grow exponentially.

Parsing beyond CFG 7 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Tree Adjoining Grammars (6) TAGs are mildly context-sensitive:

TAGs are slightly more powerful than CFG, they can describe

a limited amount of cross-serial dependencies.

TAGs are polynomially parsable (complexity O(n6)).
TALs are of constant growth.

Parsing beyond CFG 8 TAG Parsing

SLIDE 3

Kallmeyer/Maier ESSLLI 2008

Earley Parsing: Introduction (1)

Left-to-right CKY parser (Vijay-Shanker & Joshi, 1985) very

slow: O(n6) worst case and best case (just as in CFG version

f CKY, to many partial trees not pertinent to the final tree

are produced)

Behaviour is due to pure bottom-up approach, no predictive

information whatsoever is used

Goal: Earley-style parser! First in Schabes & Joshi (1988).

Here, we present the algorithm from Joshi & Schabes (1997). We assume a TAG without substitution nodes.

Parsing beyond CFG 9 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Earley Parsing: Introduction (2)

Earley Parsing: Left-to-right scanning of the string (using

predictions to restrict hypothesis space)

Traversal of elementary trees, current position marked with a

dot. The dot can have exactly four positions with respect to the node: left above (la), left below (lb), right above (ra), right below (rb).

Parsing beyond CFG 10 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Earley Parsing: Introduction (3) General idea: Whenever we are

left above a node, we can predict an adjunction and start the

traversal of the adjoined tree;

left of a foot node, we can move back to the adjunction site and

traverse the tree below it;

right of an adjunction site, we continue the traversal of the

adjoined tree at the right of its foot node;

right above the root of an auxiliary tree, we can move back to

the right of the adjunction site.

Parsing beyond CFG 11 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Earley Parsing: Items (1) What kind of information do we need in an item characterizing a partial parsing result? [α, dot, pos, i, j, k, l, sat?] where

α ∈ I ∪ A is a (dotted) tree, dot and pos the address and

location of the dot

i, j, k, l are indices on the input string, where i, l ∈ {0, . . ., n},

j, k ∈ {0, . . ., n} ∪ {−}, n = |w|, − means unbound value

sat? is a flag. It controls (prevents) multiple adjunctions at a

single node (sat? = 1 means that something has already been adjoined to the dotted node)

Parsing beyond CFG 12 TAG Parsing

SLIDE 4

Kallmeyer/Maier ESSLLI 2008

Earley Parsing: Items (2) What do the items mean?

[α, dot, la, i, j, k, l, nil]: In α part left of the dot ranges from i to
l. If α is an auxiliary tree, part below foot node ranges from j

to k.

[α, dot, lb, i, −, −, i, nil]: In α part below dotted node starts at

position i.

[α, dot, rb, i, j, k, l, sat?]: In α part below dotted node ranges

from i to l. If α is an auxiliary tree, part below foot node ranges from j to k. If sat? = nil, nothing was adjoined to dotted node, sat? = 1 means that adjunction took place.

[α, dot, ra, i, j, k, l, nil]: In α part left and below dotted node

ranges from i to l. If α is an auxiliary tree, part below foot node ranges from j to k.

Parsing beyond CFG 13 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Earley Parsing: Items (3) Some notational conventions:

We use Gorn addresses for the nodes: 0 is the address of the

root, i (1 ≤ i) is the address of the ith daughter of the root, and for p = 0, p · i is the address of the ith daughter of the node at address p.

For a tree α and a Gorn address dot, α(dot) denotes the node

at address dot in α (if defined).

For a node n, Adj(n) is the set of trees adjoinable at n.

nil ∈ Adj(n) signifies that adjunction is not obligatory. Adj(n) = ∅ if n has a terminal or ǫ as label.

Parsing beyond CFG 14 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Earley Parsing: Inference Rules (1) ScanTerm [α, dot, la, i, j, k, l, nil] [α, dot, ra, i, j, k, l + 1, nil] α(dot) labelled wl+1

wl+1

wi+1 . . . wl

Scan-ǫ [α, dot, la, i, j, k, l, nil] [α, dot, ra, i, j, k, l, nil] α(dot) labelled ǫ

Parsing beyond CFG 15 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Earley Parsing: Inference Rules (2) PredictAdjoinable [α, dot, la, i, j, k, l, nil] [β, 0, la, l, −, −, l, nil] β ∈ Adj(α(dot))

A

wi+1 . . . wl

A

⇒ A∗ PredictNoAdj [α, dot, la, i, j, k, l, nil] [α, dot, lb, l, −, −, l, nil] nil ∈ Adj(α(dot))

Parsing beyond CFG 16 TAG Parsing

SLIDE 5

Kallmeyer/Maier ESSLLI 2008

Earley Parsing: Inference Rules (3) PredictAdjoined [β, dot, lb, l, −, −, l, nil] [α, dot′, lb, l, −, −, l, nil] dot = foot(β), β ∈ Adj(α(dot′))

A

A ⇒ A∗

Parsing beyond CFG

17 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Earley Parsing: Inference Rules (4) Complete I [α, dot, rb, i, j, k, l, 1], [β, dot′, lb, i, −, −, i, nil] [β, dot′, rb, i, i, l, l, nil]

dot′=foot(β), β∈Adj(α(dot))

A

⇒

wi+1 . . . wl

A A∗

A

A∗

Parsing beyond CFG

18 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Earley Parsing: Inference Rules (5) Complete II [α, dot, rb, i, j, k, l, sat?], [α, dot, la, h, −, −, i, nil] [α, dot, ra, h, j, k, l, nil] β(dot) ∈ N

r

[α, dot, rb, i, −, −, l, sat?], [α, dot, la, h, j, k, i, nil] [α, dot, ra, h, j, k, l, nil] β(dot) ∈ N

A

⇒

wi+1 . . . wl

A

wh+1 . . . wi

A

wh+1 . . . wl

Parsing beyond CFG 19 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Earley Parsing: Inference Rules (6) Adjoin [β, 0, ra, i, j, k, l, nil], [α, dot, rb, j, p, q, k, nil] [α, dot, rb, i, p, q, l, 1] β ∈ Adj(α(dot))

A

A∗

wi+1 . . . wj wk+1 . . . wl

A

⇒

wj+1 . . . wk

Aadj

wi+1 . . . wl

sat? = 1 prevents the new item from being reused in another Adjoin application. Parsing beyond CFG 20 TAG Parsing

SLIDE 6

Kallmeyer/Maier ESSLLI 2008

Earley Parsing: Inference Rules (7) Move the dot to daughter/sister/mother: MoveDown: [α, p, lb, i, j, k, l, nil] [α, p · 1, la, i, j, k, l, nil] α(p · 1) is defined MoveRight: [α, p, ra, i, j, k, l, nil] [α, p + 1, la, i, j, k, l, nil] α(p + 1) is defined MoveUp: [α, p · m, ra, i, j, k, l, nil] [α, p, rb, i, j, k, l, nil] α(p · m + 1) is not defined

Parsing beyond CFG 21 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Earley Parsing: Inference Rules (8) Initialize: [α, 0, la, 0, −, −, 0, nil] α ∈ I Goal item: [α, 0, ra, 0, −, −, n, nil], α ∈ I

Parsing beyond CFG 22 TAG Parsing Kallmeyer/Maier ESSLLI 2008

Earley Parsing: Summary

We have seen an Earley-type recognition algorithm for TAG.

We can turn our recognizer into a parser by storing each item with a set of pairs of other items from which it can be inferred.

The parser has Complete, Scan and Predict operations plus an

Adjunction operation.

The algorithm has an upper time bound of O(n6)
The parser does not have the Valid Prefix Property. Ensuring

this property for TAG parsing is costly.

Parsing beyond CFG 23 TAG Parsing Kallmeyer/Maier ESSLLI 2008

LR parsing: Introduction (1)

LR parsing: Left-to-right scanning and Right-to-left reduction
We compile a finite-state automaton from the grammar

(offline) and use it to guide actions during parsing (online)

What does the automaton represent?

– States: Correspond to sets of items closed under prediction – Edges: Correspond to scanning a terminal symbol or consuming an already recognized nonterminal Roughly, LR parsing is Earley parsing with precompiled predictions.

Parsing beyond CFG 24 TAG Parsing

SLIDE 7

Kallmeyer/Maier ESSLLI 2008

LR parsing: Introduction (2) An LR automaton is typically represented by two tables.

The Action table lists what action must be performed (shift or

reduce). This action depends on – the current state in the automaton – the next preterminal to be read

The Goto table lists the states where the automaton has to go

after reducing a production

Parsing beyond CFG 25 TAG Parsing Kallmeyer/Maier ESSLLI 2008

LR parsing: Introduction (3) In CFG LR parsing, we dispose of two operations on a stack:

1. shift(k): Scans a terminal, pushes the corresponding

pre-terminal on the stack and switches to state k

2. reduce(A): The RHS of some production A → A1 . . . An has

been recognized, i.e. is on the stack. reduce(A) pops A1, . . ., An from the stack and pushes the LHS A on the stack, then switches to the next state (provided by the goto table)

Parsing beyond CFG 26 TAG Parsing Kallmeyer/Maier ESSLLI 2008

LR parsing: Introduction (4)

Nederhof (1998) extends traditional LR parsing to TAG
His algorithm is based on

– a LR parse automaton (automaton) – a function to scan the next symbol of the input: shift(∆, aw) – two functions to reduce partial results on the stack: reduceSubtree(∆, w) and reduceAuxtree(∆, w) where w is the input and ∆ is the LR stack.

Nederhof (1998) mentions an implementation of the parser

generator

LR automaton generation for the XTAG grammar seemed to

be feasible

Parsing beyond CFG 27 TAG Parsing Kallmeyer/Maier ESSLLI 2008

LR parsing: Introduction (5) Notations:

N (t) is the set of nodes of a tree t.
children(N) is the list of the children of a node N, given in

linear precedence order.

Parsing beyond CFG 28 TAG Parsing

SLIDE 8

Kallmeyer/Maier ESSLLI 2008

LR parsing: Introduction (6)

The elementary trees are extended with artifical new nodes:

– For each t ∈ I ∪ A, we add a unique node ⊤ immediately dominating Rt (the root of t). – For each t ∈ A, we add a unique node ⊥ immediately dominated by Ft (the foot of t).

For a t ∈ I ∪ A, (t, N) denotes the subtree of t rooted in N.

T = I ∪ A ∪ {(t, N)|t ∈ I ∪ A, N ∈ N (t)} is the set of all subtrees of elementary trees, including the elementary trees themselves. Assume that our TAG has no substitution nodes and does not contain empty words.

Parsing beyond CFG 29 TAG Parsing Kallmeyer/Maier ESSLLI 2008

LR parsing: Construction of the automaton (1)

The states of the LR automaton are sets of items
Transitions are labeled with terminals and nonterminals

An item represents a subtree of height 1 (mother node N and its daughters) in one of the τ ∈ T together with a dot • that specifies up to which daughter the subtree has been recognized. This subtree is notated as a dotted production N → α • β.

Parsing beyond CFG 30 TAG Parsing Kallmeyer/Maier ESSLLI 2008

LR parsing: Construction of the automaton (2) Items have the form [τ, N → α • β], where

τ ∈ T,
N ∈ N (τ), and
αβ are the daughters of N.

An item is called completed if is has the form

either [t, ⊤ → Rt•] with t ∈ I ∪ A,
or [(t, N), N → α•].

Parsing beyond CFG 31 TAG Parsing Kallmeyer/Maier ESSLLI 2008

LR parsing: Construction of the automaton (3)

The construction of the set of states of the automaton starts

with an initial LR state qin = {[t, ⊤ → •Rt]|t ∈ I}

From each state, new states can be computed using functions

goto and goto⊥.

To compute these functions for a given state q, one needs the

closure closure(q) of this state. Intuition: the closure contains all items that can be obtained from an item [τ, . . .] in q by moving down or up in τ or predicting an adjunction or predicting the part below a foot node.

Parsing beyond CFG 32 TAG Parsing

SLIDE 9

Kallmeyer/Maier ESSLLI 2008

LR parsing: Construction of the automaton (4) Definition of the closure of a state q: Let q be a set of items. closure(q) is then defined by the following inference rules:

x

x ∈ q

[τ, N → α • Mβ]

[τ, M → •γ] nil ∈ Adj(M), children(M) = γ

[τ, N → α • Mβ]

[t, ⊤ → •Rt] t ∈ Adj(M)

[τ, Ft → •⊥]

[(t′, N), N → •γ] t ∈ Adj(N).N ∈ N (t′), children(N) = γ

Parsing beyond CFG 33 TAG Parsing Kallmeyer/Maier ESSLLI 2008

LR parsing: Construction of the automaton (5) Intuition behind the goto-functions: goto shifts the dot over a node, goto⊥ shifts the dot over a ⊥ (i.e., a foot node daughter). Definition of goto and goto⊥: Let q be a set of items, M a terminal leaf or a node with Adj(M) ∩ A = ∅ (no NA constraint).

goto(q, M) = {[τ, N → αM • β]|[τ, N → α • Mβ] ∈ closure(q)}
goto⊥(q, M) = {[τ, Ft → ⊥•]|[τ, Ft → •⊥] ∈ closure(q) ∧ t ∈

Adj(M)}

Parsing beyond CFG 34 TAG Parsing Kallmeyer/Maier ESSLLI 2008

LR parsing: Construction of the automaton (6) Now we can define the set Q of LR states of our automaton as follows:

qin
q

q′ q′ = goto(q, M) = ∅ for some node M

q

q′ q′ = goto⊥(q, M) = ∅ for some node M A state is final (in Qfin) if its closure contains a completed item for some initial tree: Qfin = {q ∈ Q|closure(q) ∩ {[t, ⊤ → Rt•]|t ∈ I} = ∅}

Parsing beyond CFG 35 TAG Parsing Kallmeyer/Maier ESSLLI 2008

LR parsing: Construction of the automaton (7) For the definition of the recognizer, we also need the notion of reductions(q) for a given state q. Intuition: If the closure of q contains a completed item, then the LHS node of the dotted production or, if this is a ⊤ in an auxiliary tree, the whole tree are part of the reductions. Definition of reductions(q) for a given state q: reductions(q) = {t ∈ A|[t, ⊤ → Rt•] ∈ closure(q)} ∪ {N ∈ N |[(t, N), N → α•] ∈ closure(q)}

Parsing beyond CFG 36 TAG Parsing

SLIDE 10

Kallmeyer/Maier ESSLLI 2008

LR parsing: Construction of the automaton (8) We also need the definition of cross-sections through a tree rooted at some node N. Intuition: the sequences on the stack that can be reduced, i.e., that correspond roughly to the RHS of some completed dotted production are cross-sections. A cross-section of a node N is either the node N or a sequence of cross-sections of the daughters of N in linear precedence order. Furthermore, nodes dominating foot nodes are paired with a stack

f nodes (indicating where subsequent adjunctions took place).

Parsing beyond CFG 37 TAG Parsing Kallmeyer/Maier ESSLLI 2008

LR parsing: Construction of the automaton (9) Definition of cross-sections CS(N) of a node N: Define M := N ∪ (N × N ∗) Then for a given node N:

N ∈ CS(N) if N does not dominate a foot node,
(N, L) ∈ CS(N) for each L ∈ N ∗ if N dominates a foot node,
x1 . . .xm ∈ CS(N) if children(N) = M1 . . . Mm and

xi ∈ CS(Mi) for 1 ≤ i ≤ m. Furthermore, CS+(N) := CS(N) \ ({N} ∪ {(N, L) | L ∈ N ∗}) (the cross-sections without the node itself).

Parsing beyond CFG 38 TAG Parsing Kallmeyer/Maier ESSLLI 2008

LR parsing: The recognizer (1)

The stack ∆ contains states and symbols. The latter are either

terminal nodes or nonterminal nodes equipped with a stack.

A configuration (∆, w) consists of a stack and a word (the

remaining part of the input string).

There are three operations that allow the automaton to make a

transition (i.e., to change configuration): shift, reduce subtree and reduce aux tree.

Parsing beyond CFG 39 TAG Parsing Kallmeyer/Maier ESSLLI 2008

LR parsing: The recognizer (2) shift pushes the next input symbol followed by a new state on the stack:

a

❀ a• (∆q, aw) ⊢ (∆qaq′, w) if q′ = goto(q, a) = ∅.

Parsing beyond CFG 40 TAG Parsing

SLIDE 11

Kallmeyer/Maier ESSLLI 2008

LR parsing: The recognizer (3) Reduce subtree is applied when having completed a subtree rooted in N such that an adjunction occurs at N. In other words, it recognizes the part below a foot node. N[. . .]• X1 . . . Xm ❀ ⊥[N . . .]• (∆q0X1q1 . . .Xmqm, w) ⊢ (∆q0(⊥, [NL])q′, w) if

N ∈ reductions(qm), X1 . . . Xm ∈ CS+(N), q′ = goto⊥(q0, N) =

∅, and

L is defined as follows: if some Xj is of the form (M, L), then

this provides L, otherwise L = [ ].

Parsing beyond CFG 41 TAG Parsing Kallmeyer/Maier ESSLLI 2008

LR parsing: The recognizer (4) Reduce aux tree is applied once an auxiliary tree has been

recognized. We then go back to the node where the adjunction
ccurred.

Rt• X1 . . . Xj[N . . .] . . . Xm ❀ N[. . .]• (∆q0X1q1 . . .Xmqm, w) ⊢ (∆q0Xq′, w) if

there is a t ∈ reductions(qm) with X1 . . . Xm ∈ CS+(Rt),
q′ = goto(q0, N) = ∅ where N is obtained from the unique Xj of

the form M[NL], and

if L = [ ], then X = N, otherwise X = N[L].

Parsing beyond CFG 42 TAG Parsing Kallmeyer/Maier ESSLLI 2008

LR parsing: The recognizer (5)

The stack is initialized with the initial state qin.
The stack always contains an alternation of states q ∈ Q and

nodes or nodes with stacks X ∈ M.

A parse is successful if, in a sequence of transitions (i.e.,

applications of shift, reduce subtree and reduce aux tree), the input is completely consumed and the automaton reaches a final state: Some input v is recognized if (qin, v) ⊢∗ (qin∆q, ǫ) such that q ∈ Qfin.