Parsing beyond context-free grammar: 1. N and T are disjoint - - PowerPoint PPT Presentation

Kallmeyer/Maier ESSLLI 2008

Parsing beyond context-free grammar: Introduction

Laura Kallmeyer, Wolfgang Maier University of T¨ ubingen ESSLLI Course 2008

Parsing beyond CFG 1 Introduction Kallmeyer/Maier ESSLLI 2008

Overview

• 1. CFG and natural languages
• 2. Parsing: Some preliminary notions
• 3. Polynomial extensions of CFG
• 4. Parsing schemata

Parsing beyond CFG 2 Introduction Kallmeyer/Maier ESSLLI 2008

CFG and natural languages (1) Definition of a CFG: A context-free grammar (CFG) is a tuple G = N, T, P, S such that

• 1. N and T are disjoint alphabets, the nonterminals and

terminals of G,

• 2. P ⊂ N × (N ∪ T)∗ is a finite set of productions (also called

rewriting rules). A production A, α is usually written A → α.

• 3. S ∈ N is the start symbol.

Parsing beyond CFG 3 Introduction Kallmeyer/Maier ESSLLI 2008

CFG and natural languages (2) Sample CFG Gtelescope:

• Nonterminals: {S, NP, V P, PP, N, V, P, D}
• Terminals: {the, man, telescope, saw, girl, with, John}
• Productions:

S → NP V P NP → D N V P → V P PP | V NP N → N PP PP → P NP N → man | girl | telescope D → the N → John P → with V → saw

Parsing beyond CFG 4 Introduction

Kallmeyer/Maier ESSLLI 2008

CFG and natural languages (3) Let G = N, T, P, S be a CFG. The (string) language L(G) of G is the set {w ∈ T ∗ | S

∗

⇒ w} where

• for w, w′ ∈ (N ∪ T)∗: w ⇒ w′ iff there is a A → α ∈ P and

there are v, u ∈ (N ∪ T)∗ such that w = vAu and w′ = vαu.

• ∗

⇒ is the reflexive transitive closure of ⇒: – w ⇒ w for all w ∈ (N ∪ T)∗, and – for all w, w′ ∈ (N ∪ T)∗: w

n

⇒ w′ iff there is a v such that w ⇒ v and v

n−1

⇒ w′. – for all w, w′ ∈ (N ∪ T)∗: w

∗

⇒ w′ iff there is a i ∈ I N such that w

i

⇒ w′. A language is called context-free iff it is generated by a CFG.

Parsing beyond CFG 5 Introduction Kallmeyer/Maier ESSLLI 2008

CFG and natural languages (4) Context-free languages (CFLs)

• can be recognized in polynomial time (O(n3));
• are accepted by push-down automata;
• have nice closure properties (e.g., closure under

homomorphisms, intersection with regular languages . . . );

• satisfy a pumping lemma;
• can describe nested dependencies ({wwR | w ∈ T ∗}).

Parsing beyond CFG 6 Introduction Kallmeyer/Maier ESSLLI 2008

CFG and natural languages (5) Question: Is CFG powerful enough to describe all natural language phenomea? Answer: No. There are constructions in natural languages that cannot be adequately described with a context-free grammar. Example: cross-serial dependencies in Dutch and in Swiss German. Dutch: (1) ... dat Wim Jan Marie de kinderen zag helpen leren zwemmen ... that Wim Jan Marie the children saw help teach swim ‘... that Wim saw Jan help Marie teach the children to swim’

Parsing beyond CFG 7 Introduction Kallmeyer/Maier ESSLLI 2008

CFG and natural languages (6) Swiss German:

(2) ... das mer em Hans es huus h¨ alfed aastriiche ... that we HansDat houseAcc helped paint ‘... that we helped Hans paint the house’ (3) ... das mer d’chind em Hans es huus l¨

• nd h¨

alfe aastriiche ... that we the childrenAcc HansDat houseAcc let help paint ‘... that we let the children help Hans paint the house’

Swiss German uses case marking and displays cross-serial dependencies. Shieber (1985) shows that Swiss German is not context-free.

Parsing beyond CFG 8 Introduction

Kallmeyer/Maier ESSLLI 2008

CFG and natural languages (7) In general, because of the closure properties, the following holds: A formalism that can generate cross-serial dependencies can also generate the copy language {ww | w ∈ {a, b}∗}. The copy language is not context-free. Therefore we are interested in extensions of CFG in order to describe all natural language phenomena.

Parsing beyond CFG 9 Introduction Kallmeyer/Maier ESSLLI 2008

Parsing: Some preliminary notions (1) Let G be a grammar, L the string language of G and w ∈ T ∗. Recognition: decide whether w ∈ L. Example: Take the CFG Gtelescope. Input: “the man saw the girl”. Output: yes. Input: “the man saw saw the girl”. Output: no.

Parsing beyond CFG 10 Introduction Kallmeyer/Maier ESSLLI 2008

Parsing: Some preliminary notions (2) Parsing: decide whether w ∈ L and if so, then give its parse tree. Example: Input: “the man saw the girl”. Output:

S NP VP D N V NP the man saw D N the girl

Input: “the man saw saw the girl”. Output: no.

Parsing beyond CFG 11 Introduction Kallmeyer/Maier ESSLLI 2008

Parsing: Some preliminary notions (3) The time and space complexity of a parsing algorithm can be determined depending on the length n of the input string and (sometimes) the size of the grammar. Complexity classes: P (PTIME): problems that can be solved deterministically in an amount of time that is polynomial in the size of the input. I.e., there is a constant c and a k such that the parsing of a string of length n takes an amount of time ≤ cnk. Notation: O(nk).

Parsing beyond CFG 12 Introduction

Kallmeyer/Maier ESSLLI 2008

Parsing: Some preliminary notions (4) NP: problems whose positive solutions can be verified in polynomial time given the right information, or equivalently, whose solutions can be non-deterministically found in polynomial time. NP-complete: the hardest problems in NP. A problem is NP-complete if any problem in NP can be transformed into it in polynomial time. Open question: P = NP? In this course we are interested in extensions of CFG that are in P.

Parsing beyond CFG 13 Introduction Kallmeyer/Maier ESSLLI 2008

Polynomial extensions of CFG (1) Tree Adjoining Grammars (TAG), Joshi & Schabes (1997):

• Tree-rewriting grammar.
• Extension of CFG that allows to replace not only leaves but

also internal nodes with new trees.

• Can generate the copy language.

Example: TAG for the copy language S ǫ SNA a S S∗

NA

a SNA b S S∗

NA

b

Parsing beyond CFG 14 Introduction Kallmeyer/Maier ESSLLI 2008

Polynomial extensions of CFG (2) Example: TAG derivation of abab:

S ǫ SNA a S S∗

NA

a ❀ SNA a S S∗

NA

a ǫ SNA a S S∗

NA

a ǫ SNA b S S∗

NA

b ❀ SNA a SNA b S S∗

NA

b S∗

NA

a ǫ Parsing beyond CFG 15 Introduction Kallmeyer/Maier ESSLLI 2008

Polynomial extensions of CFG (3) Linear Indexed Grammars (LIG), Gazdar (1988), Vijayashanker (1987):

• Context-free productions where the nonterminal symbols are

equipped with stacks containing indices.

• Three types of productions:

– A[. . .] → X1 . . .Xi[. . .] . . .Xn with Xj ∈ N ∪ T for j = i, Xi ∈ N. – A[. . .] → B[f . . .] – A[f . . .] → X1 . . .Xi[. . .] . . .Xn with Xj ∈ N ∪ T for j = i, Xi ∈ N. LIGs are weakly equivalent to TAG.

Parsing beyond CFG 16 Introduction

Kallmeyer/Maier ESSLLI 2008

Polynomial extensions of CFG (4) Example: LIG for the copy language S0 → S[#] S[..] → aSa[..] Sa[..] → S[a..] S[..] → bSb[..] Sb[..] → S[b..] S → T T[a..] → T[..]a T[b..] → T[..]b T[#] → ǫ

Parsing beyond CFG 17 Introduction Kallmeyer/Maier ESSLLI 2008

Polynomial extensions of CFG (5) Linear Context-free rewriting systems (LCFRS), Weir (1988): Grammars that have an underlying context-free structure. An LCFRS consists of

• a (generalized) context-free grammar that generates a set of

terms,

• a yield function that specifies the structures corresponding to

these terms, and

• a function specifying the strings yielded by these structures.

LCFRS is more powerful than TAG and LIG.

Parsing beyond CFG 18 Introduction Kallmeyer/Maier ESSLLI 2008

Polynomial extensions of CFG (6) Multiple Context-free Grammars (MCFG), Seki et al. (1991): 5-tuple N, T, F, P, S such that

• N, T are non-terminals and terminals where each non-terminal

A has a dimension, dim(A). From a non-terminal of dimension k, k-tuples of terminal strings are derived. The dimension of the start symbol S is 1.

• F is a finite set of functions and P is a set of productions

A0 → f[A1, . . ., Ak] with f ∈ F. The idea is that f describes how to compute the yield of A0 (a dim(A0)-tuple of terminal strings) from the yields of A1, . . ., Ak. f must be linear in the sense that each of its arguments is used at most once to compute the new string tuple.

Parsing beyond CFG 19 Introduction Kallmeyer/Maier ESSLLI 2008

Polynomial extensions of CFG (7) MCFG example: S → f[A], A → g[A], A → h[ ]. h[ ] = (ab, cd), g[(x1, x2)] = (ax1b, cx2d), f[(x1, x2)] = (x1x2). Language: {anbncndn | n ≥ 1}. LCFRS and MCFG are weakly equivalent.

Parsing beyond CFG 20 Introduction

Kallmeyer/Maier ESSLLI 2008

Polynomial extensions of CFG (8) Range Concatenation Grammar (RCG), Boullier (2000):

• RCG contains clauses of the form A(. . .) → A1(. . .) . . .Ak(. . .)

where A, A1, . . . , Ak are predicates. Their arguments are words

• ver the terminal and nonterminal alphabets.
• Intuition: The predicates characterize properties of strings. A

derivation starts with S(w) where S is a start predicate. If this can be reduced to the empty word (i.e., property S is true for w), then w is in the language. MCFGs can be written as RCGs with the functions f built into the

• clauses. Example:

S(XY ) → A(X, Y ), A(aXb, cY d) → A(X, Y ), A(ab, cd) → ǫ.

Parsing beyond CFG 21 Introduction Kallmeyer/Maier ESSLLI 2008

Polynomial extensions of CFG (9) RCGs are simple if

• the arguments in the right-hand sides of the clauses are single

variables.

• no variable appears more than once in the left-hand side of a

clause or more than once in the right-hand side of a clause.

• each variable occurring in the left-hand side of a clause occurs

also in its right-hand side and vice versa. Simple RCG are weakly equivalent to LCFRS and MCFG. RCG in general are more powerful; they generate exactly the class PTIME of polynomially parsable languages.

Parsing beyond CFG 22 Introduction Kallmeyer/Maier ESSLLI 2008

Polynomial extensions of CFG (10) Summary: ✬ ✫ ✩ ✪ ✬ ✫ ✩ ✪ ✬ ✫ ✩ ✪ ☛ ✡ ✟ ✠ CFG TAG, LIG LCFRS, MCFG, simple RCG RCG (= PTIME) mildly context-sensitive (Joshi, 1985)

Parsing beyond CFG 23 Introduction Kallmeyer/Maier ESSLLI 2008

Parsing schemata (1)

• Algorithmic descriptions of parsing algorithms (e.g., in

pseudo-code) introduce data structures and control structures.

• The parsing strategy of the algorithm does not depend on

them. Question: Can we separate the parsing strategy from the control strategy? Answer: Parsing schemata (Shieber et al., 1995, Sikkel, 1997).

Parsing beyond CFG 24 Introduction

Kallmeyer/Maier ESSLLI 2008

Parsing schemata (2) Advantages:

• Concentration on parsing strategy;
• Facilitation of proofs (e.g., soundness and completeness of an

algorithm). Soundness: if algo yields true for w, then w ∈ L(G). Completeness: if w ∈ L(G), then algo yields true for w.

• Complexity of an algorithm sometimes easier to determine.

Parsing beyond CFG 25 Introduction Kallmeyer/Maier ESSLLI 2008

Parsing schemata (3) How characterize a single parsing step? During parsing, the parser produces trees (parse trees, partial results) and tries to combine them to new trees, until some tree rooted by the goal category (e.g. S) comes out.

• We can characterize parse trees
• We can characterize how new parse trees can be deduced from

existing ones

• We can fix a goal: We want to deduce a tree with root S that

spans the entire input sentence

Parsing beyond CFG 26 Introduction Kallmeyer/Maier ESSLLI 2008

Parsing schemata (4) We characterize a parse tree rooted by some nonterminal X by the terminals X spans. Example: S VP NP NP V Det N I saw a man We write parse trees/partial parse results in the form of items: [X, i, j], meaning that X derives the terminals between i and j. Here: [S, 0, 4]

Parsing beyond CFG 27 Introduction Kallmeyer/Maier ESSLLI 2008

Parsing schemata (5) Sometimes, a category and its yield have been predicted but not yet recognized. To mark this, we can use dotted items or items with dotted productions:

• 1. •S signifies that S has been predicted;
• 2. S• signifies that S has been recognized;
• 3. A → A1 . . .Ai • Ai+1 . . . An signifies that the rhs of the

production A → A1 . . . An has been recognized up to Ai while the part from Ai+1 to An has been predicted.

Parsing beyond CFG 28 Introduction

Kallmeyer/Maier ESSLLI 2008

Parsing schemata (6) Parsing schemata understand parsing as a deductive process. Deduction of new items from existing ones can be described using inference rules. General form: antecedent consequent side conditions antecedent, consequent: lists of items Application: if antecedent can be deduced and side condition holds, then the consequent can be deduced as well.

Parsing beyond CFG 29 Introduction Kallmeyer/Maier ESSLLI 2008

Parsing schemata (7) A parsing schema consists of

• deduction rules;
• an axiom (or axioms), can be written as a deduction rule with

empty antecedent;

• and a goal item.

The parsing algorithm succeeds if, for a given input, it is possible to deduce the goal item.

Parsing beyond CFG 30 Introduction Kallmeyer/Maier ESSLLI 2008

Parsing schemata (8) Example: Parsing schema for CYK (CFG-Parsing): Items have three elements:

• X ∈ N ∪ T: the nonterminal/terminal that spans a substring

wi, . . . , wj of w;

• the index i of the first terminal in the subsequence;
• the length l = j − i of the subsequence.

Item form: [X, i, l] with X ∈ N ∪ T, i ∈ [1..n + 1], l ∈ [0..n].

Parsing beyond CFG 31 Introduction Kallmeyer/Maier ESSLLI 2008

Parsing schemata (9) Goal item: [S, 1, n]. Deduction rules: Scan: [a, i, 1] wi = a ǫ-productions: [A, i, 0] A → ǫ ∈ P, i ∈ [1..n + 1] Complete: [A1, i1, l1], . . ., [Ak, ik, lk] [A, i1, l]

A → A1 . . . Ak ∈ P, l = l1 + · · · + lk, ij = i1 + l1 . . . + lj−1

Parsing beyond CFG 32 Introduction

Kallmeyer/Maier ESSLLI 2008

Parsing schemata (10)

• Algorithms specified in terms of parsing schemata can be

implemented directly

• Shieber et al. (1995): Parsers starting from parsing schemata

in PROLOG

• Other (multi-purpose) systems that allow creation of parsers

starting from parsing schemata: – DyaLog (Clergerie, 1993) – Dyna (Eisner, Goldlust & Smith, 2005) – etc.

• The system presented in G´
• mez-Rodr´

ıguez, Vilares & Alonso (2006) is specially designed and optimized for the compilation

• f parsing schemata to working parsers

Parsing beyond CFG 33 Introduction Kallmeyer/Maier ESSLLI 2008

Parsing schemata (11) Why use parsing schemata for implementation?

• During parsing, we deduce new trees from already existing

trees (partial results), present as items

• The same item can be used in different deductions (and has to

be calculated only once) = ⇒ Dynamic Programming (Computation sharing, tabulation) is made easy

Parsing beyond CFG 34 Introduction