[PPT] - M-monoid parsing and reduct generation Richard Mrbitz 15th December PowerPoint Presentation

SLIDE 1

M-monoid parsing and reduct generation

Richard Mörbitz 15th December 2017

1

SLIDE 2

Parsing?

Given G = (N, Σ, S, R)

N = {S, NP, VP, NNP, . . . }
Σ = {Fruit, bananas, . . . }
R = {S → NP VP, NNP → Fruit, . . . }

To parse e = Fruit fmies like bananas.

2

SLIDE 3

Parse trees

S NP NNP Fruit VP VBZ fmies PP IN like NP NNS bananas S NP NNP Fruit NNP fmies VP VBP like NP NNS bananas

3

SLIDE 4

Abstract syntax trees

S → NP VP NP → NNP NNP → Fruit VP → VBZ PP VBZ → fmies PP → IN NP IN → like NP → NNS NNS → bananas S → NP VP NP → NNP NNP NNP → Fruit NNP → fmies VP → VBP NP VBP → like NP → NNS NNS → bananas

4

SLIDE 5

Recognition

⊤ S → NP VP ⊤ NP → NNP ⊤ NNP → Fruit ⊤ VP → VBZ PP ⊤ VBZ → fmies ⊤ PP → IN NP ⊤ IN → like ⊤ NP → NNS ⊤ NNS → bananas ⊤ S → NP VP ⊤ NP → NNP NNP ⊤ NNP → Fruit ⊤ NNP → fmies ⊤ VP → VBP NP ⊤ VBP → like ⊤ NP → NNS ⊤ NNS → bananas ⊤

5

SLIDE 6

Recognition

⊤ ∨ S → NP VP ∧ ⊤ NP → NNP ∧ ⊤ NNP → Fruit ⊤ VP → VBZ PP ∧ ⊤ VBZ → fmies ⊤ PP → IN NP ∧ ⊤ IN → like ⊤ NP → NNS ∧ ⊤ NNS → bananas ⊤ S → NP VP ∧ ⊤ NP → NNP NNP ∧ ⊤ NNP → Fruit ⊤ NNP → fmies ⊤ VP → VBP NP ∧ ⊤ VBP → like ⊤ NP → NNS ∧ ⊤ NNS → bananas ⊤

5

SLIDE 7

String probability

0.0000495 + S → NP VP × 0.000048 NP → NNP × 0.04 NNP → Fruit 0.2 VP → VBZ PP × 0.0024 VBZ → fmies 0.4 PP → IN NP × 0.012 IN → like 0.4 NP → NNS × 0.06 NNS → bananas 0.3 S → NP VP × 0.0000015 NP → NNP NNP × 0.001 NNP → Fruit 0.2 NNP → fmies 0.1 VP → VBP NP × 0.003 VBP → like 0.5 NP → NNS × 0.06 NNS → bananas 0.3

6

SLIDE 8

Probability of the most likely derivation

0.000048 max S → NP VP × 0.000048 NP → NNP × 0.04 NNP → Fruit 0.2 VP → VBZ PP × 0.0024 VBZ → fmies 0.4 PP → IN NP × 0.012 IN → like 0.4 NP → NNS × 0.06 NNS → bananas 0.3 S → NP VP × 0.0000015 NP → NNP NNP × 0.001 NNP → Fruit 0.2 NNP → fmies 0.1 VP → VBP NP × 0.003 VBP → like 0.5 NP → NNS × 0.06 NNS → bananas 0.3

7

SLIDE 9

Generic computation

parse(e) ⊕ S → NP VP ⊗ NP → NNP ⊗ NNP → Fruit VP → VBZ PP ⊗ VBZ → fmies PP → IN NP ⊗ IN → like NP → NNS ⊗ NNS → bananas S → NP VP ⊗ NP → NNP NNP ⊗ NNP → Fruit NNP → fmies VP → VBP NP ⊗ VBP → like NP → NNS ⊗ NNS → bananas

8

SLIDE 10

Semiring parsing

SLIDE 11

Semiring

Defjnition (Semiring) A semiring (S, ⊕, ⊗, 0, 1) is an algebraic structure, such that

(S, ⊕, 0) is a commutative monoid
(S, ⊗, 1) is a monoid,
⊗ is left-distributive and right-distributive over ⊕, and
a ⊗ 0 = 0 = 0 ⊗ a for every a ∈ S.

S is complete if Σ⊕ exists.

9

SLIDE 12

Semiring parsing

Algorithm for generic complete semiring (S, ⊕, ⊗, 0, 1) [Goo99] Instances: Recognition ({⊤, ⊥}, ∨, ∧, ⊥, ⊤) String probability (R∞

0 , +, ×, 0, 1)

Probability of best derivation (R1

0, max, ×, 0, 1)

Derivation forest (2E1, ∪, ·, ∅, {ε})

1set of derivations (elements of R∗)

10

SLIDE 13

M-monoid parsing

SLIDE 14

Knuth’s generalization of Dijkstra’s algorithm [Knu77]

α β γ

A B C G = (N, Σ, C, R), with

N = {A, B, C}
Σ = {α, β, γ,ˆ

(,ˆ ),ˆ ,}

R = {C → γ(A, B), B →

β(A, A), A → α} Mapping val : Σ∗ → R∞

0 ,

each σ ∈ Σ is a mapping σ : (R∞

0 )k → R∞

Generalization (R∞

0 , ≤) (S, ) [Jun06] 11

SLIDE 15

Multioperator monoid

Defjnition (Multioperator monoid) An M-monoid is an algebraic structure (S, ⊕, 0, Ω), such that

(S, ⊕, 0) is a commutative monoid,
Ω is a set of operations on S, such that

∀ω ∈ Ω : ω(s1, . . . , sk) = 0 if ∃i : si = 0

0k ∈ Ω for all k ∈ N, 0k : Sk → S, such that

0k(s1, . . . , sk) = 0. S is complete if Σ⊕ exists.

12

SLIDE 16

M-monoid parsing problem

Given

1. a complete M-monoid (S,Σ⊕) with (S, ⊕, 0, Ω),
2. a weighted LCFRS (G, wt) over S where

G = (N, Σ, Z, R) is an LCFRS over ∆ and wt : R → Ω, and

3. a sentence e = e1 . . . en with n ≥ 1 and ei ∈ ∆

Compute parse(e) =

Σ⊕

d∈(TR)Z:πΣ(d)=e

h(d), where h : TR → S is the homomorphism from TR to (S, Ω).

13

SLIDE 17

Weighted deductive parsing [Ned03]

Items I = {[A, κ] | A ∈ N, κ range vector over e} Inference rules SCAN:

[A,(i−1,i)]

if ρ = (A → ei) in R RULE:

[B1, κ1] ... [Bk, κk] [A,σ( κ1,..., κk)]

if ρ = (A → σ(B1, . . . , Bk)) in R Goal: [Z, (0, |e|)]

14

SLIDE 18

M-monoid parsing algorithm

Input

1. an M-monoid (S, ⊕, 0, Ω),
2. an LCFRS− G = (N, Σ, Z, R) over ∆, and wt : R → Ω,
3. a function select : 2I → I specifjc to the M-monoid, and
4. a sentence e = e1 . . . en with n ≥ 1 and ei ∈ ∆

Variables V : I → S mapping Output parse(e)

15

SLIDE 19

Algorithm 2.1 M-monoid parsing for LCFRS−

1: A, C ← ∅ 2: for each A ∈ N and

κ range vector over e do

3:

V([A, κ]) ← 0

4: for each ρ = (A → σ) in R and [A,

κ] generated by SCAN [A,

κ] do

5:

V([A, κ]) ← V([A, κ]) ⊕ wt(ρ)()

6:

A ← A ∪ {[A, κ]}

7: while A = ∅ do 8:

[A, κ] ← select(A)

9:

A ← A \ {[A, κ]}

10:

C ← C ∪ {[A, κ]}

11:

for each ρ = (B → σ(B1, . . . Bk)) in R and [B, η] deduced by RULE

∗ [B, η] from [A,

κ] and other items from C do

12:

V([B, η]) ← V([B, η]) ⊕ wt(ρ)(V([B1, κ1]), . . . , V([Bk, κk]))

13:

if [B, η] ∈ C then

14:

A ← A ∪ {[B, η]}

15: return V([Z, (0, n)])

16

SLIDE 20

Reduct generation with M-monoid parsing

SLIDE 21

Reduct of a grammar and a sentence

Given

an LCFRS G = (N, Σ, Z, R) over ∆ (RTG-notation)
a sentence e ∈ ∆∗

Compute LCFRS G ⊲ψ e = (N′, Σ, Z′, R′), such that

1. L(G ⊲ψ e)LCFRS = L(G)LCFRS ∩ {e}, and
2. with the mapping ψ : N′ → N there exists a bijective

mapping from the ASTs of G ⊲ψ e to the ASTs of G

17

SLIDE 22

Preliminary defjnitions

Prototype rules: PR = {[A, κ] → σ([B1, κ1], . . . [Bk, κk]) | (A → σ(B1, . . . , Bk)) ∈ R ∧

κ,

κ1, . . . , κk are range vectors over e} Prototype nonterminals: PN = {[A, κ] | A ∈ N ∧ κ is a range vector over e}

18

SLIDE 23

The reduct problem as an instance of M-monoid parsing

Given:

1. (G,Σ∪), where
G = (2PN × 2PR, ∪, ∅ × ∅, ΩREDUCT) is the reduct M-monoid
Σ∪

i∈I si = i∈I si is used as the infjnitary sum operation

2. (G, wt) for an arbitrary G over ∆ and wt : R → ΩREDUCT
3. an arbitrary e ∈ ∆∗

Compute: parse(e) = ({A ∈ N′ | A is of the form [Z, κ]}, R′) Then G ⊲ψ e = (N′, Σ, Z′, R′), where

R′ = v where (u, v) = parse(e),
N′ = {[A,

κ] | [A, κ] is the left-hand side of a rule in R′},

Z′ = [Z, (0, |e|)]

19

SLIDE 24

The operations of ΩREDUCT

if ρ is of the form A → w, then

ωρ() = {[A, (i − 1, i)] | ei = w}, {[A, (i − 1, i)] → w | ei = w}.

if ρ is of the form A → σ(B1, . . . , Bk), then

ωρ((U1, V1), . . . , (Uk, Vk)) = U, V, where U = {[A, σ( η1, . . . , ηk)] | ( η1, . . . , ηk) ∈ fitσ(U1, . . . , Uk)} V =

1≤i≤k

Vi ∪ {[A, σ( η1, . . . , ηk)] → σ([B1, η1], . . . , [Bk, ηk]) | ( η1, . . . , ηk) ∈ fitσ(U1, . . . , Uk)}

20

SLIDE 25

Example: reduct generation

0Fruit1 1fmies2 2like3 3bananas4 [NNP, (0, 1)] [NNP, (1, 2)] [VBZ, (1, 2)] [IN, (2, 3)] [VBP, (2, 3)] [NNS, (3, 4)] Fruit fmies fmies like like bananas 21

0Fruit1 1fmies2 2like3 3bananas4 [NNP, (0, 1)] [NNP, (1, 2)] [VBZ, (1, 2)] [IN, (2, 3)] [VBP, (2, 3)] [NNS, (3, 4)] [NP, (0, 1)] [NP, (0, 2)] [NP, (3, 4)] Fruit fmies fmies like like bananas x1

1

x1

1x2 1

x1

1

21

SLIDE 32

Example: reduct generation

0Fruit1 1fmies2 2like3 3bananas4 [NNP, (0, 1)] [NNP, (1, 2)] [VBZ, (1, 2)] [IN, (2, 3)] [VBP, (2, 3)] [NNS, (3, 4)] [NP, (0, 1)] [NP, (0, 2)] [NP, (3, 4)] Fruit fmies fmies like like bananas x1

1

x1

1x2 1

x1

1

21

SLIDE 33

Example: reduct generation

0Fruit1 1fmies2 2like3 3bananas4 [NNP, (0, 1)] [NNP, (1, 2)] [VBZ, (1, 2)] [IN, (2, 3)] [VBP, (2, 3)] [NNS, (3, 4)] [NP, (0, 1)] [NP, (0, 2)] [NP, (3, 4)] Fruit fmies fmies like like bananas x1

1

x1

1x2 1

x1

1

21

SLIDE 34

Example: reduct generation

0Fruit1 1fmies2 2like3 3bananas4 [NNP, (0, 1)] [NNP, (1, 2)] [VBZ, (1, 2)] [IN, (2, 3)] [VBP, (2, 3)] [NNS, (3, 4)] [NP, (0, 1)] [NP, (0, 2)] [NP, (3, 4)] [PP, (2, 4)] [VP, (2, 4)] Fruit fmies fmies like like bananas x1

1

x1

1x2 1

x1

1

x1

1x2 1

x1

1x2 1

21

SLIDE 35

Example: reduct generation

0Fruit1 1fmies2 2like3 3bananas4 [NNP, (0, 1)] [NNP, (1, 2)] [VBZ, (1, 2)] [IN, (2, 3)] [VBP, (2, 3)] [NNS, (3, 4)] [NP, (0, 1)] [NP, (0, 2)] [NP, (3, 4)] [PP, (2, 4)] [VP, (2, 4)] [VP, (1, 4)] Fruit fmies fmies like like bananas x1

1

x1

1x2 1

x1

1

x1

1x2 1

x1

1x2 1

x1

1x2 1

21

SLIDE 36

Example: reduct generation

0Fruit1 1fmies2 2like3 3bananas4 [NNP, (0, 1)] [NNP, (1, 2)] [VBZ, (1, 2)] [IN, (2, 3)] [VBP, (2, 3)] [NNS, (3, 4)] [NP, (0, 1)] [NP, (0, 2)] [NP, (3, 4)] [PP, (2, 4)] [VP, (2, 4)] [VP, (1, 4)] [S, (0, 4)] Fruit fmies fmies like like bananas x1

1

x1

1x2 1

x1

1

x1

1x2 1

x1

1x2 1

x1

1x2 1

x1

1x2 1

21

SLIDE 37

Example: reduct generation

0Fruit1 1fmies2 2like3 3bananas4 [NNP, (0, 1)] [NNP, (1, 2)] [VBZ, (1, 2)] [IN, (2, 3)] [VBP, (2, 3)] [NNS, (3, 4)] [NP, (0, 1)] [NP, (0, 2)] [NP, (3, 4)] [PP, (2, 4)] [VP, (2, 4)] [VP, (1, 4)] [S, (0, 4)] Fruit fmies fmies like like bananas x1

1

x1

1x2 1

x1

1

x1

1x2 1

x1

1x2 1

x1

1x2 1

x1

1x2 1

x1

1x2 1

21

SLIDE 38

Example: reduct generation

0Fruit1 1fmies2 2like3 3bananas4 [NNP, (0, 1)] [NNP, (1, 2)] [VBZ, (1, 2)] [IN, (2, 3)] [VBP, (2, 3)] [NNS, (3, 4)] [NP, (0, 1)] [NP, (0, 2)] [NP, (3, 4)] [PP, (2, 4)] [VP, (2, 4)] [VP, (1, 4)] [S, (0, 4)] Fruit fmies fmies like like bananas x1

1

x1

1x2 1

x1

1

x1

1x2 1

x1

1x2 1

x1

1x2 1

x1

1x2 1

x1

1x2 1

21

SLIDE 39

Properties of the algorithm

SLIDE 40

Termination and correctness

Termination

the algorithm always terminates

Correctness

not correct in general
correct for not cyclic LCFRS
correct for inferior M-monoids
correct for the reduct generation

22

SLIDE 41

Termination and correctness

Termination

the algorithm always terminates

Correctness

not correct in general
correct for not cyclic LCFRS
correct for inferior M-monoids
correct for the reduct generation

22

SLIDE 42

Termination and correctness

Termination

the algorithm always terminates

Correctness

not correct in general
correct for not cyclic LCFRS
correct for inferior M-monoids
correct for the reduct generation

22

SLIDE 43

Termination

Lemma The M-monoid parsing algorithm always terminates. Proof.

termination of while loop if A = ∅
set of all possible elements of A is fjnite
every element is added to A at most once
only a fjnite number of elements is added to A
in each iteration, one element is removed from A

23

SLIDE 44

Correctness for inferior M-monoids i

Defjnition (Inferior operation) Let (S, ) be a totally ordered set and ω : Sk → S (k ∈ N) be an operation. We call ω -inferior if for every s1, . . . , sk, s ∈ S and for every i ∈ {1, . . . , k} the following properties hold:

1. if s si then

ω(s1, . . . , si−1, s, si+1, . . . , sk) ω(s1, . . . , si−1, si, si+1, . . . , sk)

2. ω(s1, . . . , sk) min{s1, . . . , sk}

24

SLIDE 45

Correctness for inferior M-monoids ii

Defjnition (Inferior M-monoid) Let (S, ⊕, 0, Ω) be an M-monoid. Moreover, let ⊕ be the binary relation on S defjned for every a, b ∈ S as follows: a ⊕ b if a ⊕ b = b. If ⊕ is a total order such that 0 ⊕ a for every a ∈ S and every ω ∈ Ω is ⊕-inferior, then we call the M-monoid S inferior. Lemma Let S be the class of inferior M-monoids. The algorithm is correct for S and the select function arg max.

25

SLIDE 46

Conclusion

Dijkstra’s algorithm Algebraic path problem Semiring parsing Knuth’s algorithm Weighted deductive parsing Generalization Multioperator monoid parsing

SLIDE 47

Infjnitary sum operation

Defjnition (Infjnitary sum operation)

Σ⊕ is a family (Σ⊕I | I countable) of mappings Σ⊕I : SI → S.

SI is represented as a family (si | i ∈ I) with si ∈ S
notion: Σ⊕i∈I si

SLIDE 48

The mapping fit

fitσ : (PN)k → 2(

κk), where for each σ ∈ Σ and arity k ∈ N:

fitσ(U1, . . . , Uk) = {( η1, . . . , ηk) | [B1, η1] ∈ U1, . . . , [Bk, ηk] ∈ Uk, B1, . . . , Bk ∈ N ∧ σ( η1, . . . , ηk) is a range vector over e}.

SLIDE 49

Cyclic and weakly cyclic LCFRS−

We call an LCFRS− G cyclic for e ∈ ∆∗ if there are A ∈ N, range vectors κ over e, and sentential forms α, β, α′, β′ such that Z(ε) ⇒∗ αA( κ(e))β ⇒+ α′A( κ(e))β′ ⇒∗ ε We call an LCFRS− G weakly cyclic for e ∈ ∆∗ if there are A ∈ N, range vectors κ over e, and sentential forms α, β such that A( κ(e)) ⇒+ αA( κ(e))β ⇒∗ ε We call an LCFRS− cyclic (weakly-cyclic) if there is an e ∈ ∆∗ such that G is cyclic for e (respectively, weakly cyclic for e).

SLIDE 50

Correctness for not cyclic LCFRS−

Lemma Let G = (N, Σ, Z, R) be a LCFRS− not cyclic for e ∈ ∆∗. Then, for every M-monoid S, wt : R → S, and select : 2I → I it holds that the algorithm is correct for S, (G, wt), select, and e. In particular, for every LCFRS− G and every sentence e for which G is not weakly cyclic, after termination, for each [A, κ] ∈ C, it holds that V([A, κ]) =

Σ⊕

d∈(TR)A:πΣ(d)= κ(e)

h(d) , Corollary The algorithm is correct for the class of all not cyclic LCFRS−.

SLIDE 51

Joshua Goodman. Semiring parsing. Computational Linguistics, 25(4):573–605, 1999. Jean Christoph Jung. Knuth’s generalization of Dijkstra’s algorithm. 2006. D.E. Knuth. A Generalization of Dijkstra’s Algorithm.

Inform. Process. Lett., 6(1):1–5, February 1977.