Transition-Based Parsing Joakim Nivre Uppsala University - - PowerPoint PPT Presentation

▶
transition based parsing
SMART_READER_LITE
LIVE PREVIEW

Transition-Based Parsing Joakim Nivre Uppsala University - - PowerPoint PPT Presentation

Dec 30, 2022 471 likes •1.03k views

Transition-Based Parsing Joakim Nivre Uppsala University Department of Linguistics and Philology joakim.nivre@lingfil.uu.se Transition-Based Parsing 1(19) 1. Transition Systems 2. Greedy Classifier-Based Parsing 3. Beam Search and

slide-1
SLIDE 1

Transition-Based Parsing

Joakim Nivre

Uppsala University Department of Linguistics and Philology joakim.nivre@lingfil.uu.se

Transition-Based Parsing 1(19)

slide-2
SLIDE 2
  • 1. Transition Systems
  • 2. Greedy Classifier-Based Parsing
  • 3. Beam Search and Structured Learning
  • 4. Non-Projective Parsing

Transition-Based Parsing 2(19)

slide-3
SLIDE 3

Transition Systems

S = (C, T, cs, Ct)

  • 1. C is a set of configurations
  • 2. T is a set of transitions, each t : C → C
  • 3. cs is an initialization function, cs(x) ∈ C
  • 4. Ct ⊆ C is a set of terminal configurations

Transition-Based Parsing 3(19)

slide-4
SLIDE 4

Transition Systems

c = (Σ, B, A)

  • 1. Σ is a list of nodes in Vx, known as the stack
  • 2. B is a list of nodes in Vx, known as the buffer
  • 3. A is a set of dependency arcs in Vx × L × Vx

Transition-Based Parsing 4(19)

slide-5
SLIDE 5

Transition Systems

C0,m = (c0, c1, . . . , cm)

  • 1. c0 = cs(x),
  • 2. cm ∈ Ct,
  • 3. for every i (1 ≤ i ≤ m), ci = t(ci−1) for some t ∈ T

Gcm = (Vx, Acm)

Transition-Based Parsing 5(19)

slide-6
SLIDE 6

Transition Systems

Initialization: cs(x = x1, . . . , xn) = ([0], [1, . . . , n], ∅) Terminal: Ct = {c ∈ C|c = ([0], [ ], A)} Transitions: (σ, [i|β], A) ⇒ ([σ|i], β, A) (Shift) ([σ|i|j], B, A) ⇒ ([σ|j], B, A∪{(j, l, i)})1 (Left-Arcl) ([σ|i|j], B, A) ⇒ ([σ|i], B, A∪{(i, l, j)}) (Right-Arcl)

1 Permitted only if i = 0.

Transition-Based Parsing 6(19)

slide-7
SLIDE 7

Transition Systems

[ROOT]Σ [Economic, news, had, little, effect, on, financial, markets, .]B

Transition-Based Parsing 7(19)

slide-8
SLIDE 8

Transition Systems

[ROOT, Economic]Σ [news, had, little, effect, on, financial, markets, .]B

Transition-Based Parsing 7(19)

slide-9
SLIDE 9

Transition Systems

[ROOT, Economic, news]Σ [had, little, effect, on, financial, markets, .]B

Transition-Based Parsing 7(19)

slide-10
SLIDE 10

Transition Systems

[ROOT, news]Σ [had, little, effect, on, financial, markets, .]B

Transition-Based Parsing 7(19)

slide-11
SLIDE 11

Transition Systems

[ROOT, news, had]Σ [little, effect, on, financial, markets, .]B

Transition-Based Parsing 7(19)

slide-12
SLIDE 12

Transition Systems

[ROOT, had]Σ [little, effect, on, financial, markets, .]B

Transition-Based Parsing 7(19)

slide-13
SLIDE 13

Transition Systems

[ROOT, had, little]Σ [effect, on, financial, markets, .]B

Transition-Based Parsing 7(19)

slide-14
SLIDE 14

Transition Systems

[ROOT, had, little, effect]Σ [on, financial, markets, .]B

Transition-Based Parsing 7(19)

slide-15
SLIDE 15

Transition Systems

[ROOT, had, effect]Σ [on, financial, markets, .]B

Transition-Based Parsing 7(19)

slide-16
SLIDE 16

Transition Systems

[ROOT, had, effect, on]Σ [financial, markets, .]B

Transition-Based Parsing 7(19)

slide-17
SLIDE 17

Transition Systems

[ROOT, had, effect, on, financial]Σ [markets, .]B

Transition-Based Parsing 7(19)

slide-18
SLIDE 18

Transition Systems

[ROOT, had, effect, on, financial, markets]Σ [.]B

Transition-Based Parsing 7(19)

slide-19
SLIDE 19

Transition Systems

[ROOT, had, effect, on, markets]Σ [.]B

Transition-Based Parsing 7(19)

slide-20
SLIDE 20

Transition Systems

[ROOT, had, effect, on]Σ [.]B

Transition-Based Parsing 7(19)

slide-21
SLIDE 21

Transition Systems

[ROOT, had, effect]Σ [.]B

Transition-Based Parsing 7(19)

slide-22
SLIDE 22

Transition Systems

[ROOT, had]Σ [.]B

Transition-Based Parsing 7(19)

slide-23
SLIDE 23

Transition Systems

[ROOT, had, .]Σ [ ]B

Transition-Based Parsing 7(19)

slide-24
SLIDE 24

Transition Systems

[ROOT, had]Σ [ ]B

Transition-Based Parsing 7(19)

slide-25
SLIDE 25

Transition Systems

[ROOT]Σ [ ]B

Transition-Based Parsing 7(19)

slide-26
SLIDE 26

Transition Systems

Initialization: cs(x = x1, . . . , xn) = ([0], [1, . . . , n], ∅) Terminal: Ct = {c ∈ C|c = (Σ, [ ], A)} Transitions: (σ, [i|β], A) ⇒ ([σ|i], β, A) (Shift) ([σ|i], [j|β], A) ⇒ (σ, [j|β], A∪{(j, l, i)})1 (Left-Arcl) ([σ|i], [j|β], A) ⇒ ([σ|i|j], β, A∪{(i, l, j)}) (Right-Arcl) ([σ|i], B, A) ⇒ (σ, B, A)2 (Reduce)

1 Permitted only if i = 0 and there are no k, l′ such that (k, l′, i) ∈ A. 2 Permitted only if there are k, l′ such that (k, l′, i) ∈ A.

Transition-Based Parsing 8(19)

slide-27
SLIDE 27

Transition Systems

[ROOT]Σ [Economic, news, had, little, effect, on, financial, markets, .]B

Transition-Based Parsing 9(19)

slide-28
SLIDE 28

Transition Systems

[ROOT, Economic]Σ [news, had, little, effect, on, financial, markets, .]B

Transition-Based Parsing 9(19)

slide-29
SLIDE 29

Transition Systems

[ROOT]Σ [news, had, little, effect, on, financial, markets, .]B

Transition-Based Parsing 9(19)

slide-30
SLIDE 30

Transition Systems

[ROOT, news]Σ [had, little, effect, on, financial, markets, .]B

Transition-Based Parsing 9(19)

slide-31
SLIDE 31

Transition Systems

[ROOT]Σ [had, little, effect, on, financial, markets, .]B

Transition-Based Parsing 9(19)

slide-32
SLIDE 32

Transition Systems

[ROOT, had]Σ [little, effect, on, financial, markets, .]B

Transition-Based Parsing 9(19)

slide-33
SLIDE 33

Transition Systems

[ROOT, had, little]Σ [effect, on, financial, markets, .]B

Transition-Based Parsing 9(19)

slide-34
SLIDE 34

Transition Systems

[ROOT, had]Σ [effect, on, financial, markets, .]B

Transition-Based Parsing 9(19)

slide-35
SLIDE 35

Transition Systems

[ROOT, had, effect]Σ [on, financial, markets, .]B

Transition-Based Parsing 9(19)

slide-36
SLIDE 36

Transition Systems

[ROOT, had, effect, on]Σ [financial, markets, .]B

Transition-Based Parsing 9(19)

slide-37
SLIDE 37

Transition Systems

[ROOT, had, effect, on, financial]Σ [markets, .]B

Transition-Based Parsing 9(19)

slide-38
SLIDE 38

Transition Systems

[ROOT, had, effect, on]Σ [markets, .]B

Transition-Based Parsing 9(19)

slide-39
SLIDE 39

Transition Systems

[ROOT, had, effect, on, markets]Σ [.]B

Transition-Based Parsing 9(19)

slide-40
SLIDE 40

Transition Systems

[ROOT, had, effect, on]Σ [.]B

Transition-Based Parsing 9(19)

slide-41
SLIDE 41

Transition Systems

[ROOT, had, effect]Σ [.]B

Transition-Based Parsing 9(19)

slide-42
SLIDE 42

Transition Systems

[ROOT, had]Σ [.]B

Transition-Based Parsing 9(19)

slide-43
SLIDE 43

Transition Systems

[ROOT, had, .]Σ [ ]B

Transition-Based Parsing 9(19)

slide-44
SLIDE 44

Greedy Classifier-Based Parsing

Parse(x = (w0, w1, . . . , wn)) 1 c ← cs(x) 2 while c ∈ Ct 3 t∗ ← argmaxt Score(c, t) 4 c ← t∗(c) 5 return Gc

Transition-Based Parsing 10(19)

slide-45
SLIDE 45

Greedy Classifier-Based Parsing

Score(c, t) =

K

  • k=1

fk(c, t) · wk

Transition-Based Parsing 11(19)

slide-46
SLIDE 46

Greedy Classifier-Based Parsing

Unigrams Bigrams Σ0.pos Σ0.pos, B0.pos Σ1.pos Σ0.pos, Σ0.lab B0.pos B0.pos, ldep(B0).lab B1.pos B2.pos B3.pos Trigrams Σ0.lab Σ1.pos, Σ0.pos, B0.pos ldep(Σ0).lab Σ0.pos, B0.pos, B1.pos rdep(Σ0).lab B0.pos, B1.pos, B2.pos ldep(B0).lab B1.pos, B2.pos, B3.pos Σ0.word Σ0.pos, ldep(Σ0).lab, rdep(Σ0).lab B0.word B1.word hd(Σ0).word

Transition-Based Parsing 12(19)

slide-47
SLIDE 47

Beam Search and Structured Learning

Score(c0,m, x) =

m−1

  • i=0

Score(ci, ti) [where ti(ci) = ci+1]

Transition-Based Parsing 13(19)

slide-48
SLIDE 48

Beam Search and Structured Learning

Parse(x = (w0, w1, . . . , wn)) 1 Beam ← {cs(x), 0.0} 2 while ∃c, s ∈ Beam : c ∈ Ct 3 NewBeam ← ∅ 4 for every (c, s) ∈ Beam 5 for every t ∈ T 6 NewBeam ← NewBeam ∪ {t(c), s+Score(c, t)} 7 Beam → QBest(NewBeam) 8 return ← 1Best(Beam)

Transition-Based Parsing 14(19)

slide-49
SLIDE 49

Beam Search and Structured Learning

Training data: T = {(xi, ci

0,m)}|T | i=1

1 w ← 0 2 for n : 1..N 3 for i : 1..|T | 4 c∗

0,m ← Parse(xi, w)

5 if c∗

0,m = ci 0,m

6 w ← Update(w, c∗

0,m, ci 0,m)

7 return w

Transition-Based Parsing 15(19)

slide-50
SLIDE 50

Beam Search and Structured Learning

Update(w, c∗

0,m, ci 0,m)

1 for k : 1..K 2 for i : 0..m − 1 3 wk ← wk − fk(ci, ti) 4 for i : 0..m − 1 5 wk ← wk + fk(ci, ti)

Transition-Based Parsing 16(19)

slide-51
SLIDE 51

Non-Projective Parsing

Initialization: cs(x = x1, . . . , xn) = ([0], [1, . . . , n], ∅) Terminal: Ct = {c ∈ C|c = ([0], [ ], A)} Transitions: (σ, [i|β], A) ⇒ ([σ|i], β, A) (Shift) ([σ|i|j], B, A) ⇒ ([σ|j], B, A∪{(j, l, i)})1 (Left-Arcl) ([σ|i|j], B, A) ⇒ ([σ|i], B, A∪{(i, l, j)}) (Right-Arcl) ([σ|i|k|j], B, A) ⇒ ([σ|k|j], B, A∪{(j, l, i)})1 (Left-Arc2l) ([σ|i|k|j], B, A) ⇒ ([σ|i|k], B, A∪{(i, l, j)}) (Right-Arc2l) ([σ|i|k1|k2|j], B, A) ⇒ ([σ|k1|k2|j], B, A∪{(j, l, i)})1 (Left-Arc3l) ([σ|i|k1|k2|j], B, A) ⇒ ([σ|i|k1|k2], B, A∪{(i, l, j)}) (Right-Arc3l)

1 Permitted only if i = 0. Transition-Based Parsing 17(19)

slide-52
SLIDE 52

Non-Projective Parsing

Initialization: cs(x = x1, . . . , xn) = ([0], [1, . . . , n], ∅) Terminal: Ct = {c ∈ C|c = ([0], [ ], A)} Transitions: (σ, [i|β], A) ⇒ ([σ|i], β, A) (Shift) ([σ|i|j], B, A) ⇒ ([σ|j], B, A∪{(j, l, i)})1 (Left-Arcl) ([σ|i|j], B, A) ⇒ ([σ|i], B, A∪{(i, l, j)}) (Right-Arcl) ([σ|i|j], β, A) ⇒ ([σ|j], [i|β], A)2 (Swap)

1 Permitted only if i = 0. 2 Permitted only if i = 0 and i < j.

Transition-Based Parsing 18(19)

slide-53
SLIDE 53

Non-Projective Parsing

Transition-Based Parsing 19(19)