Dependency Parsing Joakim Nivre Uppsala University Department of - - PowerPoint PPT Presentation

Dependency Parsing

Joakim Nivre

Uppsala University Department of Linguistics and Philology joakim.nivre@lingfil.uu.se

Dependency Parsing 1(14)

• 1. Dependency Trees
• 2. Arc-Factored Models
• 3. Online Learning
• 4. Eisner’s Algorithm
• 5. Spanning Tree Parsing

Dependency Parsing 2(14)

Dependency Trees ◮ Input sentence x = x1, . . . , xn ◮ Dependency graph G = (Vx, A)

◮ Vx = {0, . . . , n} is a set of nodes, one for each xi + root ◮ A ⊆ (Vx × L × Vx) is a set of labeled arcs (i, l, j)

◮ Dependency tree = dependency graph satisfying:

• 1. Root: No arcs into node 0.
• 2. Single-Head: At most one incoming arc to any node.
• 3. Connected: Graph is weakly connected.

Dependency Parsing 3(14)

Dependency Trees

Projectivity: For every arc (i, l, j), there is a directed path from i to every word k such that min(i, j) < k < max(i, j).

Dependency Parsing 4(14)

Dependency Trees

Language Trees Arcs Arabic 11.2% 0.4% Basque 26.2% 2.9% Czech 23.2% 1.9% Danish 15.6% 1.0% Greek 20.3% 1.1% Russian 10.6% 0.9% Slovene 22.2% 1.9% Turkish 11.6% 1.5%

Dependency Parsing 5(14)

Dependency Trees

Parsing problem:

◮ Input: x = x1, . . . , xn ◮ Output: dependency tree y for x

Equivalent to:

◮ Assign a head i and a label l to every node j (1 ≤ j ≤ n)

under the tree constraint

◮ Find a directed spanning tree in the complete graph

Gx = (Vx, Vx × L × Vx)

Dependency Parsing 6(14)

Arc-Factored Models

Score(x, y) =

• (i,l,j)∈Ay

Score(i, l, j, x) GEN(x) = {y | y is a spanning tree in Gx = (Vx, Vx × L × Vx)} EVAL(x, y) = Score(x, y) =

• (i,l,j)∈Ay

Score(i, l, j, x)

Dependency Parsing 7(14)

Arc-Factored Models

Score(i, l, j, x) =

K

• k=1

fk(i, l, j, x) · wk y∗ = argmax

y∈GEN(x)

• (i,l,j)∈Ay

K

• k=1

fk(i, l, j, x) · wk

Dependency Parsing 8(14)

Arc-Factored Models

Unigram Bigram In-Between PoS xi-w, xi-p xi-w, xi-p, xj-w, xj-p xi-p, b-p, xj-p xi-w xi-p, xj-w, xj-p Surrounding PoS xi-p xi-w, xj-w, xj-p xi-p, xi+1-p, xj−1-p, xj-p xj-w, xj-p xi-w, xi-p, xj-p xi−1-p, xi-p, xj−1-p, xj-p xj-w xi-w, xi-p, xj-w xi-p, xi+1-p, xj-p, xj+1-p xj-p xi-w, xj-w xi−1-p, xi-p, xj-p, xj+1-p xi-p, xj-p

Dependency Parsing 9(14)

Online Learning

Training data: T = {(xi, yi)}|T |

i=1

1 w ← 0 2 for n : 1..N 3 for i : 1..|T | 4 y∗ ← Parse(xi, w) 5 if y∗ = yi 6 w ← Update(w, y∗, yi) 7 return w

Dependency Parsing 10(14)

Online Learning

Parse(x, w) 1 return argmaxy∈GEN(xi)

• (i,l,j)∈Ay

K

k=1 fk(i, l, j, xi) · wk

Update(w, y∗, yi) 1 for k : 1..K 2 for (i, l, j) ∈ Ay∗ 3 wk ← wk − fk(i, l, j, x) 4 for (i, l, j) ∈ Ayi 5 wk ← wk + fk(i, l, j, x)

Dependency Parsing 11(14)

Eisner’s Algorithm

CKY Eisner

Dependency Parsing 12(14)

Eisner’s Algorithm

1 for i : 0..n and all d, c 2 C[i][i][d][c] ← 0.0 3 for m : 1..n 4 for i : 0..n−m 5 j ← i +m 6 C[i][j][←][0] ← maxi≤k<j C[i][k][→][1] + C[k+1][j][←][1] + Score(j, i) 7 C[i][j][→][0] ← maxi≤k<j C[i][k][→][1] + C[k+1][j][←][1] + Score(i, j) 8 C[i][j][←][1] ← maxi≤k<j C[i][k][←][1] + C[k][j][←][0] 9 C[i][j][→][1] ← maxi<k≤j C[i][k][→][0] + C[k][j][→][1] 10 return C[0][n][→][1]

Dependency Parsing 13(14)

Spanning Tree Parsing

John saw Mary ROOT 9 10 20 9 30 11 3 30 John saw Mary ROOT 10 30 30 Dependency Parsing 14(14)