[PPT] - Structured Perceptron with Inexact Search x x the man bit PowerPoint Presentation

SLIDE 1

Liang Huang Suphan Fayong Yang Guo Information Sciences Institute University of Southern California NAACL 2012 Montréal June 2012

Structured Perceptron with Inexact Search

the man bit the dog DT NN VBD DT NN

x y x y=-1 y=+1 x

the man bit the dog

x

那人咬了狗

y

SLIDE 2

Structured Perceptron (Collins 02)

2

x y=-1 y=+1 x y

update weights if y ≠ z

w

x z

exact inference

binary classification

SLIDE 3

Structured Perceptron (Collins 02)

2

the man bit the dog DT NN VBD DT NN

x y x y=-1 y=+1 x y

update weights if y ≠ z

w

x z

exact inference

binary classification structured classification

SLIDE 4

Structured Perceptron (Collins 02)

2

the man bit the dog DT NN VBD DT NN

x y y

update weights if y ≠ z

w

x z

exact inference

x y=-1 y=+1 x y

update weights if y ≠ z

w

x z

exact inference

binary classification structured classification

SLIDE 5

Structured Perceptron (Collins 02)

challenge: search efficiency (exponentially many classes)
often use dynamic programming (DP)
but still too slow for repeated use, e.g. parsing is O(n3)
and can’t use non-local features in DP

2

the man bit the dog DT NN VBD DT NN

x y y

update weights if y ≠ z

w

x z

exact inference

x y=-1 y=+1 x y

update weights if y ≠ z

w

x z

exact inference

trivial hard

constant # of classes exponential # of classes

binary classification structured classification

SLIDE 6

Perceptron w/ Inexact Inference

3

the man bit the dog DT NN VBD DT NN

x y x z

inexact inference

y

update weights if y ≠ z

w

beam search greedy search

SLIDE 7

Perceptron w/ Inexact Inference

routine use of inexact inference in NLP (e.g. beam search)

3

the man bit the dog DT NN VBD DT NN

x y x z

inexact inference

y

update weights if y ≠ z

w

beam search greedy search

SLIDE 8

Perceptron w/ Inexact Inference

routine use of inexact inference in NLP (e.g. beam search)
how does structured perceptron work with inexact search?

3

the man bit the dog DT NN VBD DT NN

x y x z

inexact inference

y

update weights if y ≠ z

w

beam search greedy search

SLIDE 9

Perceptron w/ Inexact Inference

routine use of inexact inference in NLP (e.g. beam search)
how does structured perceptron work with inexact search?
so far most structured learning theory assume exact search

3

the man bit the dog DT NN VBD DT NN

x y x z

inexact inference

y

update weights if y ≠ z

w

beam search greedy search

SLIDE 10

Perceptron w/ Inexact Inference

routine use of inexact inference in NLP (e.g. beam search)
how does structured perceptron work with inexact search?
so far most structured learning theory assume exact search
would search errors break these learning properties?

3

the man bit the dog DT NN VBD DT NN

x y x z

inexact inference

y

update weights if y ≠ z

w

beam search greedy search

SLIDE 11

Perceptron w/ Inexact Inference

routine use of inexact inference in NLP (e.g. beam search)
how does structured perceptron work with inexact search?
so far most structured learning theory assume exact search
would search errors break these learning properties?
if so how to modify learning to accommodate inexact search?

3

the man bit the dog DT NN VBD DT NN

x y x z

inexact inference

y

update weights if y ≠ z

w

does it still work???

beam search greedy search

SLIDE 12

Prior work: Early update (Collins/Roark)

4

x z

greedy

r beam

y

early update on prefixes y’, z’

w

SLIDE 13

Prior work: Early update (Collins/Roark)

a partial answer: “early update” (Collins & Roark, 2004)
a heuristic for perceptron with greedy or beam search
updates on prefixes rather than full sequences
works much better than standard update in practice, but...

4

x z

greedy

r beam

y

early update on prefixes y’, z’

w

SLIDE 14

Prior work: Early update (Collins/Roark)

a partial answer: “early update” (Collins & Roark, 2004)
a heuristic for perceptron with greedy or beam search
updates on prefixes rather than full sequences
works much better than standard update in practice, but...
two major problems for early update
there is no theoretical justification -- why does it work?
it learns too slowly (due to partial examples); e.g. 40 epochs

4

x z

greedy

r beam

y

early update on prefixes y’, z’

w

SLIDE 15

Prior work: Early update (Collins/Roark)

a partial answer: “early update” (Collins & Roark, 2004)
a heuristic for perceptron with greedy or beam search
updates on prefixes rather than full sequences
works much better than standard update in practice, but...
two major problems for early update
there is no theoretical justification -- why does it work?
it learns too slowly (due to partial examples); e.g. 40 epochs
we’ll solve problems in a much larger framework

4

x z

greedy

r beam

y

early update on prefixes y’, z’

w

SLIDE 16

Our Contributions

theory: a framework for perceptron w/ inexact search
explains early update (and others) as a special case
practice: new update methods within the framework
converges faster and better than early update
real impact on state-of-the-art parsing and tagging
more advantageous when search error is severer

5

x z

greedy

r beam

y

early update on prefixes y’, z’

w

SLIDE 17

In this talk...

Motivations: Structured Learning and Search Efficiency
Structured Perceptron and Inexact Search
perceptron does not converge with inexact search
early update (Collins/Roark ’04) seems to help; but why?
New Perceptron Framework for Inexact Search
explains early update as a special case
convergence theory with arbitrarily inexact search
new update methods within this framework
Experiments

6

SLIDE 18

Structured Perceptron (Collins 02)

simple generalization from binary/multiclass perceptron
online learning: for each example (x, y) in data
inference: find the best output z given current weight w
update weights when if y ≠ z

7

x y=-1 y=+1 x y

update weights if y ≠ z

w

x z

exact inference

SLIDE 19

Structured Perceptron (Collins 02)

simple generalization from binary/multiclass perceptron
online learning: for each example (x, y) in data
inference: find the best output z given current weight w
update weights when if y ≠ z

7

the man bit the dog DT NN VBD DT NN

x y x y=-1 y=+1 x y

update weights if y ≠ z

w

x z

exact inference

SLIDE 20

Structured Perceptron (Collins 02)

simple generalization from binary/multiclass perceptron
online learning: for each example (x, y) in data
inference: find the best output z given current weight w
update weights when if y ≠ z

7

the man bit the dog DT NN VBD DT NN

x y y

update weights if y ≠ z

w

x z

exact inference

x y=-1 y=+1 x y

update weights if y ≠ z

w

x z

exact inference

SLIDE 21

Structured Perceptron (Collins 02)

simple generalization from binary/multiclass perceptron
online learning: for each example (x, y) in data
inference: find the best output z given current weight w
update weights when if y ≠ z

7

the man bit the dog DT NN VBD DT NN

x y y

update weights if y ≠ z

w

x z

exact inference

x y=-1 y=+1 x y

update weights if y ≠ z

w

x z

exact inference

trivial hard

constant classes exponential classes

SLIDE 22

Convergence with Exact Search

linear classification: converges iff. data is separable
structured: converges iff. data separable & search exact
there is an oracle vector that correctly labels all examples
one vs the rest (correct label better than all incorrect labels)
theorem: if separable, then # of updates ≤ R2 / δ2 R: diameter

8

y=+1 y=-1 y100 z ≠ y100 x100 x100 x111 x2000 x3012

δ

Rosenblatt => Collins 1957 2002

SLIDE 23

Convergence with Exact Search

linear classification: converges iff. data is separable
structured: converges iff. data separable & search exact
there is an oracle vector that correctly labels all examples
one vs the rest (correct label better than all incorrect labels)
theorem: if separable, then # of updates ≤ R2 / δ2 R: diameter

8

y=+1 y=-1 y100 z ≠ y100 x100 x100 x111 x2000 x3012

R: diameter

δ

Rosenblatt => Collins 1957 2002

SLIDE 24

Convergence with Exact Search

linear classification: converges iff. data is separable
structured: converges iff. data separable & search exact
there is an oracle vector that correctly labels all examples
one vs the rest (correct label better than all incorrect labels)
theorem: if separable, then # of updates ≤ R2 / δ2 R: diameter

8

y=+1 y=-1 y100 z ≠ y100 x100 x100 x111 x2000 x3012

R: diameter

δ δ

Rosenblatt => Collins 1957 2002

SLIDE 25

Convergence with Exact Search

linear classification: converges iff. data is separable
structured: converges iff. data separable & search exact
there is an oracle vector that correctly labels all examples
one vs the rest (correct label better than all incorrect labels)
theorem: if separable, then # of updates ≤ R2 / δ2 R: diameter

8

y=+1 y=-1 y100 z ≠ y100 x100 x100 x111 x2000 x3012

R: diameter R: diameter

δ δ

Rosenblatt => Collins 1957 2002

SLIDE 26

No Convergence w/ Greedy Search

9

w(k)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

correct label

current model

SLIDE 27

No Convergence w/ Greedy Search

9

w(k)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

w(k)

correct label

current model

SLIDE 28

No Convergence w/ Greedy Search

9

w(k)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

V N

w(k)

correct label

current model

SLIDE 29

No Convergence w/ Greedy Search

9

w(k)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

V N

w(k)

correct label

current model

SLIDE 30

No Convergence w/ Greedy Search

9

w(k)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

V V N N V V

w(k)

correct label

current model

SLIDE 31

No Convergence w/ Greedy Search

9

w(k)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

V V N N V V

w(k)

correct label

current model

SLIDE 32

No Convergence w/ Greedy Search

9

w(k)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N N V V

w(k)

correct label

current model

SLIDE 33

No Convergence w/ Greedy Search

9

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N N V V

w(k)

correct label

u p d a t e

current model

SLIDE 34

No Convergence w/ Greedy Search

9

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N N V V

w(k)

w(k+1)

correct label

u p d a t e

current model new model

SLIDE 35

No Convergence w/ Greedy Search

9

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N N V V

w(k)

w(k+1)

standard perceptron does not converge with greedy search

correct label

u p d a t e

current model new model

SLIDE 36

Early update (Collins/Roark 2004) to rescue

10

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N

w(k)

w(k+1)

stop and update at the first mistake

standard perceptron does not converge with greedy search

correct label

current model

u p d a t e

new model

SLIDE 37

Early update (Collins/Roark 2004) to rescue

10

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N

w(k)

w(k+1)

w(k)

stop and update at the first mistake

standard perceptron does not converge with greedy search

correct label

current model

u p d a t e

new model

SLIDE 38

Early update (Collins/Roark 2004) to rescue

10

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N

w(k)

w(k+1)

w(k)

stop and update at the first mistake

V N

standard perceptron does not converge with greedy search

correct label

current model

u p d a t e

new model

SLIDE 39

Early update (Collins/Roark 2004) to rescue

10

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N

w(k)

w(k+1)

w(k)

stop and update at the first mistake

V N

standard perceptron does not converge with greedy search

correct label

current model

u p d a t e

new model

SLIDE 40

Early update (Collins/Roark 2004) to rescue

10

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N

w(k)

w(k+1)

∆Φ(x, y, z)

w(k)

stop and update at the first mistake

V N

standard perceptron does not converge with greedy search

correct label

current model

u p d a t e

new model

SLIDE 41

Early update (Collins/Roark 2004) to rescue

10

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N

w(k)

w(k+1)

∆Φ(x, y, z)

w(k+1)

w(k)

stop and update at the first mistake

V N

standard perceptron does not converge with greedy search

correct label

current model new model

u p d a t e

new model

SLIDE 42

Why?

why does inexact search break convergence property?
what is required for convergence? exactness?
why does early update (Collins/Roark 04) work?
it works well in practice and is now a standard method
but there has been no theoretical justification
we answer these Qs by inspecting the convergence proof

11

V V N N V N V V

∆Φ(x, y, z)

w(k+1)

w(k)

SLIDE 43

Geometry of Convergence Proof pt 1

12

y

update weights if y ≠ z

w

x z

exact inference

y correct

label

SLIDE 44

Geometry of Convergence Proof pt 1

12

y

update weights if y ≠ z

w

x z

exact inference

y

w(k) correct label

current model

z

exact 1-best

SLIDE 45

Geometry of Convergence Proof pt 1

12

y

update weights if y ≠ z

w

x z

exact inference

y

w(k) correct label

∆Φ(x, y, z)

update current model update

perceptron update:

z

exact 1-best

SLIDE 46

Geometry of Convergence Proof pt 1

12

y

update weights if y ≠ z

w

x z

exact inference

y

w(k)

w(k+1)

correct label

∆Φ(x, y, z)

update current model update new model

perceptron update:

z

exact 1-best

SLIDE 47

Geometry of Convergence Proof pt 1

12

y

update weights if y ≠ z

w

x z

exact inference

y

w(k)

w(k+1)

correct label

∆Φ(x, y, z)

update current model update new model

perceptron update:

z

exact 1-best

unit oracle vector u

SLIDE 48

Geometry of Convergence Proof pt 1

12

y

update weights if y ≠ z

w

x z

exact inference

y

w(k)

w(k+1)

correct label

∆Φ(x, y, z)

update current model update new model

perceptron update:

z

exact 1-best

δ separation

unit oracle vector u

margin

≥ δ

SLIDE 49

Geometry of Convergence Proof pt 1

12

y

update weights if y ≠ z

w

x z

exact inference

y

w(k)

w(k+1)

correct label

∆Φ(x, y, z)

update current model update new model

perceptron update:

z

exact 1-best

(by induction)

δ separation

unit oracle vector u

margin

≥ δ

SLIDE 50

Geometry of Convergence Proof pt 1

12

y

update weights if y ≠ z

w

x z

exact inference

y

w(k)

w(k+1)

correct label

∆Φ(x, y, z)

update current model update new model

perceptron update:

z

exact 1-best

(by induction)

δ separation

unit oracle vector u

margin

≥ δ

(part 1: upperbound)

SLIDE 51

Geometry of Convergence Proof pt 2

13

y

update weights if y ≠ z

w

x z

exact inference

y correct

label

SLIDE 52

Geometry of Convergence Proof pt 2

13

y

update weights if y ≠ z

w

x z

exact inference

y

w(k) correct label

current model

z

exact 1-best

SLIDE 53

Geometry of Convergence Proof pt 2

13

y

update weights if y ≠ z

w

x z

exact inference

y

w(k) correct label

∆Φ(x, y, z)

update current model update

perceptron update:

z

exact 1-best

SLIDE 54

Geometry of Convergence Proof pt 2

13

y

update weights if y ≠ z

w

x z

exact inference

y

w(k)

w(k+1)

correct label

∆Φ(x, y, z)

update current model update new model

perceptron update:

z

exact 1-best

SLIDE 55

Geometry of Convergence Proof pt 2

13

y

update weights if y ≠ z

w

x z

exact inference

y

w(k)

w(k+1)

correct label

∆Φ(x, y, z)

update current model update new model

perceptron update:

z

exact 1-best

SLIDE 56

Geometry of Convergence Proof pt 2

13

y

update weights if y ≠ z

w

x z

exact inference

y

w(k)

w(k+1)

correct label

∆Φ(x, y, z)

update current model update new model

perceptron update:

z

exact 1-best

SLIDE 57

Geometry of Convergence Proof pt 2

13

y

update weights if y ≠ z

w

x z

exact inference

y

w(k)

w(k+1)

correct label

∆Φ(x, y, z)

update current model update new model

perceptron update:

R: max diameter

z

exact 1-best

diameter ≤ R2

SLIDE 58

Geometry of Convergence Proof pt 2

13

y

update weights if y ≠ z

w

x z

exact inference

y

w(k)

w(k+1)

correct label

∆Φ(x, y, z)

update current model update new model

perceptron update:

R: max diameter

z

exact 1-best

diameter ≤ R2

SLIDE 59

<90˚

Geometry of Convergence Proof pt 2

13

y

update weights if y ≠ z

w

x z

exact inference

y

w(k)

w(k+1)

violation: incorrect label scored higher

correct label

∆Φ(x, y, z)

update current model update new model

perceptron update: violation

R: max diameter

z

exact 1-best

diameter ≤ R2

SLIDE 60

<90˚

Geometry of Convergence Proof pt 2

13

y

update weights if y ≠ z

w

x z

exact inference

y

w(k)

w(k+1)

violation: incorrect label scored higher

correct label

∆Φ(x, y, z)

update current model update new model

perceptron update: violation

by induction:

R: max diameter

z

exact 1-best

diameter ≤ R2 (part 2: upperbound)

SLIDE 61

<90˚

Geometry of Convergence Proof pt 2

13

y

update weights if y ≠ z

w

x z

exact inference

y

w(k)

w(k+1)

violation: incorrect label scored higher

parts 1+2 => update bounds:

correct label

∆Φ(x, y, z)

update current model update new model

perceptron update: violation

by induction:

R: max diameter

z

exact 1-best

diameter ≤ R2 k ≤ R2/δ2 (part 2: upperbound)

SLIDE 62

<90˚

Violation is All we need!

14

y

violation: incorrect label scored higher

correct label

∆Φ(x, y, z)

update

exact search is not really required by the proof
rather, it is only used to ensure violation!

w(k)

w(k+1)

current model update new model

R: max diameter

z

exact 1-best

the proof only uses 3 facts:

1. separation (margin)
2. diameter (always finite)
3. violation (but no need for exact)

SLIDE 63

<90˚

Violation is All we need!

14

y

violation: incorrect label scored higher

correct label

∆Φ(x, y, z)

update

exact search is not really required by the proof
rather, it is only used to ensure violation!

w(k)

w(k+1)

current model update new model

R: max diameter

all violations

z

exact 1-best

the proof only uses 3 facts:

1. separation (margin)
2. diameter (always finite)
3. violation (but no need for exact)

SLIDE 64

y

Violation-Fixing Perceptron

if we guarantee violation, we don’t care about exactness!
violation is good b/c we can at least fix a mistake

15

y

update weights if y ≠ z

w

x z

exact inference

y

update weights if y’ ≠ z

w

x z

find violation

y ’

same mistake bound as before!

standard perceptron violation-fixing perceptron

all violations

SLIDE 65

y

Violation-Fixing Perceptron

if we guarantee violation, we don’t care about exactness!
violation is good b/c we can at least fix a mistake

15

y

update weights if y ≠ z

w

x z

exact inference

y

update weights if y’ ≠ z

w

x z

find violation

y ’

same mistake bound as before!

standard perceptron violation-fixing perceptron

all violations

SLIDE 66

y

Violation-Fixing Perceptron

if we guarantee violation, we don’t care about exactness!
violation is good b/c we can at least fix a mistake

15

y

update weights if y ≠ z

w

x z

exact inference

y

update weights if y’ ≠ z

w

x z

find violation

y ’

same mistake bound as before!

standard perceptron violation-fixing perceptron

all violations

a l l p

s

s i b l e u p d a t e s

SLIDE 67

What if can’t guarantee violation

16

current model

SLIDE 68

What if can’t guarantee violation

this is why perceptron doesn’t work well w/ inexact search
because not every update is guaranteed to be a violation
thus the proof breaks; no convergence guarantee

16

current model

SLIDE 69

What if can’t guarantee violation

this is why perceptron doesn’t work well w/ inexact search
because not every update is guaranteed to be a violation
thus the proof breaks; no convergence guarantee
example: beam or greedy search
the model might prefer the correct label (if exact search)
but the search prunes it away
such a non-violation update is “bad”

because it doesn’t fix any mistake

the new model still misguides the search

16

beam

b a d u p d a t e current model

SLIDE 70

Standard Update: No Guarantee

17

w(k)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

correct label

SLIDE 71

Standard Update: No Guarantee

17

w(k)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

w(k)

correct label

SLIDE 72

Standard Update: No Guarantee

17

w(k)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

V N

w(k)

correct label

SLIDE 73

Standard Update: No Guarantee

17

w(k)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

V V N N V V

w(k)

correct label

SLIDE 74

Standard Update: No Guarantee

17

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N N V V

w(k)

w(k+1)

correct label

SLIDE 75

Standard Update: No Guarantee

17

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N N V V

w(k)

w(k+1)

standard update doesn’t converge b/c it doesn’t guarantee violation

correct label scores higher. non-violation: bad update!

correct label

SLIDE 76

Early Update: Guarantees Violation

18

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N N V V

w(k)

w(k+1)

standard update doesn’t converge b/c it doesn’t guarantee violation

correct label

SLIDE 77

Early Update: Guarantees Violation

18

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N N V V

w(k)

w(k+1)

w(k)

standard update doesn’t converge b/c it doesn’t guarantee violation

correct label

SLIDE 78

Early Update: Guarantees Violation

18

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N N V V

w(k)

w(k+1)

w(k)

V N

standard update doesn’t converge b/c it doesn’t guarantee violation

correct label

SLIDE 79

Early Update: Guarantees Violation

18

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N N V V

w(k)

w(k+1)

w(k)

V N

standard update doesn’t converge b/c it doesn’t guarantee violation

correct label

SLIDE 80

Early Update: Guarantees Violation

18

w(k)

∆Φ(x, y, z)

V V N N V N V V

training example

time flies N V

utput space

{N,V} x {N, V}

N V V V N N V V

w(k)

w(k+1)

∆Φ(x, y, z)

w(k+1)

w(k)

early update: incorrect prefix scores higher: a violation!

V N

standard update doesn’t converge b/c it doesn’t guarantee violation

correct label

SLIDE 81

Early Update: from Greedy to Beam

beam search is a generalization of greedy (where b=1)
at each stage we keep top b hypothesis
widely used: tagging, parsing, translation...
early update -- when correct label first falls off the beam
up to this point the incorrect prefix should score higher
standard update (full update) -- no guarantee!

19

standard update (no guarantee!)

SLIDE 82

Early Update: from Greedy to Beam

beam search is a generalization of greedy (where b=1)
at each stage we keep top b hypothesis
widely used: tagging, parsing, translation...
early update -- when correct label first falls off the beam
up to this point the incorrect prefix should score higher
standard update (full update) -- no guarantee!

19

correct label falls off beam (pruned)

correct

standard update (no guarantee!)

SLIDE 83

Early Update: from Greedy to Beam

beam search is a generalization of greedy (where b=1)
at each stage we keep top b hypothesis
widely used: tagging, parsing, translation...
early update -- when correct label first falls off the beam
up to this point the incorrect prefix should score higher
standard update (full update) -- no guarantee!

19

correct label falls off beam (pruned)

correct incorrect

standard update (no guarantee!)

SLIDE 84

Early Update: from Greedy to Beam

beam search is a generalization of greedy (where b=1)
at each stage we keep top b hypothesis
widely used: tagging, parsing, translation...
early update -- when correct label first falls off the beam
up to this point the incorrect prefix should score higher
standard update (full update) -- no guarantee!

19

early update

correct label falls off beam (pruned)

correct incorrect violation guaranteed: incorrect prefix scores higher up to this point

standard update (no guarantee!)

SLIDE 85

Early Update as Violation-Fixing

20

beam

correct label falls off beam (pruned)

standard update (bad!)

y

update weights if y’ ≠ z

w

x z

find violation

y’

prefix violations

y’ z y

early update

SLIDE 86

Early Update as Violation-Fixing

20

beam

correct label falls off beam (pruned)

standard update (bad!)

y

update weights if y’ ≠ z

w

x z

find violation

y’

prefix violations

y’ z y

also new definition of “beam separability”: a correct prefix should score higher than any incorrect prefix

f the same length

(maybe too strong)

cf. Kulesza and Pereira,2007

early update

SLIDE 87

New Update Methods: max-violation, ...

21

beam

early standard (bad!) max-violation latest

we now established a theory for early update (Collins/Roark)
but it learns too slowly due to partial updates
max-violation: use the prefix where violation is maximum
“worst-mistake” in the search space
all these update methods are violation-fixing perceptrons

SLIDE 88

Experiments

the man bit the dog DT NN VBD DT NN

x y

the man bit the dog

x y

bit man the dog the

trigram part-of-speech tagging incremental dependency parsing

local features only, exact search tractable (proof of concept) non-local features, exact search intractable (real impact)

SLIDE 89

1) Trigram Part of Speech Tagging

23

standard update performs terribly with greedy search (b=1)
because search error is severe at b=1: half updates are bad!
no real difference beyond b=2: search error becomes rare

% of bad (non-violation) standard updates

53% 10% 1.5% 0.5%

SLIDE 90

Max-Violation Reduces Training Time

24

max-violation peaks at b=2, greatly reduced training time
early update achieves the highest dev/test accuracy
higher than the best published accuracy (Shen et al ‘07)
future work: add non-local features to tagging

beam iter time test

standard early

max-violation

6 162m 97.28

4 6 37m 97.35 2 3 26m 97.33

Shen et Shen et al (2007) (2007) 2007)

97.33

SLIDE 91

2) Incremental Dependency Parsing

DP incremental dependency parser (Huang and Sagae 2010)
non-local history-based features rule out exact DP
we use beam search, and search error is severe
baseline: early update. extremely slow: 38 iterations

25

SLIDE 92

Max-violation converges much faster

early update: 38 iterations, 15.4 hours (92.24)
max-violation: 10 iterations, 4.6 hours (92.25)

12 iterations, 5.5 hours (92.32)

26

SLIDE 93

Comparison b/w tagging & parsing

search error is much more severe in parsing than in tagging
standard update is OK in tagging except greedy search (b=1)
but performs horribly in parsing even at large beam (b=8)
because ~50% of standard updates are bad (non-violation)!

27

test

standard early max-violation

79.1

92.1 92.2

% of bad standard updates

SLIDE 94

Comparison b/w tagging & parsing

search error is much more severe in parsing than in tagging
standard update is OK in tagging except greedy search (b=1)
but performs horribly in parsing even at large beam (b=8)
because ~50% of standard updates are bad (non-violation)!

27

test

standard early max-violation

79.1

92.1 92.2

% of bad standard updates

take-home message:

ur methods are more helpful

for harder search problems!

SLIDE 95

Related Work and Discussions

28

SLIDE 96

Related Work and Discussions

our “violation-fixing” framework include as special cases
early-update (Collins and Roark, 2004)
a variant of LaSO (Daume and Marcu, 2005)
not sure about Searn (Daume et al, 2009)

28

SLIDE 97

Related Work and Discussions

our “violation-fixing” framework include as special cases
early-update (Collins and Roark, 2004)
a variant of LaSO (Daume and Marcu, 2005)
not sure about Searn (Daume et al, 2009)
“beam-separability” or “greedy-separability” related to:
“algorithmic-separability” of (Kulesza and Pereira, 2007)
but these conditions are too strong to hold in practice

28

SLIDE 98

Related Work and Discussions

our “violation-fixing” framework include as special cases
early-update (Collins and Roark, 2004)
a variant of LaSO (Daume and Marcu, 2005)
not sure about Searn (Daume et al, 2009)
“beam-separability” or “greedy-separability” related to:
“algorithmic-separability” of (Kulesza and Pereira, 2007)
but these conditions are too strong to hold in practice
under-generating (beam) vs. over-generating (LP-relax.)
Kulesza & Pereira and Martins et al (2011): LP-relaxation
Finley and Joachims (2008): both under and over for SVM

28

SLIDE 99

Conclusions

Structured Learning with Inexact Search is Important
Two contributions from this work:
theory: a general violation-fixing perceptron framework
convergence for inexact search under new defs of separability
subsumes previous work (early update & LaSO) as special cases
practice: new update methods within this framework
“max-violation” learns faster and better than early update
dramatically reducing training time by 3-5 folds
improves over state-of-the-art tagging and parsing systems
our methods are more helpful to harder search problems! :)

29

SLIDE 100

Thank you!

Liang apologizes for not able to come due to visa/passport reasons. Many thanks to Philipp Koehn for presenting it! Danke! % of bad updates in standard perceptron

lhuang@isi.edu

parsing accuracy

n held-out