Discriminative Training February 19, 2013 Tuesday, February 19, 13 - - PowerPoint PPT Presentation

Discriminative Training

February 19, 2013

Tuesday, February 19, 13

Noisy Channels Again

English

source

p(e)

Tuesday, February 19, 13

Noisy Channels Again

German English

source

p(e) p(g | e)

Tuesday, February 19, 13

Noisy Channels Again

German English

source

decoder

p(e) p(g | e) e∗ = arg max

e

p(e | g) = arg max

e

p(g | e) × p(e) p(g) = arg max

e

p(g | e) × p(e)

Tuesday, February 19, 13

Noisy Channels Again

e∗ = arg max

e

p(e | g) = arg max

e

p(g | e) × p(e) p(g) = arg max

e

p(g | e) × p(e)

Tuesday, February 19, 13

Noisy Channels Again

e∗ = arg max

e

p(e | g) = arg max

e

p(g | e) × p(e) p(g) = arg max

e

p(g | e) × p(e) = arg max

e

log p(g | e) + log p(e) = arg max

e

1 1 > | {z }

w>

log p(g | e) log p(e)

• |

{z }

h(g,e)

Tuesday, February 19, 13

Noisy Channels Again

e∗ = arg max

e

p(e | g) = arg max

e

p(g | e) × p(e) p(g) = arg max

e

p(g | e) × p(e) = arg max

e

log p(g | e) + log p(e) = arg max

e

1 1 > | {z }

w>

log p(g | e) log p(e)

• |

{z }

h(g,e)

Tuesday, February 19, 13

Noisy Channels Again

e∗ = arg max

e

p(e | g) = arg max

e

p(g | e) × p(e) p(g) = arg max

e

p(g | e) × p(e) = arg max

e

log p(g | e) + log p(e) = arg max

e

1 1 > | {z }

w>

log p(g | e) log p(e)

• |

{z }

h(g,e)

Does this look familiar?

Tuesday, February 19, 13

The Noisy Channel

• log p(g | e)
• log p(e)

Tuesday, February 19, 13

As a Linear Model

• log p(g | e)
• log p(e)

~ w

Tuesday, February 19, 13

As a Linear Model

• log p(g | e)
• log p(e)

~ w

Tuesday, February 19, 13

As a Linear Model

• log p(g | e)
• log p(e)

~ w

Tuesday, February 19, 13

As a Linear Model

• log p(g | e)
• log p(e)

~ w

g

Improvement 1: change to find better translations ~ w

Tuesday, February 19, 13

As a Linear Model

• log p(g | e)
• log p(e)

~ w

Tuesday, February 19, 13

As a Linear Model

• log p(g | e)
• log p(e)

~ w

Tuesday, February 19, 13

As a Linear Model

• log p(g | e)
• log p(e)

~ w

Tuesday, February 19, 13

As a Linear Model

• log p(g | e)
• log p(e)

~ w

Improvement 2: Add dimensions to make points separable

Tuesday, February 19, 13

Linear Models

• Improve the modeling capacity of the noisy

channel in two ways

• Reorient the weight vector
• Add new dimensions (new features)
• Questions
• What features?
• How do we set the weights?

e⇤ = arg max

e

w>h(g, e) h(g, e) w

Tuesday, February 19, 13

Mann beißt Hund

18

Tuesday, February 19, 13

Mann beißt Hund

18

x BITES y

Tuesday, February 19, 13

Mann beißt Hund man bites cat

Mann beißt Hund

man bite cat

Mann beißt Hund

man bites dog

Mann beißt Hund

man chase dog

Mann beißt Hund

man bite dog

Mann beißt Hund

man bites dog

Mann beißt Hund

19

x BITES y

Tuesday, February 19, 13

Mann beißt Hund man bites cat

Mann beißt Hund

man bite cat

Mann beißt Hund

man bites dog

Mann beißt Hund

man chase dog

Mann beißt Hund

man bite dog

Mann beißt Hund

man bites dog

Mann beißt Hund

19

x BITES y

Tuesday, February 19, 13

Mann beißt Hund man bites cat

Mann beißt Hund

man bite cat

Mann beißt Hund

man bites dog

Mann beißt Hund

man chase dog

Mann beißt Hund

man bite dog

Mann beißt Hund

man bites dog

Mann beißt Hund

19

x BITES y

Tuesday, February 19, 13

Mann beißt Hund man bites cat

Mann beißt Hund

man bite cat

Mann beißt Hund

man bites dog

Mann beißt Hund

man chase dog

Mann beißt Hund

man bite dog

Mann beißt Hund

man bites dog

Mann beißt Hund

19

x BITES y

Tuesday, February 19, 13

Mann beißt Hund man bites cat

Mann beißt Hund

man bite cat

Mann beißt Hund

man bites dog

Mann beißt Hund

man chase dog

Mann beißt Hund

man bite dog

Mann beißt Hund

man bites dog

Mann beißt Hund

19

x BITES y

Tuesday, February 19, 13

Feature Classes

20

Lexical

bank = “River bank” vs. “Financial institution”

Are lexical choices appropriate?

Tuesday, February 19, 13

Feature Classes

20

Lexical

bank = “River bank” vs. “Financial institution”

Are lexical choices appropriate?

Configurational

Are semantic/syntactic relations preserved?

“Dog bites man” vs. “Man bites dog”

Tuesday, February 19, 13

Feature Classes

20

Lexical

bank = “River bank” vs. “Financial institution”

Are lexical choices appropriate?

Configurational

Are semantic/syntactic relations preserved?

“Dog bites man” vs. “Man bites dog”

Grammatical

Is the output fluent / well-formed?

“Man bites dog” vs. “Man bite dog”

Tuesday, February 19, 13

21

man bites cat

Mann beißt Hund

What do lexical features look like?

Tuesday, February 19, 13

21

man bites cat

Mann beißt Hund

What do lexical features look like?

Tuesday, February 19, 13

21

man bites cat

Mann beißt Hund

What do lexical features look like?

score(g, e) = w>h(g, e) h15,342(g, e) = ( 1, ∃i, j : gi = Hund, ej = cat 0,

• therwise

First attempt:

Tuesday, February 19, 13

But what if a cat is being chased by a Hund?

21

man bites cat

Mann beißt Hund

What do lexical features look like?

score(g, e) = w>h(g, e) h15,342(g, e) = ( 1, ∃i, j : gi = Hund, ej = cat 0,

• therwise

First attempt:

Tuesday, February 19, 13

man bites cat

Mann beißt Hund

Latent variables enable more precise features:

22

What do lexical features look like?

score(g, e, a) = w>h(g, e, a) h15,342(g, e, a) = X

(i,j)2a

( 1, if gi = Hund, ej = cat 0,

• therwise

Tuesday, February 19, 13

Standard Features

• Target side features
• log p(e) [ n-gram language model ]
• Number of words in hypothesis
• Non-English character count
• Source + target features
• log relative frequency e|f of each rule [ log #(e,f) - log #(f) ]
• log relative frequency f|e of each rule [ log #(e,f) - log #(e) ]
• “lexical translation” log probability e|f of each rule [ ≈ log pmodel1(e|f) ]
• “lexical translation” log probability f|e of each rule [ ≈ log pmodel1(f|e) ]
• Other features
• Count of rules/phrases used
• Reordering pattern probabilities

23

Tuesday, February 19, 13

Parameter Learning

24

Tuesday, February 19, 13

25

h1 h2

Hypothesis Space

Tuesday, February 19, 13

25

h1 h2

Hypothesis Space

Hypotheses

Tuesday, February 19, 13

26

h1 h2

Hypothesis Space

References

Tuesday, February 19, 13

Preliminaries

27

We assume a decoder that computes:

he⇤, a⇤i = arg max

he,ai w>h(g, e, a)

And K-best lists of, that is:

{e⇤

i , a⇤ i ⇥}i=K i=1 = arg ith- max he,ai w>h(g, e, a)

Standard, efficient algorithms exist for this.

Tuesday, February 19, 13

Learning Weights

• Try to match the reference translation exactly
• Conditional random field
• Maximize the conditional probability of the

reference translations

• “Average” over the different latent variables
• Max-margin
• Find the weight vector that separates the reference

translation from others by the maximal margin

• Maximal setting of the latent variables

28

Tuesday, February 19, 13

Learning Weights

• Try to match the reference translation exactly
• Conditional random field
• Maximize the conditional probability of the

reference translations

• “Average” over the different latent variables
• Max-margin
• Find the weight vector that separates the reference

translation from others by the maximal margin

• Maximal setting of the latent variables

28

Tuesday, February 19, 13

Problems

• These methods give “full credit” when the

model exactly produces the reference and no credit otherwise

• What is the problem with this?
• There are many ways to translate a sentence
• What if we have multiple reference

translations?

• What about partial credit?

29

Tuesday, February 19, 13

Problems

• These methods give “full credit” when the

model exactly produces the reference and no credit otherwise

• What is the problem with this?
• There are many ways to translate a sentence
• What if we have multiple reference

translations?

• What about partial credit?

29

Tuesday, February 19, 13

Cost-Sensitive Training

• Assume we have a cost function that gives

a score for how good/bad a translation is

• Optimize the weight vector by making

reference to this function

• We will talk about two ways to do this

30

(ˆ e, E) ⇥ [0, 1]

Tuesday, February 19, 13

31

h1 h2 ~ w

K-Best List Example

Tuesday, February 19, 13

31

h1 h2 ~ w

#2#1

K-Best List Example

#3 #4 #5 #6 #7 #8 #9 #10

Tuesday, February 19, 13

32

h1 h2

#2#1

K-Best List Example

#3 #4 #5 #6 #7 #8 #9 #10

0.8 ≤ < 1.0 0.6 ≤ < 0.8 0.4 ≤ < 0.6 0.2 ≤ < 0.4 0.0 ≤ < 0.2 ~ w

Tuesday, February 19, 13

Training as Classification

• Pairwise Ranking Optimization
• Reduce training problem to binary classification

with a linear model

• Algorithm
• For i=1 to N
• Pick random pair of hypotheses (A,B) from K-best list
• Use cost function to determine if is A or B better
• Create ith training instance
• Train binary linear classifier

33

Tuesday, February 19, 13

34

h1 h2

#2 #1 #3 #4 #5 #6 #7 #8 #9 #10

0.8 ≤ < 1.0 0.6 ≤ < 0.8 0.4 ≤ < 0.6 0.2 ≤ < 0.4 0.0 ≤ < 0.2 h1 h2

Tuesday, February 19, 13

34

h1 h2

#2 #1 #3 #4 #5 #6 #7 #8 #9 #10

0.8 ≤ < 1.0 0.6 ≤ < 0.8 0.4 ≤ < 0.6 0.2 ≤ < 0.4 0.0 ≤ < 0.2 h1 h2

Tuesday, February 19, 13

35

h1 h2

#2 #1 #3 #4 #5 #6 #7 #8 #9 #10

0.8 ≤ < 1.0 0.6 ≤ < 0.8 0.4 ≤ < 0.6 0.2 ≤ < 0.4 0.0 ≤ < 0.2 h1 h2

Worse!

Tuesday, February 19, 13

36

h1 h2

#2 #1 #3 #4 #5 #6 #7 #8 #9 #10

0.8 ≤ < 1.0 0.6 ≤ < 0.8 0.4 ≤ < 0.6 0.2 ≤ < 0.4 0.0 ≤ < 0.2 h1 h2

Worse!

Tuesday, February 19, 13

37

h1 h2

#2 #1 #3 #4 #5 #6 #7 #8 #9 #10

0.8 ≤ < 1.0 0.6 ≤ < 0.8 0.4 ≤ < 0.6 0.2 ≤ < 0.4 0.0 ≤ < 0.2 h1 h2

Tuesday, February 19, 13

38

h1 h2

#2 #1 #3 #4 #5 #6 #7 #8 #9 #10

0.8 ≤ < 1.0 0.6 ≤ < 0.8 0.4 ≤ < 0.6 0.2 ≤ < 0.4 0.0 ≤ < 0.2 h1 h2

Better!

Tuesday, February 19, 13

39

h1 h2

#2 #1 #3 #4 #5 #6 #7 #8 #9 #10

0.8 ≤ < 1.0 0.6 ≤ < 0.8 0.4 ≤ < 0.6 0.2 ≤ < 0.4 0.0 ≤ < 0.2 h1 h2

Better!

Tuesday, February 19, 13

40

h1 h2

#2 #1 #3 #4 #5 #6 #7 #8 #9 #10

0.8 ≤ < 1.0 0.6 ≤ < 0.8 0.4 ≤ < 0.6 0.2 ≤ < 0.4 0.0 ≤ < 0.2 h1 h2

Worse!

Tuesday, February 19, 13

41

h1 h2

#2 #1 #3 #4 #5 #6 #7 #8 #9 #10

0.8 ≤ < 1.0 0.6 ≤ < 0.8 0.4 ≤ < 0.6 0.2 ≤ < 0.4 0.0 ≤ < 0.2 h1 h2

Better!

Tuesday, February 19, 13

42

h1 h2

#2 #1 #3 #4 #5 #6 #7 #8 #9 #10

0.8 ≤ < 1.0 0.6 ≤ < 0.8 0.4 ≤ < 0.6 0.2 ≤ < 0.4 0.0 ≤ < 0.2 h1 h2

Tuesday, February 19, 13

43

h1 h2

Fit a linear model

Tuesday, February 19, 13

44

h1 h2

Fit a linear model

~ w

Tuesday, February 19, 13

45

h1 h2

#2#1

K-Best List Example

#3 #4 #5 #6 #7 #8 #9 #10

0.8 ≤ < 1.0 0.6 ≤ < 0.8 0.4 ≤ < 0.6 0.2 ≤ < 0.4 0.0 ≤ < 0.2 ~ w

Tuesday, February 19, 13

MERT

• Minimum Error Rate Training
• Directly target an automatic evaluation

metric

• BLEU is defined at the corpus level
• MERT optimizes at the corpus level
• Downsides
• Does not deal well with > ~20 features

46

Tuesday, February 19, 13

MERT

47

Now pick a search vector , and consider how the score of this hypothesis will change: Given weight vector , any hypothesis will have a (scalar) score

w he, ai m = w>h(g, e, a) m = (w + γv)>h(g, e, a) = w>h(g, e, a) | {z }

b

+γ v>h(g, e, a) | {z }

a

= aγ + b v wnew = w + γv m

Tuesday, February 19, 13

MERT

48

Now pick a search vector , and consider how the score of this hypothesis will change: Given weight vector , any hypothesis will have a (scalar) score

w he, ai m = w>h(g, e, a) m = (w + γv)>h(g, e, a) = w>h(g, e, a) | {z }

b

+γ v>h(g, e, a) | {z }

a

= aγ + b v wnew = w + γv m

Tuesday, February 19, 13

MERT

49

Now pick a search vector , and consider how the score of this hypothesis will change: Given weight vector , any hypothesis will have a (scalar) score

w he, ai m = w>h(g, e, a) m = (w + γv)>h(g, e, a) = w>h(g, e, a) | {z }

b

+γ v>h(g, e, a) | {z }

a

= aγ + b v wnew = w + γv m

Tuesday, February 19, 13

MERT

50

Now pick a search vector , and consider how the score of this hypothesis will change: Given weight vector , any hypothesis will have a (scalar) score

w he, ai m = w>h(g, e, a) m = (w + γv)>h(g, e, a) = w>h(g, e, a) | {z }

b

+γ v>h(g, e, a) | {z }

a

= aγ + b v wnew = w + γv m

Tuesday, February 19, 13

MERT

51

Now pick a search vector , and consider how the score of this hypothesis will change: Given weight vector , any hypothesis will have a (scalar) score

w he, ai m = w>h(g, e, a) m = (w + γv)>h(g, e, a) = w>h(g, e, a) | {z }

b

+γ v>h(g, e, a) | {z }

a

= aγ + b v wnew = w + γv m

Tuesday, February 19, 13

MERT

51

Now pick a search vector , and consider how the score of this hypothesis will change: Given weight vector , any hypothesis will have a (scalar) score

w he, ai m = w>h(g, e, a) m = (w + γv)>h(g, e, a) = w>h(g, e, a) | {z }

b

+γ v>h(g, e, a) | {z }

a

= aγ + b v wnew = w + γv m

Linear function in 2D!

Tuesday, February 19, 13

MERT

52

m γ

Tuesday, February 19, 13

MERT

53

Recall our k-best set {e∗

i , a∗ i ⇥}K i=1

m γ

Tuesday, February 19, 13

MERT

54

Recall our k-best set {e∗

i , a∗ i ⇥}K i=1

m γ

Tuesday, February 19, 13

MERT

55

m γ

Tuesday, February 19, 13

MERT

56

m he∗

162, a∗ 162i

he∗

28, a∗ 28i

he∗

73, a∗ 73i

γ

Tuesday, February 19, 13

MERT

57

m γ he∗

162, a∗ 162i

he∗

28, a∗ 28i

he∗

73, a∗ 73i

Tuesday, February 19, 13

MERT

58

m γ he∗

162, a∗ 162i

he∗

28, a∗ 28i

he∗

73, a∗ 73i

γ

errors

Tuesday, February 19, 13

MERT

59

m γ he∗

162, a∗ 162i

he∗

28, a∗ 28i

he∗

73, a∗ 73i

γ

errors

Tuesday, February 19, 13

MERT

60

m γ γ

errors

Tuesday, February 19, 13

MERT

61

m γ γ

errors

γ

Tuesday, February 19, 13

62

γ

errors

wnew = γ∗v + w

Let

γ∗

Tuesday, February 19, 13

MERT

• In practice “errors” are sufficient statistics

for evaluation metrics (e.g., BLEU)

• Can maximize or minimize!
• Envelope can also be computed using

dynamic programming

• Interesting complexity bounds
• How do you pick the search direction?

63

Tuesday, February 19, 13

Summary

• Evaluation metrics
• Figure out how well we’re doing
• Figure out if a feature helps
• But ALSO: train your system!
• What’s a great way to improve translation?
• Improve evaluation!

64

Tuesday, February 19, 13

Thank You!

65

m

γ he∗

162, a∗ 162i

he∗

28, a∗ 28i

he∗

73, a∗ 73i

Tuesday, February 19, 13