Syntactic Transfer Using a Bilingual Lexicon Greg Durrett, Adam - - PowerPoint PPT Presentation

▶
syntactic transfer using a bilingual lexicon
SMART_READER_LITE
LIVE PREVIEW

Syntactic Transfer Using a Bilingual Lexicon Greg Durrett, Adam - - PowerPoint PPT Presentation

Dec 05, 2023 180 likes •928 views

Syntactic Transfer Using a Bilingual Lexicon Greg Durrett, Adam Pauls, and Dan Klein UC Berkeley Parsing a New Language Mosambik hofft auf Handel mit anderen Mitgliedern Parsing a New Language Mosambik hofft auf Handel mit anderen

slide-1
SLIDE 1

Syntactic Transfer Using a Bilingual Lexicon

Greg Durrett, Adam Pauls, and Dan Klein

UC Berkeley

slide-2
SLIDE 2

Parsing a New Language

Mosambik hofft auf Handel mit anderen Mitgliedern

slide-3
SLIDE 3

Parsing a New Language

Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

slide-4
SLIDE 4

Parsing a New Language

Mosambik hofft auf Handel mit anderen Mitgliedern [Berg-Kirkpatrick and Klein (2010), Petrov et al. (2011)] NOUN VERB PREP NOUN PREP ADJ NOUN Mozambique hope on trade with other members

slide-5
SLIDE 5

Parsing a New Language

[Berg-Kirkpatrick and Klein (2010), Petrov et al. (2011)] NOUN VERB PREP NOUN PREP ADJ NOUN [McDonald et al. (2011), Cohen et al. (2011)] Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

slide-6
SLIDE 6

Parsing a New Language

[Berg-Kirkpatrick and Klein (2010), Petrov et al. (2011)] NOUN VERB PREP NOUN PREP ADJ NOUN [McDonald et al. (2011), Cohen et al. (2011)] Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

slide-7
SLIDE 7

Parsing a New Language

[Berg-Kirkpatrick and Klein (2010), Petrov et al. (2011)] NOUN VERB PREP NOUN PREP ADJ NOUN [McDonald et al. (2011), Cohen et al. (2011)] Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

slide-8
SLIDE 8

Parsing a New Language

[Berg-Kirkpatrick and Klein (2010), Petrov et al. (2011)] NOUN VERB PREP NOUN PREP ADJ NOUN [McDonald et al. (2011), Cohen et al. (2011)] Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

slide-9
SLIDE 9

Parsing a New Language

[Berg-Kirkpatrick and Klein (2010), Petrov et al. (2011)] NOUN VERB PREP NOUN PREP ADJ NOUN [McDonald et al. (2011), Cohen et al. (2011)]

?

Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

slide-10
SLIDE 10

Parsing a New Language

[Berg-Kirkpatrick and Klein (2010), Petrov et al. (2011)] NOUN VERB PREP NOUN PREP ADJ NOUN [McDonald et al. (2011), Cohen et al. (2011)]

?

Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

slide-11
SLIDE 11

Parsing a New Language

[Berg-Kirkpatrick and Klein (2010), Petrov et al. (2011)] NOUN VERB PREP NOUN PREP ADJ NOUN [McDonald et al. (2011), Cohen et al. (2011)]

?

Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

slide-12
SLIDE 12

Token-level Projection

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hopes for trade with other members [Hwa et al. (2005), Ganchev et al. (2009), McDonald et al. (2011)]

slide-13
SLIDE 13

Token-level Projection

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hopes for trade with other members [Hwa et al. (2005), Ganchev et al. (2009), McDonald et al. (2011)] Assumes access to parallel data

slide-14
SLIDE 14

Token-level Projection

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hopes for trade with other members [Hwa et al. (2005), Ganchev et al. (2009), McDonald et al. (2011)] Asymmetric train and test conditions Assumes access to parallel data

slide-15
SLIDE 15

Type-level Projection

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern

slide-16
SLIDE 16

Type-level Projection

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern

DE–EN dict

Handel trade, deal hofft hope mit with

slide-17
SLIDE 17

Type-level Projection

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern trade, deal with

DE–EN dict

Handel trade, deal hofft hope mit with

slide-18
SLIDE 18

Type-level Projection

... a deal with China ... DET NOUN ADP NOUN ... trade with those regions ... NOUN ADP DET NOUN

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern

EN trees

trade, deal with

DE–EN dict

Handel trade, deal hofft hope mit with

slide-19
SLIDE 19

Type-level Projection

... a deal with China ... DET NOUN ADP NOUN ... trade with those regions ... NOUN ADP DET NOUN

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern

Feature: “Goodness in English” Value: high EN trees

trade, deal with

DE–EN dict

Handel trade, deal hofft hope mit with

slide-20
SLIDE 20

Type-level Projection

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern

DE–EN dict

Handel trade, deal hofft hope mit with

slide-21
SLIDE 21

Type-level Projection

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern hope

DE–EN dict

Handel trade, deal hofft hope mit with with

slide-22
SLIDE 22

Type-level Projection

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern hope

EN trees DE–EN dict no examples

Handel trade, deal hofft hope mit with with

slide-23
SLIDE 23

Type-level Projection

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern

Feature: “Goodness in English” Value: low

hope

EN trees DE–EN dict no examples

Handel trade, deal hofft hope mit with with

slide-24
SLIDE 24

Model

Edge-factored discriminative parser, based on MSTParser (McDonald et al. 2005) DELEX features: universal POS (McDonald et al. 2011) PROJ features: “Goodness in English”

slide-25
SLIDE 25

Model: DELEX Features

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

slide-26
SLIDE 26

Model: DELEX Features

ATTACH

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

slide-27
SLIDE 27

Model: DELEX Features

ATTACH

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

slide-28
SLIDE 28

Model: DELEX Features

ATTACH

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

Feature Value NOUN → PREP 1

slide-29
SLIDE 29

Model: DELEX Features

ATTACH

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

Feature Value NOUN → PREP 1

+ indicators including direction and distance

slide-30
SLIDE 30

Model: PROJ Features

ATTACH

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

slide-31
SLIDE 31

Feature Value Handel → mit 1

Model: PROJ Features

ATTACH

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

slide-32
SLIDE 32

Feature Value Handel → mit 1

Model: PROJ Features

ATTACH

Insufficient target-language supervised data to use NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

X

slide-33
SLIDE 33

Model: PROJ Features

ATTACH

Feature Value

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

Handel → mit

slide-34
SLIDE 34

Model: PROJ Features

ATTACH

Feature Value

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

Score(Handel → mit)

slide-35
SLIDE 35

Model: PROJ Features

ATTACH

Feature Value Query

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

Score(Handel → mit)

slide-36
SLIDE 36

Model: PROJ Features

ATTACH

Feature Value Query Goodness in English

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

Score(Handel → mit)

slide-37
SLIDE 37

Query Score Computation

→

Score(Handel

PREP) = log p(PREP|Handel)

slide-38
SLIDE 38

Query Score Computation

→

Score(Handel

PREP) = log p(PREP|Handel)

= log X

ws

p(PREP|ws) p(ws|Handel)

slide-39
SLIDE 39

Query Score Computation

trade 0.5 deal 0.5

p(ws|Handel) p(ws|

→

Score(Handel

PREP) = log p(PREP|Handel)

= log X

ws

p(PREP|ws) p(ws|Handel)

slide-40
SLIDE 40

Query Score Computation

... trade with those regions ... NOUN PREP DET NOUN ... a good trade ... DET ADJ NOUN

trade 0.5 deal 0.5

p(ws|Handel) p(ws|

EN trees

→

Score(Handel

PREP) = log p(PREP|Handel)

= log X

ws

p(PREP|ws) p(ws|Handel) ws = trade

slide-41
SLIDE 41

Query Score Computation

trade 0.5 deal 0.5

p(ws|Handel) p(ws|

→

Score(Handel

PREP) = log p(PREP|Handel)

= log X

ws

p(PREP|ws) p(ws|Handel)

PREP 0.5 ADJ 0.5

p(x|trade)

p(x|

ws = trade

slide-42
SLIDE 42

Feature Specialization

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

Feature: Goodness in English Value: Score(hofft → PREP) Feature: Goodness in English Value: Score(Handel → PREP)

slide-43
SLIDE 43

Feature Specialization

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

Feature: Goodness (VERB link) in English Value: Score(hofft → PREP) Feature: Goodness in English Value: Score(Handel → PREP)

slide-44
SLIDE 44

Feature Specialization

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

Feature: Goodness (VERB link) in English Value: Score(hofft → PREP) Feature: Goodness (NOUN link) in English Value: Score(Handel → PREP)

slide-45
SLIDE 45

Feature Specialization

NOUN VERB PREP NOUN PREP ADJ NOUN Mosambik hofft auf Handel mit anderen Mitgliedern Mozambique hope on trade with other members

Feature: Goodness (VERB link) in English Value: Score(hofft → PREP) Feature: Goodness (NOUN link) in English Value: Score(Handel → PREP)

Similar to selective sharing of Naseem et al. (2012)

slide-46
SLIDE 46

Experiments

Sources of bilingual dictionaries Training Results

slide-47
SLIDE 47

MANUAL Lexicon

slide-48
SLIDE 48

MANUAL Lexicon

slide-49
SLIDE 49

MANUAL Lexicon

deal 0.333 trade 0.333 trading 0.333

p(ws|Handel) p(ws|

slide-50
SLIDE 50

AUTOMATIC Lexicon

Da ist zum Beispiel der Handel mit Drogen Trafficking in drugs is one example Unkontrollierter Handel mit leichten Waffen Uncontrolled trade in light weapons Ich will mit dem Handel anfangen I will start with trade

slide-51
SLIDE 51

trade 0.66 trafficking 0.33

AUTOMATIC Lexicon

Da ist zum Beispiel der Handel mit Drogen Trafficking in drugs is one example Unkontrollierter Handel mit leichten Waffen Uncontrolled trade in light weapons Ich will mit dem Handel anfangen I will start with trade

p(ws|Handel)

p(ws|

slide-52
SLIDE 52

AUTOMATIC Lexicon

Da ist zum Beispiel der Handel mit Drogen Trafficking in drugs is one example Unkontrollierter Handel mit leichten Waffen Uncontrolled trade in light weapons Ich will mit dem Handel anfangen I will start with trade

trade 0.768 trafficking 0.057 commerce 0.05 trading 0.03 market 0.02 marketing 0.02 business 0.013 traffic 0.007 sales 0.002 deal 0.002 ... ...

p(ws|Handel)

p(ws|

slide-53
SLIDE 53

Small Target Training Set

8 target languages, CoNLL shared task treebanks Train on 400 trees from CoNLL training set Test on CoNLL test set

slide-54
SLIDE 54

Small Target Training Set

8 target languages, CoNLL shared task treebanks Train on 400 trees from CoNLL training set Test on CoNLL test set Universal POS tags generated from gold tags using mapping of Petrov et al. (2011)

slide-55
SLIDE 55

Small Target Training Set

8 target languages, CoNLL shared task treebanks Train on 400 trees from CoNLL training set Test on CoNLL test set Universal POS tags generated from gold tags using mapping of Petrov et al. (2011) Source corpus: parsed English Gigaword

slide-56
SLIDE 56

70 72 74 76 78 80 400 train trees UAS (8 languages averaged)

75.1

DELEX DELEX+PROJ(MANUAL) DELEX+PROJ(AUTOMATIC) M

  • D

Small Target Training Set

slide-57
SLIDE 57

70 72 74 76 78 80 400 train trees UAS (8 languages averaged)

75.1 76.7

+1.6

DELEX DELEX+PROJ(MANUAL) DELEX+PROJ(AUTOMATIC) M

  • D

Small Target Training Set

slide-58
SLIDE 58

70 72 74 76 78 80 400 train trees UAS (8 languages averaged)

75.1 77.4 76.7 +2.3

+1.6

DELEX DELEX+PROJ(MANUAL) DELEX+PROJ(AUTOMATIC) M

  • D

Small Target Training Set

slide-59
SLIDE 59

70 72 74 76 78 80 400 train trees UAS (8 languages averaged)

75.1 77.4 76.7

25 50 75 100 Open-class token coverage

89.9 60.8

+2.3 +1.6

DELEX DELEX+PROJ(MANUAL) DELEX+PROJ(AUTOMATIC) M

  • D

MANUAL AUTOMATIC

Small Target Training Set

slide-60
SLIDE 60

70 72 74 76 78 80 400 train trees UAS (8 languages averaged)

DELEX DELEX+PROJ M

  • D

75.1 77.4

+2.3

Small Target Training Set

slide-61
SLIDE 61

70 72 74 76 78 80 100 train trees 200 target trees 400 train trees UAS (8 languages averaged)

71.8 73.5 73.6 75.7 75.1 77.4

+2.3 +2.1 +1.7

DELEX DELEX+PROJ M

  • D

Small Target Training Set

slide-62
SLIDE 62

Train Test

DE CoNLL test set

DET NOUN VERB ADJ

ES trees PT trees

NOUN VERB NOUN PREP DET NOUN

SV trees

PRON VERB PRON ADV El amarillismo es imparcial Bruxelas adia fiscalidade de os combustíveis Jag tycker det inte

[McDonald et al. (2011)]

...

No Target Training Set

slide-63
SLIDE 63

Train Test

DE CoNLL test set

DET NOUN VERB ADJ

ES trees PT trees

NOUN VERB NOUN PREP DET NOUN

SV trees

PRON VERB PRON ADV

[McDonald et al. (2011)]

...

No Target Training Set

slide-64
SLIDE 64

Train Test

DE CoNLL test set

DET NOUN VERB ADJ

ES trees PT trees

NOUN VERB NOUN PREP DET NOUN

SV trees

PRON VERB PRON ADV El amarillismo es imparcial Bruxelas adia fiscalidade de os combustíveis Jag tycker det inte

[McDonald et al. (2011)]

...

No Target Training Set

slide-65
SLIDE 65

55 57 59 61 63 65 UAS (8 languages averaged)

60.6 62.6

+2.0

DELEX DELEX+PROJ M

  • D

No Target Training Set

slide-66
SLIDE 66

55 57 59 61 63 65 UAS (8 languages averaged)

60.6 62.6

+2.0

Type-level* (this work)

DELEX DELEX+PROJ M

  • D

No Target Training Set

slide-67
SLIDE 67

55 57 59 61 63 65 UAS (8 languages averaged)

60.6 62.6 61.1 63.8

Token-level (McDonald et al. 2011)

+2.7 +2.0

Type-level* (this work)

DELEX DELEX+PROJ M

  • D

D +P MULTIDIR MULTIPROJ

No Target Training Set

slide-68
SLIDE 68

Gains

2 4 6 8 10

DA DE EL ES IT NL PT SV

Gain over delexicalized baseline

DELEX+PROJ MULTIPROJ

slide-69
SLIDE 69

Related Work

Täckström et al. (2012): type-level transfer using word clusters

slide-70
SLIDE 70

Related Work

Täckström et al. (2012): type-level transfer using word clusters Naseem et al. (2012): adaptation of universal POS transfer to the target language

slide-71
SLIDE 71

Conclusion

Bilingual dictionaries allow transfer of linguistic knowledge at a type level

slide-72
SLIDE 72

Conclusion

Bilingual dictionaries allow transfer of linguistic knowledge at a type level Type-level transfer: Comparable to token-level transfer Simpler to implement Does not require bitext

slide-73
SLIDE 73

Conclusion

Bilingual dictionaries allow transfer of linguistic knowledge at a type level

Thank you!

Type-level transfer: Comparable to token-level transfer Simpler to implement Does not require bitext