Translation Model Interpolation for Domain Adaptation in TectoMT - - PowerPoint PPT Presentation

▶
translation model interpolation for domain adaptation in
SMART_READER_LITE
LIVE PREVIEW

Translation Model Interpolation for Domain Adaptation in TectoMT - - PowerPoint PPT Presentation

Apr 23, 2023 269 likes •565 views

Rudolf Rosa , Ondej Duek, Michal Novk, Martin Popel {rosa,odusek,mnovak,popel}@ufal.mff.cuni.cz Translation Model Interpolation for Domain Adaptation in TectoMT Charles University in Prague Faculty of Mathematics and Physics Institute

slide-1
SLIDE 1

DMTW, Praha, 3 September 2015 Rudolf Rosa, Ondřej Dušek, Michal Novák, Martin Popel {rosa,odusek,mnovak,popel}@ufal.mff.cuni.cz

Translation Model Interpolation for Domain Adaptation in TectoMT

Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics

slide-2
SLIDE 2

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

2/29

Why domain adaptation?

 not to do MT as “Europarl science”

 reasonably good for e.g. economic news  not particularly good for IT helpdesk

slide-3
SLIDE 3

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

3/29

Why domain adaptation?

 not to do MT as “Europarl science”

 reasonably good for e.g. economic news  not particularly good for IT helpdesk

Jak pošlu přílohu mailem? How do I send an attachment by e-mail? Click on the paperclip icon and select the file. Klikněte na ikonu sponky a vyberte soubor.

slide-4
SLIDE 4

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

4/29

Why model interpolation?

 available parallel data

 out-of-domain (Europarl): 2 000 000 sentences  in-domain (IT helpdesk): 1 000 sentences

slide-5
SLIDE 5

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

5/29

Why model interpolation?

 available parallel data

 out-of-domain (Europarl): 2 000 000 sentences  in-domain (IT helpdesk): 1 000 sentences

 train on out+in concatenation

 effect of in negligible (overpowered by out)

slide-6
SLIDE 6

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

6/29

Why model interpolation?

 available parallel data

 out-of-domain (Europarl): 2 000 000 sentences  in-domain (IT helpdesk): 1 000 sentences

 train on out+in concatenation

 effect of in negligible (overpowered by out)

 train on in only

 too weak, low coverage, worse than out alone

slide-7
SLIDE 7

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

7/29

Why model interpolation?

 available parallel data

 out-of-domain (Europarl): 2 000 000 sentences  in-domain (IT helpdesk): 1 000 sentences

 train on out+in concatenation

 effect of in negligible (overpowered by out)

 train on in only

 too weak, low coverage, worse than out alone

 train 2 translation models, one on in, one on out

 final TM = (weighted) interpolation of inTM and outTM

slide-8
SLIDE 8

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

8/29

Why TectoMT (deep transfer MT)?

✔ better meaning preservation ✔ better generalization from small data ✗ limited support for TM interpolation ✗ no support for tuning of TM weights

slide-9
SLIDE 9

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

9/29

Why TectoMT (deep transfer MT)?

✔ better meaning preservation ✔ better generalization from small data ✗ limited support for TM interpolation – solved ✗ no support for tuning of TM weights – TODO

slide-10
SLIDE 10

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

10/29

Outline

 Motivation  TectoMT (deep transfer)  Translation model interpolation in TectoMT  Evaluation  Conclusion

slide-11
SLIDE 11

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

11/29

TectoMT deep transfer

Jak pošlu přílohu mailem?

slide-12
SLIDE 12

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

12/29

TectoMT deep transfer

#PersPron drop poslat v:fin příloha n:4 Jak adv mail n:7

Jak pošlu přílohu mailem?

slide-13
SLIDE 13

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

13/29

TectoMT deep transfer

#PersPron drop poslat v:fin příloha n:4 Jak adv mail n:7

Jak pošlu přílohu mailem?

slide-14
SLIDE 14

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

14/29

TectoMT deep transfer

#PersPron drop poslat v:fin příloha n:4 Jak adv mail n:7

Jak pošlu přílohu mailem?

send v:fin

slide-15
SLIDE 15

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

15/29

TectoMT deep transfer

#PersPron drop poslat v:fin příloha n:4 Jak adv mail n:7

Jak pošlu přílohu mailem?

send v:fin How adv

slide-16
SLIDE 16

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

16/29

TectoMT deep transfer

#PersPron drop poslat v:fin příloha n:4 Jak adv mail n:7

Jak pošlu přílohu mailem?

#PersPron n:subj send v:fin How adv

slide-17
SLIDE 17

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

17/29

TectoMT deep transfer

#PersPron drop poslat v:fin příloha n:4 Jak adv mail n:7

Jak pošlu přílohu mailem?

#PersPron n:subj send v:fin attachment n:obj How adv

slide-18
SLIDE 18

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

18/29

TectoMT deep transfer

#PersPron drop poslat v:fin příloha n:4 Jak adv mail n:7

Jak pošlu přílohu mailem?

#PersPron n:subj send v:fin attachment n:obj How adv e-mail n:by+X

slide-19
SLIDE 19

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

19/29

TectoMT deep transfer

#PersPron drop poslat v:fin příloha n:4 Jak adv mail n:7

Jak pošlu přílohu mailem? How do I send an attachment by e-mail?

#PersPron n:subj send v:fin attachment n:obj How adv e-mail n:by+X

slide-20
SLIDE 20

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

20/29

T-lemma and formeme transfer

poslat v:fin mail n:7 ?

slide-21
SLIDE 21

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

21/29

T-lemma and formeme transfer

poslat v:fin mail n:7 e-mail (n) 0.32

  • nline (adv) 0.25

address (n) 0.11

  • nline (adj)

0.05 ... n:by+X 0.23 n:with+X 0.22 adv 0.13 n:via+X 0.04 ...

slide-22
SLIDE 22

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

22/29

T-lemma and formeme transfer

poslat v:fin mail n:7 e-mail (n) 0.32

  • nline (adv) 0.25

address (n) 0.11

  • nline (adj)

0.05 ... n:by+X 0.23 n:with+X 0.22 adv 0.13 n:via+X 0.04 ... e-mail n:by+X

slide-23
SLIDE 23

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

23/29

Translation model interpolation

P(e-mail∣mail)= 2 3 MaxEnt mail(e-mail ,context )+ 1 3 Static(e-mail ,mail)

slide-24
SLIDE 24

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

24/29

Translation model interpolation

P(e-mail∣mail)= 2 3 MaxEnt mail(e-mail ,context )+ 1 3 Static(e-mail ,mail) P(e-mail∣mail)= 2 6 OutMaxEntmail (e-mail, context)+ 1 6 OutStatic(e-mail , mail) + 2 6 InMaxEnt mail(e-mail , context) + 1 6 InStatic (e-mail , mail)

 past:  present:

slide-25
SLIDE 25

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

25/29

Translation model interpolation

P(e-mail∣mail)=w1⋅ OutMaxEnt mail(e-mail ,context )+w2⋅OutStatic(e-mail , mail) +w3⋅InMaxEnt mail(e-mail ,context ) +w4⋅InStatic(e-mail , mail)

 future:  interpolation weights tuned on development set

slide-26
SLIDE 26

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

26/29

Translation model interpolation

P(e-mail∣mail)=w1⋅ OutMaxEnt mail(e-mail ,context )+w2⋅OutStatic(e-mail , mail) +w3⋅InMaxEnt mail(e-mail ,context ) +w4⋅InStatic(e-mail , mail)

 future:  interpolation weights tuned on development set  not available at the moment:

 tuning in TectoMT  development set

slide-27
SLIDE 27

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

27/29

Evaluation

EN→CS CS→EN EN→ES ES→EN EN→NL NL→EN 16 18 20 22 24 26 28 30 32 34 36 38 40

30.6 27.1 20.4 18.5 23.0 37.0 28.4 21.5 23.3 18.5 21.4 33.7 31.3 28.3 26.5 20.4 24.3 38.9

  • ut

in interpol

slide-28
SLIDE 28

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

28/29

Conclusion

 TectoMT deep transfer system

 IT helpdesk domain (1000 parallel sentences)  domain adaptation by TM interpolation  no tuning of interpolation weights  consistently positive results, up to +3 BLEU

 future work

 tuning of interpolation weights  other domain adaptation methods

slide-29
SLIDE 29

Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT

29/29

Thank you for your attention

http://ufal.mff.cuni.cz/rudolf-rosa/

Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics Translation Model Interpolation for Domain Adaptation in TectoMT Rudolf Rosa, Ondřej Dušek, Michal Novák, Martin Popel {rosa,odusek,mnovak,popel}@ufal.mff.cuni.cz