translation model interpolation for domain adaptation in
play

Translation Model Interpolation for Domain Adaptation in TectoMT - PowerPoint PPT Presentation

Apr 23, 2023 •269 likes •561 views

Rudolf Rosa , Ondej Duek, Michal Novk, Martin Popel {rosa,odusek,mnovak,popel}@ufal.mff.cuni.cz Translation Model Interpolation for Domain Adaptation in TectoMT Charles University in Prague Faculty of Mathematics and Physics Institute

  1. Rudolf Rosa , Ondřej Dušek, Michal Novák, Martin Popel {rosa,odusek,mnovak,popel}@ufal.mff.cuni.cz Translation Model Interpolation for Domain Adaptation in TectoMT Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics DMTW, Praha, 3 September 2015

  2. Why domain adaptation?  not to do MT as “Europarl science”  reasonably good for e.g. economic news  not particularly good for IT helpdesk Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 2/29

  3. Why domain adaptation?  not to do MT as “Europarl science”  reasonably good for e.g. economic news  not particularly good for IT helpdesk Jak pošlu přílohu mailem? How do I send an attachment by e-mail? Click on the paperclip icon and select the file. Klikněte na ikonu sponky a vyberte soubor. Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 3/29

  4. Why model interpolation?  available parallel data  out-of-domain (Europarl): 2 000 000 sentences  in-domain (IT helpdesk): 1 000 sentences Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 4/29

  5. Why model interpolation?  available parallel data  out-of-domain (Europarl): 2 000 000 sentences  in-domain (IT helpdesk): 1 000 sentences  train on out + in concatenation  effect of in negligible (overpowered by out ) Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 5/29

  6. Why model interpolation?  available parallel data  out-of-domain (Europarl): 2 000 000 sentences  in-domain (IT helpdesk): 1 000 sentences  train on out + in concatenation  effect of in negligible (overpowered by out )  train on in only  too weak, low coverage, worse than out alone Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 6/29

  7. Why model interpolation?  available parallel data  out-of-domain (Europarl): 2 000 000 sentences  in-domain (IT helpdesk): 1 000 sentences  train on out + in concatenation  effect of in negligible (overpowered by out )  train on in only  too weak, low coverage, worse than out alone  train 2 translation models, one on in , one on out  final TM = (weighted) interpolation of in TM and out TM Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 7/29

  8. Why TectoMT (deep transfer MT)? ✔ better meaning preservation ✔ better generalization from small data ✗ limited support for TM interpolation ✗ no support for tuning of TM weights Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 8/29

  9. Why TectoMT (deep transfer MT)? ✔ better meaning preservation ✔ better generalization from small data ✗ limited support for TM interpolation – solved ✗ no support for tuning of TM weights – TODO Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 9/29

  10. Outline  Motivation  TectoMT (deep transfer)  Translation model interpolation in TectoMT  Evaluation  Conclusion Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 10/29

  11. TectoMT deep transfer Jak pošlu přílohu mailem? Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 11/29

  12. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 12/29

  13. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 13/29

  14. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 send v:fin Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 14/29

  15. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 send v:fin How adv Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 15/29

  16. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 send v:fin How #PersPron adv n:subj Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 16/29

  17. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 send v:fin How #PersPron attachment adv n:subj n:obj Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 17/29

  18. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 send v:fin How #PersPron attachment e-mail adv n:subj n:obj n:by+X Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 18/29

  19. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 send v:fin How #PersPron attachment e-mail adv n:subj n:obj n:by+X How do I send an attachment by e-mail? Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 19/29

  20. T-lemma and formeme transfer poslat v:fin mail ? n:7 Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 20/29

  21. T-lemma and formeme transfer e-mail (n) 0.32 online (adv) 0.25 poslat address (n) 0.11 v:fin online (adj) 0.05 ... mail n:7 n:by+X 0.23 n:with+X 0.22 adv 0.13 n:via+X 0.04 ... Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 21/29

  22. T-lemma and formeme transfer e-mail (n) 0.32 online (adv) 0.25 poslat address (n) 0.11 v:fin online (adj) 0.05 ... mail e-mail n:7 n:by+X n:by+X 0.23 n:with+X 0.22 adv 0.13 n:via+X 0.04 ... Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 22/29

  23. Translation model interpolation P ( e-mail ∣ mail )= 2 3 MaxEnt mail ( e-mail ,context )+ 1 3 Static ( e-mail , mail ) Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 23/29

  24. Translation model interpolation  past: P ( e-mail ∣ mail )= 2 3 MaxEnt mail ( e-mail ,context )+ 1 3 Static ( e-mail , mail )  present: P ( e-mail ∣ mail )= 2 6 OutMaxEnt mail ( e-mail , context )+ 1 6 OutStatic ( e-mail , mail ) + 2 6 InMaxEnt mail ( e-mail , context ) + 1 6 InStatic ( e-mail , mail ) Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 24/29

  25. Translation model interpolation  future: P ( e-mail ∣ mail )= w 1 ⋅ OutMaxEnt mail ( e-mail ,context )+ w 2 ⋅ OutStatic ( e-mail , mail ) + w 3 ⋅ InMaxEnt mail ( e-mail ,context ) + w 4 ⋅ InStatic ( e-mail , mail )  interpolation weights tuned on development set Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 25/29

  26. Translation model interpolation  future: P ( e-mail ∣ mail )= w 1 ⋅ OutMaxEnt mail ( e-mail ,context )+ w 2 ⋅ OutStatic ( e-mail , mail ) + w 3 ⋅ InMaxEnt mail ( e-mail ,context ) + w 4 ⋅ InStatic ( e-mail , mail )  interpolation weights tuned on development set  not available at the moment:  tuning in TectoMT  development set Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 26/29

  27. Evaluation 40 38.9 out 38 37.0 in 36 interpol 33.7 34 32 31.3 30.6 30 28.4 28.3 28 27.1 26.5 26 24.3 24 23.3 23.0 21.5 22 21.4 20.4 20.4 20 18.5 18.5 18 16 EN→CS CS→EN EN→ES ES→EN EN→NL NL→EN Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 27/29

  28. Conclusion  TectoMT deep transfer system  IT helpdesk domain (1000 parallel sentences)  domain adaptation by TM interpolation  no tuning of interpolation weights  consistently positive results, up to +3 BLEU  future work  tuning of interpolation weights  other domain adaptation methods Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 28/29

  29. Thank you for your attention Rudolf Rosa , Ondřej Dušek, Michal Novák, Martin Popel {rosa,odusek,mnovak,popel}@ufal.mff.cuni.cz Translation Model Interpolation for Domain Adaptation in TectoMT Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics http://ufal.mff.cuni.cz/rudolf-rosa/ Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 29/29

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Translation Model Adaptation Using Genre-Revealing Text Features Marlies van der Wees, Arianna

Translation Model Adaptation Using Genre-Revealing Text Features Marlies van der Wees, Arianna

Translation Model Adaptation Using Genre-Revealing Text Features Marlies van der Wees, Arianna Bisazza, Christof Monz Domain adaptation for SMT Prioritize translation candidates that are most relevant to a specific task Translation Model

758 views • 74 slides
Adaptation Philipp Koehn 27 October 2020 Philipp Koehn Machine Translation: Adaptation 27

Adaptation Philipp Koehn 27 October 2020 Philipp Koehn Machine Translation: Adaptation 27

Adaptation Philipp Koehn 27 October 2020 Philipp Koehn Machine Translation: Adaptation 27 October 2020 Adaptation 1 Better quality when system is adapted to a task Domain adaptation to a specific domain, e.g., information technology

962 views • 46 slides
Domain Adaptation in Statistical Machine Translation Logic, Language and Computation Bart

Domain Adaptation in Statistical Machine Translation Logic, Language and Computation Bart

Domain Adaptation in Statistical Machine Translation Logic, Language and Computation Bart Mellebeek October 28, 2013 Bart Mellebeek Domain Adaptation in Statistical Machine Translation Outline Machine Translation and AI Statistical Machine

889 views • 54 slides
What is domain adaptation?

What is domain adaptation?

A Two-Stage Approach to Domain Adaptation for Statistical Classifiers Jing Jiang & ChengXiang Zhai Department of Computer Science University of Illinois at Urbana-Champaign What is domain adaptation?

523 views • 20 slides
Domain adaptation model for retinopathy detection from cross-domain OCT images Jing Wang 1;2 ,

Domain adaptation model for retinopathy detection from cross-domain OCT images Jing Wang 1;2 ,

Domain adaptation model for retinopathy detection from cross-domain OCT images Jing Wang 1;2 , Yiwei Chen 2 , Wanyue Li1; 2 , Wen Kong 1;2 , Yi He 2 , Chuihui Jiang *3 , Guohua Shi *2;4 1 University of Science and technology of China, Hefei 230026,

348 views • 9 slides
discrepancy for unsupervised domain adaptation Hongliang Yan 2017/06/21 Domain Adaptation DA

discrepancy for unsupervised domain adaptation Hongliang Yan 2017/06/21 Domain Adaptation DA

Mind the class weight bias: weighted maximum mean discrepancy for unsupervised domain adaptation Hongliang Yan 2017/06/21 Domain Adaptation DA Training Test Problem: (Source (Target) Training and test sets are related but under

495 views • 21 slides
Domain Adaptation with Asymmetrically Relaxed Distribution Alignment Yifan Wu , Ezra Winston,

Domain Adaptation with Asymmetrically Relaxed Distribution Alignment Yifan Wu , Ezra Winston,

Domain Adaptation with Asymmetrically Relaxed Distribution Alignment Yifan Wu , Ezra Winston, Divyansh Kaushik, Zachary Lipton Carnegie Mellon University ICML 2019 1 / 8 Background - Unsupervised Domain Adaptation Unsupervised Domain

332 views • 8 slides
Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation Kaichao You 1 , Ximei Wang

Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation Kaichao You 1 , Ximei Wang

Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation Kaichao You 1 , Ximei Wang 1 , Mingsheng Long 1 , Michael I. Jordan 2 1 School of Software, Tsinghua University 1 National Engineering Lab for Big Data Software 2 University of

324 views • 18 slides
Investigations on Translation Model Adaptation Using Monolingual Data Patrik Lambert, Holger

Investigations on Translation Model Adaptation Using Monolingual Data Patrik Lambert, Holger

Investigations on Translation Model Adaptation Using Monolingual Data Patrik Lambert, Holger Schwenk Christophe Servan and Sadaf Abdul-Rauf LIUM (Computing Laboratory) University of Le Mans France WMT 2011 Lambert, Schwenk, Servan, Abdul-Rauf

485 views • 30 slides
Domain Adaptation from a Pre-trained Source Model Application on fraud detection tasks Presenter:

Domain Adaptation from a Pre-trained Source Model Application on fraud detection tasks Presenter:

Domain Adaptation from a Pre-trained Source Model Application on fraud detection tasks Presenter: Luxin Zhang (Worldline & Inria) Supervisors: Christophe Biernacki (Inria), Pascal Germain (Inria), Yacine Kessaci (Worldline) CMStatistics 2019

531 views • 17 slides
Implicit Class-Conditioned Domain Alignment for Unsupervised Domain Adaptation 1,2 1,4 Xiang

Implicit Class-Conditioned Domain Alignment for Unsupervised Domain Adaptation 1,2 1,4 Xiang

Implicit Class-Conditioned Domain Alignment for Unsupervised Domain Adaptation 1,2 1,4 Xiang Jiang Qicheng Lao 1,3 1 Stan Matwin Mohammad Havaei 1 Imagia 2 Dalhousie University 3 Polish Academy of Sciences 4 Mila, Universit e de Montr

1.02k views • 43 slides
Towards Assumption-free Unsupervised Domain Adaptation for Visual recognition

Towards Assumption-free Unsupervised Domain Adaptation for Visual recognition

Towards Assumption-free Unsupervised Domain Adaptation for Visual recognition Outline Background and Motivations Unsupervised Domain Adaptation - Estimating label information in

1.51k views • 47 slides
Interpolation CS3220 - Summer 2008 Jonathan Kaldor Interpolation Weve looked at the

Interpolation CS3220 - Summer 2008 Jonathan Kaldor Interpolation Weve looked at the

Interpolation CS3220 - Summer 2008 Jonathan Kaldor Interpolation Weve looked at the problem of model fitting using least squares: given some sample points, determine linear parameters for a model that best fits the data Note:

813 views • 63 slides
Instance Weighting for Domain Adaptation in NLP Jing Jiang & ChengXiang Zhai University of

Instance Weighting for Domain Adaptation in NLP Jing Jiang & ChengXiang Zhai University of

Instance Weighting for Domain Adaptation in NLP Jing Jiang & ChengXiang Zhai University of Illinois at Urbana-Champaign June 25, 2007 Domain Adaptation Many NLP tasks are cast into classification

582 views • 20 slides
Few-shot Domain Adaptation 1/12 by Causal Mechanism Transfer Domain adaptation Causal mechanism

Few-shot Domain Adaptation 1/12 by Causal Mechanism Transfer Domain adaptation Causal mechanism

Few-shot Domain Adaptation 1/12 by Causal Mechanism Transfer Domain adaptation Causal mechanism Q. When is it possible? A. Common causal Transfer assumption (TA)? mechanism as the relation. Summary Common data generating (causal) mechanism

1.03k views • 15 slides
Theoretical Analysis of Domain Adaptation Current state of the art Shai Ben-David September 14,

Theoretical Analysis of Domain Adaptation Current state of the art Shai Ben-David September 14,

Theoretical Analysis of Domain Adaptation Current state of the art Shai Ben-David September 14, 2012 Domain Adaptation Most of the statistical learning guarantees are based on assuming that the learning environment is unchanged throughout the

685 views • 64 slides
Nat Natio ional al Soil il Mois isture Map ap Steven M. Quiring, Ohio State University

Nat Natio ional al Soil il Mois isture Map ap Steven M. Quiring, Ohio State University

Nat Natio ional al Soil il Mois isture Map ap Steven M. Quiring, Ohio State University Quiring.10@osu.edu Developing National Soil Moisture Products to Improve Drought Monitoring (NOAA MAPP): 2017-2019 Developing Gridded Soil Moisture Maps

522 views • 26 slides
GEWEX WATER VAPOR ISSUES OLR too low in subtropics (upper atmosphere too moist) TOA LW CRE

GEWEX WATER VAPOR ISSUES OLR too low in subtropics (upper atmosphere too moist) TOA LW CRE

GEWEX WATER VAPOR ISSUES OLR too low in subtropics (upper atmosphere too moist) TOA LW CRE depends on profile of WV SST and LST retrieval sensitive to Qa (mostly), especially in low latitudes SRF netLW sensitive to Qa SRF E

836 views • 42 slides
Animation Wrinkling Augmenting Coarse Cloth Simulation with Realistic-Looking Wrinkles Damien

Animation Wrinkling Augmenting Coarse Cloth Simulation with Realistic-Looking Wrinkles Damien

Animation Wrinkling Augmenting Coarse Cloth Simulation with Realistic-Looking Wrinkles Damien Rohmer Grenoble Marie-Paule Cani University, INRIA Stefanie Hahmann ETH Zurich Tiberiu Popa University of Alla Sheffer British Columbia

735 views • 37 slides
A 0.5-to-1 V 9-bit 15-to-90 MS/s Digitally Interpolated Pipelined-SAR ADC Using Dynamic Amplifier

A 0.5-to-1 V 9-bit 15-to-90 MS/s Digitally Interpolated Pipelined-SAR ADC Using Dynamic Amplifier

A 0.5-to-1 V 9-bit 15-to-90 MS/s Digitally Interpolated Pipelined-SAR ADC Using Dynamic Amplifier James Lin, Zule Xu, Masaya Miyahara, and Akira Matsuzawa Tokyo Institute of Technology, Japan b. Matsuzawa & Okada Lab y Outline

328 views • 30 slides
Flanders Brussels Wallonia www.vmm.be www.ibgebim.be airclimat.wallonie.be Delta-tool a

Flanders Brussels Wallonia www.vmm.be www.ibgebim.be airclimat.wallonie.be Delta-tool a

Flanders Brussels Wallonia www.vmm.be www.ibgebim.be airclimat.wallonie.be Delta-tool a MS-experience FAIRMODE recommandations: use of models : use of models : population exposure spatial extent of exceedances combined use

487 views • 32 slides
NONLINEAR CONSTANT LIFE MODEL FOR FATIGUE LIFE PREDICTION OF COMPOSITE MATERIALS T. Park 1 , M.

NONLINEAR CONSTANT LIFE MODEL FOR FATIGUE LIFE PREDICTION OF COMPOSITE MATERIALS T. Park 1 , M.

18 TH INTERNATIONAL CONFERENCE ON COMPOSITE MATERIALS NONLINEAR CONSTANT LIFE MODEL FOR FATIGUE LIFE PREDICTION OF COMPOSITE MATERIALS T. Park 1 , M. Kim 1 , B. Jang 1 , J. Lee 2 , J. Park 3 * 1 Graduate School, Korea Aerospace University, Koyang,

183 views • 6 slides
In the name of Allah In the name of Allah the compassionate, the merciful Digital Video

In the name of Allah In the name of Allah the compassionate, the merciful Digital Video

In the name of Allah In the name of Allah the compassionate, the merciful Digital Video Processing S. Kasaei S. Kasaei Room: CE 307 Department of Computer Engineering Sharif University of Technology E-Mail: skasaei@sharif.edu Web Page:

679 views • 28 slides
Spatial Analysis on the Clouds: DEM Interpolation on the Microsoft Azure Platform Abdelmounaam

Spatial Analysis on the Clouds: DEM Interpolation on the Microsoft Azure Platform Abdelmounaam

Spatial Analysis on the Clouds: DEM Interpolation on the Microsoft Azure Platform Abdelmounaam Rezgui Center of Intelligent Spatial Computing for Water/Energy Science George Mason University Spatial Cloud Computing Developing and/or

501 views • 17 slides

More recommend