lexical translation models 1
play

Lexical Translation Models 1 January 24, 2013 Thursday, January - PowerPoint PPT Presentation

Jan 12, 2023 •197 likes •1.23k views

Lexical Translation Models 1 January 24, 2013 Thursday, January 24, 13 Lexical Translation How do we translate a word? Look it up in the dictionary Haus : house, home, shell, household Multiple translations Different word senses,

  1. Example 0 1 2 3 4 das Haus ist klein NULL 1 3 4 2 Start with a foreign sentence and a target length. Thursday, January 24, 13

  2. Example 0 1 2 3 4 das Haus ist klein NULL 1 3 4 2 Thursday, January 24, 13

  3. Example 0 1 2 3 4 das Haus ist klein NULL 1 3 4 2 Thursday, January 24, 13

  4. Example 0 1 2 3 4 das Haus ist klein NULL the 1 3 4 2 Thursday, January 24, 13

  5. Example 0 1 2 3 4 das Haus ist klein NULL the 1 3 4 2 Thursday, January 24, 13

  6. Example 0 1 2 3 4 das Haus ist klein NULL the house 1 3 4 2 Thursday, January 24, 13

  7. Example 0 1 2 3 4 das Haus ist klein NULL the house 1 3 4 2 Thursday, January 24, 13

  8. Example 0 1 2 3 4 das Haus ist klein NULL the house is 1 3 4 2 Thursday, January 24, 13

  9. Example 0 1 2 3 4 das Haus ist klein NULL the house is 1 3 4 2 Thursday, January 24, 13

  10. Example 0 1 2 3 4 das Haus ist klein NULL the house is small 1 3 4 2 Thursday, January 24, 13

  11. Example 0 1 2 3 4 das Haus ist klein NULL 1 3 4 2 Thursday, January 24, 13

  12. Example 0 1 2 3 4 das Haus ist klein NULL 1 3 4 2 Thursday, January 24, 13

  13. Example 0 1 2 3 4 das Haus ist klein NULL the 1 3 4 2 Thursday, January 24, 13

  14. Example 0 1 2 3 4 das Haus ist klein NULL the 1 3 4 2 Thursday, January 24, 13

  15. Example 0 1 2 3 4 das Haus ist klein NULL house the 1 3 4 2 Thursday, January 24, 13

  16. Example 0 1 2 3 4 das Haus ist klein NULL house the 1 3 4 2 Thursday, January 24, 13

  17. Example 0 1 2 3 4 das Haus ist klein NULL house the is 1 3 4 2 Thursday, January 24, 13

  18. Example 0 1 2 3 4 das Haus ist klein NULL house the is 1 3 4 2 Thursday, January 24, 13

  19. Example 0 1 2 3 4 das Haus ist klein NULL house the is small 1 3 4 2 Thursday, January 24, 13

  20. Finding the Viterbi Alignment a ⇤ = arg a 2 [0 , 1 ,...,n ] m p ( a | e , f ) max p ( e , a | f ) = arg max a 0 p ( e , a 0 | f ) P a 2 [0 , 1 ,...,n ] m = arg a 2 [0 , 1 ,...,n ] m p ( e , a | f ) max 1 n a ∗ i = arg max 1 + np ( e i | f a i ) a i =0 n = arg max a i =0 p ( e i | f a i ) Thursday, January 24, 13

  21. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  22. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  23. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  24. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  25. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  26. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  27. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  28. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  29. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  30. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  31. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  32. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  33. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  34. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  35. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  36. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  37. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  38. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  39. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  40. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  41. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  42. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  43. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  44. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  45. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  46. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  47. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  48. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  49. Learning Lexical Translation Models • How do we learn the parameters p ( e | f ) • “Chicken and egg” problem • If we had the alignments, we could estimate the parameters (MLE) • If we had parameters, we could find the most likely alignments Thursday, January 24, 13

  50. EM Algorithm • pick some random (or uniform) parameters • Repeat until you get bored (~ 5 iterations for lexical translation models) • using your current parameters, compute “expected” alignments for every target word token in the training data (on board) p ( a i | e , f ) • keep track of the expected number of times f translates into e throughout the whole corpus • keep track of the expected number of times that f is used as the source of any translation • use these expected counts as if they were “real” counts in the standard MLE equation Thursday, January 24, 13

  51. EM for Model 1 Thursday, January 24, 13

  52. EM for Model 1 Thursday, January 24, 13

  53. EM for Model 1 Thursday, January 24, 13

  54. EM for Model 1 Thursday, January 24, 13

  55. EM for Model 1 Thursday, January 24, 13

  56. Convergence Thursday, January 24, 13

  57. Evaluation • Since we have a probabilistic model, we can evaluate perplexity . 1 ( e , f ) ∈ D | e | log Q ( e , f ) ∈ D p ( e | f ) − P PPL = 2 Iter ∞ Iter 1 Iter 2 Iter 3 Iter 4 ... -log likelihood - 7.66 7.21 6.84 ... -6 perplexity - 2.42 2.30 2.21 ... 2 Thursday, January 24, 13

  58. Alignment Error Rate Thursday, January 24, 13

  59. Alignment Error Rate Possible links P Thursday, January 24, 13

  60. Alignment Error Rate Possible links P Thursday, January 24, 13

  61. Alignment Error Rate Possible links Sure links S P Thursday, January 24, 13

  62. Alignment Error Rate Possible links Sure links S P Thursday, January 24, 13

  63. Alignment Error Rate Possible links Sure links S P Precision( A, P ) = | P ∩ A | | A | Thursday, January 24, 13

  64. Alignment Error Rate Possible links Sure links S P Precision( A, P ) = | P ∩ A | Recall( A, S ) = | S ∩ A | | A | | S | Thursday, January 24, 13

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Chapter 4 Word-based models Statistical Machine Translation Lexical Translation How to

Chapter 4 Word-based models Statistical Machine Translation Lexical Translation How to

Chapter 4 Word-based models Statistical Machine Translation Lexical Translation How to translate a word look up in dictionary Haus house, building, home, household, shell. Multiple translations some more frequent than others

1.2k views • 61 slides
Three models for discriminative machine Three models for discriminative machine translation using

Three models for discriminative machine Three models for discriminative machine translation using

Three models for discriminative machine Three models for discriminative machine translation using Global Lexical Selection translation using Global Lexical Selection and Sentence Reconstruction and Sentence Reconstruction Sriram Venkatapathy

645 views • 32 slides
Using Word Embeddings to Enforce Document-Level Lexical Consistency in Machine Translation Eva

Using Word Embeddings to Enforce Document-Level Lexical Consistency in Machine Translation Eva

Using Word Embeddings to Enforce Document-Level Lexical Consistency in Machine Translation Eva Martnez Garcia Carles Creus Cristina Espaa-Bonet Llus Mrquez EAMT 2017 May 30th Prague Outline Motivation 1 Lexical Consistency

808 views • 47 slides
Lexical Syntax for Dependency-based Language Models Statistical Machine Translation Incremental

Lexical Syntax for Dependency-based Language Models Statistical Machine Translation Incremental

Hany Hassan Introduction Syntax for Phrase-based SMT Supertagged Phrase-based SMT From Supertagged to Lexical Syntax for Dependency-based Language Models Statistical Machine Translation Incremental Dependency-based Language Model (IDLM)

804 views • 67 slides
4CSLL5 IBM Translation Models Martin Emms October 22, 2020 4CSLL5 IBM Translation Models IBM

4CSLL5 IBM Translation Models Martin Emms October 22, 2020 4CSLL5 IBM Translation Models IBM

4CSLL5 IBM Translation Models 4CSLL5 IBM Translation Models Martin Emms October 22, 2020 4CSLL5 IBM Translation Models IBM models Probabilities and Translation Alignments IBM Model 1 definitions 4CSLL5 IBM Translation Models IBM models

1.23k views • 103 slides
4CSLL5 IBM Translation Models IBM models Probabilities and Translation Alignments Martin Emms

4CSLL5 IBM Translation Models IBM models Probabilities and Translation Alignments Martin Emms

4CSLL5 IBM Translation Models 4CSLL5 IBM Translation Models 4CSLL5 IBM Translation Models IBM models Probabilities and Translation Alignments Martin Emms IBM Model 1 definitions October 22, 2020 4CSLL5 IBM Translation Models 4CSLL5 IBM

571 views • 7 slides
4CSLL5 IBM Translation Models Martin Emms October 23, 2020 4CSLL5 IBM Translation Models

4CSLL5 IBM Translation Models Martin Emms October 23, 2020 4CSLL5 IBM Translation Models

4CSLL5 IBM Translation Models 4CSLL5 IBM Translation Models Martin Emms October 23, 2020 4CSLL5 IBM Translation Models Parameter learning (brute force) Introduction The brute force EM algorithm defined A formula for p ( a | o , s ) Examples

759 views • 50 slides
4CSLL5 IBM Translation Models Martin Emms October 29, 2020 4CSLL5 IBM Translation Models

4CSLL5 IBM Translation Models Martin Emms October 29, 2020 4CSLL5 IBM Translation Models

4CSLL5 IBM Translation Models 4CSLL5 IBM Translation Models Martin Emms October 29, 2020 4CSLL5 IBM Translation Models Parameter learning (efficient) How to sum alignments efficiently Efficient EM via p (( j , i ) a | o , s ) 4CSLL5 IBM

658 views • 52 slides
Latent Models: Sequence Models Beyond HMMs and Machine Translation Alignment CMSC 473/673 UMBC

Latent Models: Sequence Models Beyond HMMs and Machine Translation Alignment CMSC 473/673 UMBC

Latent Models: Sequence Models Beyond HMMs and Machine Translation Alignment CMSC 473/673 UMBC Outline Review: EM for HMMs Machine Translation Alignment Limited Sequence Models Maximum Entropy Markov Models Conditional Random Fields

1.37k views • 100 slides
Latent Models: Sequence Models Beyond HMMs and Machine Translation Alignment CMSC 473/673 UMBC

Latent Models: Sequence Models Beyond HMMs and Machine Translation Alignment CMSC 473/673 UMBC

Latent Models: Sequence Models Beyond HMMs and Machine Translation Alignment CMSC 473/673 UMBC Outline Review: EM for HMMs Machine Translation Alignment Limited Sequence Models Maximum Entropy Markov Models Conditional Random Fields

1.2k views • 101 slides
Cross-Cutting Models of Lexical Semantics Joseph Reisinger and Raymond Mooney Distributional

Cross-Cutting Models of Lexical Semantics Joseph Reisinger and Raymond Mooney Distributional

Cross-Cutting Models of Lexical Semantics Joseph Reisinger and Raymond Mooney Distributional Lexical Semantics Represent meaning as a point/vector in a high-dimensional space Word relatedness correlates with some distance metric

804 views • 32 slides
Heterogeneous Lexical Resources MultiJEDI ERC 259234 Lexical Resource Lexical Resource Lexical

Heterogeneous Lexical Resources MultiJEDI ERC 259234 Lexical Resource Lexical Resource Lexical

A Robust Approach to Aligning Heterogeneous Lexical Resources MultiJEDI ERC 259234 Lexical Resource Lexical Resource Lexical Resource Why combine resources? e.g., named entities, new senses dozens of new languages

1.18k views • 64 slides
Chapter 11 Tree-based models Statistical Machine Translation Tree-Based Models Traditional

Chapter 11 Tree-based models Statistical Machine Translation Tree-Based Models Traditional

Chapter 11 Tree-based models Statistical Machine Translation Tree-Based Models Traditional statistical models operate on sequences of words Many translation problems can be best explained by pointing to syntax reordering, e.g., verb

1.39k views • 120 slides
Rule-based approach: attempts to program grammatical and lexical rules. Largely failed and

Rule-based approach: attempts to program grammatical and lexical rules. Largely failed and

A brief history of Machine Translation (MT) http://blog.systransoft.com/how-does-neural-machine-translation-work/ Rule-based approach: attempts to program grammatical and lexical rules. Largely failed and showed how limited grammar is in

207 views • 10 slides
Lexical Databases Like a dictionary Lexical properties of interest to psycholinguists

Lexical Databases Like a dictionary Lexical properties of interest to psycholinguists

A General Introduction to Lexical Databases Emmanuel Keuleers Department of Experimental Psychology Ghent University EMLAR 2015 - Utrecht, April 15-17, 2015 What can you find in a lexical database? How can you find it? Lexical

905 views • 45 slides
Data-driven sense induction for disambiguation and lexical selection in translation Marianna

Data-driven sense induction for disambiguation and lexical selection in translation Marianna

Data-driven sense induction for disambiguation and lexical selection in translation Marianna Apidianaki, University Paris 7 22 October 2008 Plan of the presentation a. Towards data-driven sense acquisition and Word Sense Disambiguation (WSD)

808 views • 47 slides
Worm enabling exploits Cyber Security Lab Spring 10 Background reading Worm Anatomy and

Worm enabling exploits Cyber Security Lab Spring 10 Background reading Worm Anatomy and

Worm enabling exploits Cyber Security Lab Spring 10 Background reading Worm Anatomy and Model http://portal.acm.org/citation.cfm?id=948196 Smashing the Stack for Fun and Profit

525 views • 36 slides
Algorithms R OBERT S EDGEWICK | K EVIN W AYNE 5.2 T RIES R-way tries ternary search tries

Algorithms R OBERT S EDGEWICK | K EVIN W AYNE 5.2 T RIES R-way tries ternary search tries

Algorithms R OBERT S EDGEWICK | K EVIN W AYNE 5.2 T RIES R-way tries ternary search tries character-based operations Algorithms F O U R T H E D I T I O N R OBERT S EDGEWICK | K EVIN W AYNE http://algs4.cs.princeton.edu Summary of

986 views • 51 slides
What is an Algorithm? Quite generally, an algorithm is a set of instructions that lead to

What is an Algorithm? Quite generally, an algorithm is a set of instructions that lead to

What is an Algorithm? Quite generally, an algorithm is a set of instructions that lead to some desired Fundamentals: All About result Algorithms Must first decompose a problem, and then express its method of solution as a Computer

407 views • 7 slides
Foundations of Network and Foundations of Network and Computer Security Computer Security J ohn

Foundations of Network and Foundations of Network and Computer Security Computer Security J ohn

Foundations of Network and Foundations of Network and Computer Security Computer Security J ohn Black J Lecture #18 Oct 28 th 2004 CSCI 6268/TLEN 5831, Fall 2004 Announcements Quiz #3 Thurs, Nov 4 th A week from today How to

695 views • 50 slides
The Sources of Certainty in Computation and Formal Systems Michael J. ODonnell The University

The Sources of Certainty in Computation and Formal Systems Michael J. ODonnell The University

The Sources of Certainty in Computation and Formal Systems Michael J. ODonnell The University of Chicago (revised 2010 November 16) computation = derivation in formal system Familiarity makes a difference 1 computational is a synonym

718 views • 51 slides
Protec'on, iden'ty, and trust Illustrated with Unix setuid

Protec'on, iden'ty, and trust Illustrated with Unix setuid

Protec'on, iden'ty, and trust Illustrated with Unix setuid Jeff Chase Duke University The need for access control Processes run programs on behalf of users. ( subjects )

1.02k views • 70 slides
Experiences with High Performance Intrusion Detection in the HPC Environment Cray Users Group

Experiences with High Performance Intrusion Detection in the HPC Environment Cray Users Group

Experiences with High Performance Intrusion Detection in the HPC Environment Cray Users Group 2011 Presentation Jim Mellander Scott Campbell NERSC Agenda NERSC Description Operational Philosophy Science-Driven Cybersecurity

597 views • 23 slides
Mechanical Properties of Paint Coatings Hardness Measurement Shore Hardness D Barcol Hardness

Mechanical Properties of Paint Coatings Hardness Measurement Shore Hardness D Barcol Hardness

Mechanical Properties of Paint Coatings Hardness Measurement Shore Hardness D Barcol Hardness ASTM D2583 Fiberglass Aluminum Aluminum Alloys Soft Metals Plastics Scratch Hardness- (IS 101 - Part 5/Sec 2) BS 3900 Hardness can

1.89k views • 30 slides

More recommend