CS11-737: Multilingual Natural Language Processing Translation - - PowerPoint PPT Presentation

▶
cs11 737 multilingual natural language processing
SMART_READER_LITE
LIVE PREVIEW

CS11-737: Multilingual Natural Language Processing Translation - - PowerPoint PPT Presentation

Jul 03, 2023 347 likes •775 views

CS11-737: Multilingual Natural Language Processing Translation Yulia Tsvetkov Translation Mr. and Mrs. Dursley, who lived at El seor y la seora Dursley, que number 4 on Privet Drive, were proud vivan en el nmero 4 de Privet Drive,

slide-1
SLIDE 1

CS11-737: Multilingual Natural Language Processing

Yulia Tsvetkov

Translation

slide-2
SLIDE 2

Translation

  • Mr. and Mrs. Dursley, who lived at

number 4 on Privet Drive, were proud to say they were very normal, fortunately. El señor y la señora Dursley, que vivían en el número 4 de Privet Drive, estaban orgullosos de decir que eran muy normales, afortunadamente.

slide-3
SLIDE 3

Plan

  • The practice of translation
  • Machine translation (MT)
  • MT data sources
  • MT evaluation
slide-4
SLIDE 4

Translation is important and ubiquitous

slide-5
SLIDE 5
slide-6
SLIDE 6
slide-7
SLIDE 7

Why is it difficult to translate?

  • Lexical ambiguities and divergences across languages

[Examples from Jurafsky & Martin Speech and Language Processing 2nd ed.]

slide-8
SLIDE 8

Why is it difficult to translate?

  • Cross-lingual lexical and structural divergences

錨玉自在枕上感念寶釵。。。又聽見窗外竹梢焦葉之上, 雨聲漸沂, 清寒透幕, 不党又滴下淚來 。 dai yu zi zai zhen shang gan nian bao chai...you ting jian chuang wal zhu shao xiang ye zhe shang, yu sheng xili, qing han tou mu, bu jue you di xia lei lat From “Dream of the Red Chamber” Cao Xue Qin (1792)

slide-9
SLIDE 9

Why is it difficult to translate?

[Example from Jurafsky & Martin Speech and Language Processing 2nd ed.]

slide-10
SLIDE 10

Why is it difficult to translate?

  • Ambiguities

○ words ○ morphology ○ semantics ○ pragmatics

  • Gaps in data

○ availability of corpora ○ commonsense knowledge

  • +Understanding of context,

connotation, social norms, etc.

slide-11
SLIDE 11

3 Classical methods for MT

  • Direct
  • Transfer
  • Interlingua
slide-12
SLIDE 12

The Vauquois triangle (1968)

slide-13
SLIDE 13

Direct translation

  • Word-by-word dictionary translation
  • Rely on linguistic knowledge for simple reordering or morphological

processing

morphological analysis lexical transfer using bilingual dictionary local reordering morphological generation source language text target language text

slide-14
SLIDE 14

Direct MT dictionary entry

slide-15
SLIDE 15

Transfer approaches

  • Levels of transfer
slide-16
SLIDE 16

Transfer approaches

  • Syntactic transfer
slide-17
SLIDE 17

Transfer approaches

  • Syntactic transfer
slide-18
SLIDE 18

Transfer approaches

  • Syntactic transfer
slide-19
SLIDE 19

Transfer approaches

  • Semantic transfer
slide-20
SLIDE 20

Transfer approaches

  • Semantic transfer
slide-21
SLIDE 21

Transfer approaches

slide-22
SLIDE 22

Interlingua

slide-23
SLIDE 23
slide-24
SLIDE 24
slide-25
SLIDE 25

Learning from data

slide-26
SLIDE 26

Parallel corpora

slide-27
SLIDE 27

Parallel corpora

slide-28
SLIDE 28

Parallel corpora

slide-29
SLIDE 29

Parallel corpora

slide-30
SLIDE 30

Parallel corpora

Mining parallel data from microblogs Ling et al. 2013

slide-31
SLIDE 31
  • pus.nlpl.eu
slide-32
SLIDE 32

Is it a good translation?

slide-33
SLIDE 33

MT evaluation is hard

  • MT Evaluation is a research topic on its own
  • Language variability: there is no single correct translation

○ Is system A better than system B?

  • Human evaluation is subjective
slide-34
SLIDE 34

Human evaluation

  • Adequacy and Fluency

○ Usually on a Likert scale (1 “not adequate at all” to 5 “completely adequate”)

slide-35
SLIDE 35

Human evaluation

  • Ranking of the outputs of different systems at the system level
slide-36
SLIDE 36

Human evaluation

  • Adequacy and Fluency

○ Usually on a Likert scale (1 “not adequate at all” to 5 “completely adequate”)

  • Ranking of the outputs of different systems at the system level
  • Post editing effort: how much effort does it take for a translator (or even

monolingual) to “fix” the MT output so it is “good”

  • Task-based evaluation: was the performance of the MT system sufficient to

perform a task.

slide-37
SLIDE 37

Automatic evaluation

  • Precision-based

○ BLEU, NIST, ...

  • F-score-based

○ Meteor,...

  • Error rates

○ WER, TER, PER,...

  • Using syntax/semantics

○ PosBleu, Meant, DepRef,...

  • Embedding based

○ BertScore, chrF, YISI-1, ESIM, ...

slide-38
SLIDE 38

Automatic evaluation

  • The BLEU score proposed by IBM (Papineni et al., 2002)

○ Count n-grams overlap between machine translation output and reference reference translations ○ Compute precision for ngrams of size 1 to 4 ○ No recall (because difficult with multiple references) ○ To compensate for recall: “brevity penalty”. Translations that are too short are penalized ○ Final score is the geometric average of the n-gram precisions, times the brevity penalty ○ Calculate the aggregate score over a large test set

slide-39
SLIDE 39

BLEU vs. human judgments

slide-40
SLIDE 40

Automatic evaluation

  • Embedding based

○ BertScore, chrF, YISI-1, ESIM, ...

slide-41
SLIDE 41

MT venues and competitions

  • MT tracks in *CL conferences
  • WMT, IWSLT, AMTA...
  • www.statmt.org
slide-42
SLIDE 42

Class discussion

  • Pick a 4-line excerpt from a poem in English
  • Use Google translate to back-translate the poem via a pivot language, e.g.,

○ English → Spanish → English ○ English → L1 → L2 → English, where L1 and L2 are typologically different from English and from each other

  • Compare the original poem and its English back-translation, and share your
  • bservations. For example,

○ what information got lost in the process of translation? ○ Are there translation errors associated with linguistic properties of pivot languages and with linguistic divergences across languages? ○ Try different pivot languages: can you provide insights about the quality

  • f MT for those language pairs?