SMT PILOT PROJECT - Taiwanese Laws S.H.E.L10N Frances Chang - - PowerPoint PPT Presentation

SMT PILOT PROJECT - Taiwanese Laws

Frances Chang Clinton Lin Jenny Lowe Charlene Wang Valerie Yin

S.H.E.L10N

Overview

2

1.

BACKGROUND

3

WHO?

Law Firm, Agencies that need fast-speed translations for new laws

WHAT?

Bilingual and monolingual Taiwanese law documents from ZH-CHT > EN

INITIAL GOALS

4

QUALITY

Match WeastO’s Metrics (a QI score of 10 or below)

PRODUCTIVITY

50% Faster than HT

COST

50% Cheaper than HT

Training Process

5

2.

SYSTEMS TRAINED

SYSTEMS TRAINING TUNING TESTING BLEU SCORE #1

1 1 1 6.19

#2

1 4 4 27.63

#3

2 3 6 Failed

6

SYSTEMS TRAINED

SYSTEMS TRAINING TUNING TESTING BLEU SCORE #4

2 3 6 6.28

#5

1 4 7 Failed

#6

4 4 4 Failed

7

SYSTEMS TRAINED

SYSTEMS TRAINING TUNING TESTING BLEU SCORE #7

4 3 5 8.33

#8

10 4 4 32.21

#9

13 4 1 32.41

8

Results

9

3.

PRODUCTIVITY

SAMPLE OF 500 WORDS POST-EDITOR FOR MT HUMAN TRANSLATOR PEMT

1 hour

• HT
• 3.5 hours

REVIEW

1 hour 1 hour

PEMT + REVIEW

2 hours

• HT + REVIEW
• 4.5 hours

TIME VARIANCE

2.5 hours

TIME SAVED

41.7%

10

11

TASK RATE (PER WORD) SUBTOTAL HT

$0.12 $60

PEMT

$0.06 $30

REVIEW

$0.05 $25

HT+REVIEW

• $85

PEMT+REVIEW

• $55

COST SAVED

35.3% COST

12

WeastO Metrics

QI Rate = Total points of error weight /Total word count *1000 (per mill)

QUALITY - REVIEWER #1

13

QUALITY - REVIEWER #2

WeastO Metrics

Average of 22.68 and 15.12 = 19

Lesson Learned

14

4.

PROBLEMS ENCOUNTERED

» Original documents had wanky alignment » Source is short, target is long » Difficult topic to train without dictionary » System update takes time

15

FULL ENGINE RECOMMENDATION

» More source materials » Use CAT tools for alignment » Continue training by adding more documents and dictionaries » Use PDF » Training = Quantity > Quality » Testing = Quality > Quantity

16

TIMELINE AND COST

» Difficult to establish an exact timeline due to many factors. » Cost = $4,200+

17

YOU

QUESTIONS?

18

THANK