Creating Large-Scale Multilingual Cognate Tables Winston Wu and - - PowerPoint PPT Presentation

Creating Large-Scale Multilingual Cognate Tables

Winston Wu and David Yarowsky Center for Language and Speech Processing Johns Hopkins University

http://educationviews.org/wp-content/uploads/2013/06/world-bread-cognates-panis.jpg

Cognates and Cognate Chains

Data

• Panlex and Wiktionary

Cognate Table Construction

Initial cluster with unweighted edit distance Alignment to get lexical translation probabilities Cluster with weighted distance function

Clustering

azj: stol tat: ostal tat: tablis tuk: stol tuk: tablisa tur: tablo uig: ustel uzn: stol uzn: tablista

Bitext from Clusters

eng azj tat tuk tur uig uzn table stol stol stol table

• stal

ustel table tablo table tablis tablisa tablista

Alignment

t -> t 0.600 t -> d 0.098 t -> c 0.061 t -> r 0.057 t -> p 0.019 t -> s 0.017 t -> l 0.017 t -> n 0.015 l -> l 0.747 l -> r 0.048 l -> n 0.024 l -> t 0.019 l -> o 0.018 l -> d 0.016 l -> c 0.015 l -> a 0.015 h -> h 0.529 h -> u 0.150 h -> NULL 0.140 h -> l 0.048 h -> a 0.032 h -> j 0.019 h -> o 0.017 h -> k 0.015

ü s t e l

• s t o l

TAT UIG

Clustering Distance Function

• Language-pair-specific edit distance
• Intra-family edit distance
• Same backtranslation
• Same POS
• Same MeaningID

Cognate Tables

Experiments

• Hold out words
• Use MT to predict
• Single language pair and system combination
• Evaluate on 1-best, 10-best, MRR

Results: Romance

Results: Romance

Results: Turkic

Results: Turkic

Results: Romance

Results: Turkic

Conclusion

• Cluster-alignment-cluster process for

multilingual cognate table construction

• Experiments
• 1-best exact match accuracy is hard!
• Close languages tend to do better
• Data size matters
• Code and data at github.com/wswu/coglust