Resources for Adding Semantics to Machine Translation Jan Haji - - PowerPoint PPT Presentation
Resources for Adding Semantics to Machine Translation Jan Haji Charles University in Prague Computer Science School Institute of Formal and Applied Linguistics Major contributions by: E: Silvie Cinkov, Jana indlerov, Josef Toman, (J.
Jan Hajič
Charles University in Prague Computer Science School Institute of Formal and Applied Linguistics
Major contributions by:
E: Silvie Cinková, Jana Šindlerová, Josef Toman, (J. Semecký) C: Marie Mikulová, Zdeňka Urešová, Jan Štěpánek
IWSLT
– Incl. the Prague (Czech-)English Dependency Treebank
– Annotation layers – From Penn Treebank+ to PDT-style English annotation – TR annotation of interesting English phenomena
– “Speech reconstruction”
IWSLT
– 2001: version 1.0 (no deep syntax/semantics) – 2006: version 2.0 (w/deep syntax, semantics: “tectogrammatics”)
– 2004: automatic annotation – English: PTB, Czech: 1/3rd of PTB translated
– 2004: ~ PDT 1.0 (no deep syntax)
IWSLT
+ PropBank + BBN (co-reference and Named Entities) + NP structure (D. Vadas, J. R. Curran, ACL’07) + “Czech-like” tectogrammatics
– Manual annotation (with auto pre-annotation)
IWSLT
(predicates)
Entities (BBN)
IWSLT
– tectogrammatics
– (surface) syntax
– Morphology (POS)
– words (tokens)
IWSLT
– Form – POS Tag – Function label – (Structure)
– Lemma – Heads
IWSLT
– By J. Eisner + M. Čmejrek/J. Cuřín – 4000 rules (non-terminal based)
– Additional rules
– J. Robinson (1960s)
IWSLT
(board) (board) (the) (join) (will) (join) (join) (join) (NP (DT NN)) → NN (VP (VB NP)) → VB (VP (MD VP)) → VP (S (… VP …)) → VP
Rules:
IWSLT
– based on PTB functional tags:
– Ad-hoc rules (if functional tags missing) – Lemmatization (years → year)
IWSLT
(board) (board) (the) (join) (will) (join) (join) (join)
Penn Treebank structure (with heads added) PDT-like Analytic Representation
PRED.Fut PAT
PDT-like Tectogrammatic Representation (automatic pre-annotation)
IWSLT
IWSLT
We have not answered your question completely, for which we apologize.
IWSLT
IWSLT
IWSLT
except, with the exception of, excluding, (all/none) but, beyond, apart from, unless, bar, barring, besides
IWSLT
– Pre-annotated – Graphical
primary
– Text + TR – Czech translation
“propbanking”)
– During TR annotation – Propbank origins and examples
IWSLT
IWSLT
IWSLT
– ACT() PAT() LOC() ACT(.1) PAT(.4) LOC()
– ACT() DIR1(from.) ACT(.1) DIR1(z.[.2])
IWSLT
2007-2009:
(lower numbers)
phenomenon”
IWSLT
– Version 1.0 now (PTB license needed)
– Full version (parallel with Czech): early 2011
– Full WSJ portion of PTB (2312 files)
– 49208 sentences, 1253013 tokens
IWSLT
– morphology, syntax, tectogrammatical
– Automatic (many tools by many authors)
– Complete now, co-reference annotation finishing – Top-down
IWSLT
– Transcription, even if perfect, is hard to analyze – ~ “people [when spekaing] are ungrammatical” – ~ editing recorded dialogs for print
Transcript: [breath] i think I th - see Si I think in this picture …after speech reconstruction: I think I see Si in this picture.
IWSLT
– Linking words, free movement of words – Editing, inserting, deleting words – Manual/auto transcripts (simultaneously visible) – Listening (as in transcription)
IWSLT
– “Wizard-of-Oz” setting for recording – Topic: Reminiscing over photographs – Uses in the EU FP6 “Companions” project – English: 20h, Czech: 120h – Manual transcription – Double or triple SR annotated – Release: spring 2011
IWSLT
speech data:
– “Companions” corpus → PDT-like annotated
syntax, semantics, valency)
speech (“easy”)
PDTSE corpus
he is a member they’re [UN] yeah, the yankees member of the club
He is a member of the Club – they were the Yankees.
/ACT
/PAT
/RSTR
/ACT
/PAT
transcript audio “Reconstructed” POS, surface syntax, … Deep syntax / tectogrammatics
IWSLT
– (Family of) dependency-based treebanking project(s)
– ~ 1mil. words
– 4 interlinked layers of annotation
– Parallel corpus Cze <-> Eng -> Machine Translation
– Speech, transcription, speech reconstruction
IWSLT
– FP7 – Network “META-NET” – FP6-IST “Euromatrix”, Companions – FP7-IST “Euromatrix+”, “Faust” – LC536 (Center for Computational Linguistics) – GAČR 405/06/0589 (Speech and deep syntax) – MŠMT: MSM0021620838, ME838, ME09008