Synchronous Grammars Synchronous grammars are a way of - PowerPoint PPT Presentation

Synchronous Grammars

Synchronous grammars are a way of simultaneously generating pairs of recursively related strings Synchronous grammar w w ʹ

Synchronous grammars were originally invented for programming language compilation Synchronous grammar for i := 1 to 10 do mov ax, 1 begin loop: add bx, ax n := n + i cmp ax, 10 end jle loop

Synchronous grammars have been proposed as a way of doing semantic interpretation Synchronous grammar open ′ (me ′ , box ′ ) I open the box

Synchronous grammars have been used for syntax-based machine translation Synchronous grammar I open the box watashi wa hako wo akemasu

Synchronous grammars can do much fancier transformations than finite-state methods The boy stated that the student said that the teacher danced shoonen ga gakusei ga sensei ga odotta to itta to hanashita boy student teacher danced that said that stated

Synchronous grammars can do much fancier transformations than finite-state methods …that John saw Peter help the children swim …dat Jan Piet de kinderen zag helpen zwemmen John Peter the children saw help swim

Overview Definitions Properties Algorithms Extensions

Definitions

Context-free grammars S → NP VP S NP → I NP VP NP → the box I NP V VP → V NP open the box V → open

Context-free grammars S → NP VP S NP → watashi wa NP VP NP → hako wo watashi wa NP V VP → NP V V → akemasu hako wo akemasu

Synchronous CFGs S → NP 1 VP 2 S → NP 1 VP 2 NP → I NP → watashi wa NP → the box NP → hako wo VP → V 1 NP 2 VP → NP 2 V 1 V → open V → akemasu

Synchronous CFGs S → NP 1 VP 2 , NP 1 VP 2 S → NP 1 VP 2 , NP 1 VP 2 NP → I, watashi wa NP → I, watashi wa NP → the box, hako wo NP → the box, hako wo VP → V 1 NP 2, NP 2 V 1 VP → V 1 NP 2, NP 2 V 1 V → open, akemasu V → open, akemasu

Synchronous CFGs S 1 S 1 S 1 S 1 NP 2 NP 2 VP 3 VP 3 NP 2 NP 2 VP 3 VP 3 I NP 5 NP 5 watashi wa NP 5 V 4 V 4 V 4 V 4 NP 5 open the box hako wo akemasu

Adding probabilities 0.3 S → NP 1 VP 2 , NP 1 VP 2 0.1 NP → I, watashi wa 0.6 NP → the box, hako wo 0.5 VP → V 1 NP 2, NP 2 V 1 0.2 V → open, akemasu

Synchronous CFGs S 1 S 1 S 1 S 1 NP 2 NP 2 VP 3 VP 3 NP 2 NP 2 VP 3 VP 3 I NP 5 NP 5 watashi wa NP 5 V 4 V 4 V 4 V 4 NP 5 open the box hako wo akemasu Derivation probability: 0.3 × 0.1 × 0.5 × 0.6 × 0.2

Other notations Syntax directed translation VP → (V 1 NP 2 , NP 2 V 1 ) schema (Aho and Ullman; Lewis and Stearns) (VP → V 1 NP 2 , VP → NP 2 V 1 ) Inversion transduction VP → 〈 V NP 〉 grammar (Wu) [1,2] ( V NP ) VP → ⋈ Multitext grammar (Melamed) [2,1] V NP

Properties

Chomsky normal form X → Y Z X → a

Chomsky normal form A → (((B C) D) E) F rank 5

Chomsky normal form A → [[[B C] D] E] F rank 5 A → V1 F V1 → V2 E rank 2 V2 → V3 D V3 → B C

A hierarchy of synchronous CFGs 1-CFG ⊊ 2-CFG = 3-CFG = 4-CFG = … 1-SCFG ⊊ 2-SCFG = 3-SCFG ⊊ 4-SCFG ⊊ … = = ITG (Wu, 1997)

Synchronous CNF? A → (B 1 [C 2 D 3 ], [C 2 D 3 ] B 1 ) rank 3

Synchronous CNF? A → (B 1 [C 2 D 3 ], [C 2 D 3 ] B 1 ) rank 3 A → ( B 1 V1 2 , V1 2 B 1 ) rank 2 V1 → ( C 1 D 2 , C 1 D 2 )

Synchronous CNF? A → (B 1 C 2 D 3 E 4 , C 2 E 4 B 1 D 3 ) rank 4 A → ([B 1 C 2 ] D 3 E 4 , [C 2 E 4 B 1 ] D 3 ) A → (B 1 [C 2 D 3 ] E 4 , [C 2 E 4 B 1 D 3 ]) A → (B 1 C 2 [D 3 E 4 ], C 2 [E 4 B 1 D 3 ])

Synchronous CNF? 1 2 3 B 1 A → (B 1 C 2 D 3 , C 2 D 3 B 1 ) C 2 D 3 1 2 3 4 B 1 C 2 A → (B 1 C 2 D 3 E 4 , C 2 E 4 B 1 D 3 ) D 3 E 4

A hierarchy of synchronous CFGs 1-CFG ⊊ 2-CFG = 3-CFG = 4-CFG = … 1-SCFG ⊊ 2-SCFG = 3-SCFG ⊊ 4-SCFG ⊊ … = = ITG (Wu, 1997)

Algorithms

Overview Translation Bitext parsing

Review: CKY S → NP VP NP → I NP NP → the box VP → V NP V V → open I I open the box open

Review: CKY S → NP VP NP → I NP NP → the box VP → V NP NP V V → open I I open the box open the box

Review: CKY S → NP VP NP → I NP VP NP → the box VP → V NP NP NP V V V → open I open the box

Review: CKY S S → NP VP NP → I NP NP VP VP NP → the box VP → V NP V → open I open the box

Review: CKY S S → NP VP NP → I NP → the box VP → V NP V → open I open the box

Review: CKY VP 4 NP 3 NP 3 V 2 V 2 open the box O ( n 3 ) ways of matching

Translation S NP 1 VP 2 I NP 4 V 3 open the box O ( n 3 ) I open the box

Translation S S S O ( n ) NP 1 VP 2 NP 1 NP 1 VP 2 VP 2 I NP 4 watashi wa NP 4 NP 4 V 3 V 3 V 3 open the box hako wo akemasu O ( n 3 ) O ( n ) I open the box watashi wa hako wo akemasu

Translation What about… A B C D E O ( n 5 ) ways of combining?

Translation A A B C D E V1 translate flatten V2 A O ( n ) O ( n ) B C D E C E D B parse O ( n 3 )

Bitext parsing S S NP 1 VP 2 NP 1 VP 2 I NP 4 watashi wa NP 4 V 3 V 3 open the box hako wo akemasu O ( n ? ) I open the box watashi wa hako wo akemasu

Bitext parsing NP 1 NP 1 NP 4 NP 4 V 3 V 3 I I open open the box the box watashi wa watashi wa hako wo hako wo akemasu akemasu Consider rank-2 synchronous CFGs for now

Bitext parsing NP 1 VP 2 NP 1 VP 2 NP 4 NP 4 NP 4 NP 4 V 3 V 3 V 3 V 3 I open the box watashi wa hako wo akemasu

Bitext parsing S S NP 1 NP 1 VP 2 VP 2 NP 1 NP 1 VP 2 VP 2 I open the box watashi wa hako wo akemasu

Bitext parsing S S I open the box watashi wa hako wo akemasu

Bitext parsing VP 2 VP 2 NP 4 NP 4 NP 4 NP 4 V 3 V 3 V 3 V 3 open the box hako wo akemasu O ( n 6 ) ways of matching

Bitext parsing A A B 1 C 2 D 3 E 4 C 2 E 4 B 1 D 3 O ( n 10 ) ways of combining!

Algorithms Translation: easy Bitext parsing: polynomial in n but worst-case exponential in rank

Algorithms Translation with an n -gram language model Offline rescoring Intersect grammar and LM (Wu 1996; Huang et al. 2005): slower Hybrid approaches (Chiang 2005; Zollman and Venugopal 2006)

Extensions

Limitations of synchronous CFGs S S NP VP NP VP John V NP Marie V PP misses Mary manque P NP à Jean

One solution S 1 S 1 NP 2 misses NP 3 manque à NP 3 NP 2 John Marie Mary Jean

Synchronous tree substitution grammars S S NP NP NP 1 VP NP 2 VP John Jean V NP 2 V PP NP NP misses manque P NP 1 Mary Marie à

Synchronous tree substitution grammars S S NP NP VP NP NP VP 1 2 John V NP NP Marie V PP 2 misses manque P NP Mary NP 1 à Jean

Limitations of synchronous TSGs …dat Jan Piet de kinderen zag helpen zwemmen …that John saw Peter help the children swim This pattern extends to n nouns and n verbs

Limitations of synchronous TSGs S S NP NP VP NP NP VP 1 1 John V S Jan V S 2 2 ? saw zag ? Piet de kinderen helpen zwemmen Peter help the children swim

Tree-adjoining grammar S S S V S V NP VP zwemmen NP VP helpen de kinderen V Piet S* V t t

S S V S V zwemmen S S V NP VP helpen NP VP zwemmen S de kinderen V Piet S V S V t NP VP helpen t Piet S* V NP VP t de kinderen V t

Synchronous TAG S S 1 V S 1 zwemmen NP 2 VP 3 NP 2 VP 3 the children V de kinderen V swim t

S S 1 V S 1 zwemmen NP 2 VP 3 NP 2 VP 3 the children V de kinderen V swim t S V S 1 S 1 helpen NP 2 VP 3 NP 2 VP 3 Piet S* V Peter V S* t help

S S V V S 1 zwemmen S 1 helpen NP 2 VP 3 NP 2 VP 3 Piet S S V Peter V t VP 5 help NP 4 VP 5 NP 4 the children V de kinderen V t swim

S S 1 NP 2 VP 3 S V S Peter V V S 1 zwemmen help NP 4 VP 5 helpen NP 2 VP 3 the children V Piet S V swim t VP 5 NP 4 S S 1 de kinderen V V S 1 NP 2 VP 3 t zag NP 2 VP 3 John V S* Jan S* V saw t

S S S 1 V S V zwemmen NP 2 VP 3 helpen John V S S 1 V zag saw NP 2 VP 3 NP 4 VP 5 S Jan Peter V S V help NP 6 VP 7 t NP 4 VP 5 the children V Piet S V swim t VP 7 NP 6 de kinderen V t

Summary Synchronous grammars are useful for various tasks including translation Some rules “in the wild” (Chiang, 2005): X → (de, ’s) X → (X 1 de X 2, the X 2 of X 1) X → (X 1 de X 2, the X 2 that) X → (zai X 1 xia, under X 1 ) X → (zai X 1 qian, before X 1 ) X → (X 1 zhiyi, one of X 1 )

Summary Synchronous context-free grammars vary in power depending on rank Translation is easy; bitext parsing is exponential in rank

Summary Beyond synchronous CFGs, synchronous TSGs allow multilevel rules synchronous TAGs allow discontinuous constituents