Vanda A Statistical Machine Translation Toolkit Matthias B uchse - - PowerPoint PPT Presentation

Vanda A Statistical Machine Translation Toolkit

Matthias B¨ uchse Toni Dietze Johannes Osterholzer Anja Fischer Linda Leuschner WATA, Dresden, 2012-05-30

1 / 15

Outline

State of the Art Vanda: Aims and Solutions

2 / 15

Outline

State of the Art Vanda: Aims and Solutions

3 / 15

Machine Translation

. . . ich s¨ age ihre ente ich sah, wie sie sich duckte ich esse spaghetti mit der gabel ich esse spaghetti mit fleischkl¨

• ßen

. . . . . . i saw her duck i saw her ducking i eat spaghetti with a fork i eat spaghetti with meatballs . . . F E

4 / 15

Machine Translation

. . . ich s¨ age ihre ente ich sah, wie sie sich duckte ich esse spaghetti mit der gabel ich esse spaghetti mit fleischkl¨

• ßen

. . . . . . i saw her duck i saw her ducking i eat spaghetti with a fork i eat spaghetti with meatballs . . . F E h

4 / 15

Machine Translation

. . . ich s¨ age ihre ente ich sah, wie sie sich duckte ich esse spaghetti mit der gabel ich esse spaghetti mit fleischkl¨

• ßen

. . . . . . i saw her duck i saw her ducking i eat spaghetti with a fork i eat spaghetti with meatballs . . . F E h

modelling select H ⊆ E F

4 / 15

Machine Translation

. . . ich s¨ age ihre ente ich sah, wie sie sich duckte ich esse spaghetti mit der gabel ich esse spaghetti mit fleischkl¨

• ßen

. . . . . . i saw her duck i saw her ducking i eat spaghetti with a fork i eat spaghetti with meatballs . . . F E h

modelling select H ⊆ E F decoding apply any given h ∈ H

4 / 15

Machine Translation

. . . ich s¨ age ihre ente ich sah, wie sie sich duckte ich esse spaghetti mit der gabel ich esse spaghetti mit fleischkl¨

• ßen

. . . . . . i saw her duck i saw her ducking i eat spaghetti with a fork i eat spaghetti with meatballs . . . F E h

modelling select H ⊆ E F decoding apply any given h ∈ H training select h ∈ H

4 / 15

Machine Translation

. . . ich s¨ age ihre ente ich sah, wie sie sich duckte ich esse spaghetti mit der gabel ich esse spaghetti mit fleischkl¨

• ßen

. . . . . . i saw her duck i saw her ducking i eat spaghetti with a fork i eat spaghetti with meatballs . . . F E h

modelling select H ⊆ E F decoding apply any given h ∈ H training select h ∈ H

4 / 15

Modelling: State Of The Art

H = {hG,LM,θ | G ∈ G, LM ∈ LM, θ ∈ R2} where hG,LM,θ : f → F E D(G) R

5 / 15

Modelling: State Of The Art

H = {hG,LM,θ | G ∈ G, LM ∈ LM, θ ∈ R2} where hG,LM,θ : f → F E D(G) R πF πE

5 / 15

Modelling: State Of The Art

H = {hG,LM,θ | G ∈ G, LM ∈ LM, θ ∈ R2} where hG,LM,θ : f → F E D(G) R πF πE

5 / 15

Modelling: State Of The Art

H = {hG,LM,θ | G ∈ G, LM ∈ LM, θ ∈ R2} where hG,LM,θ : f → F E D(G) R πF πE

5 / 15

Modelling: State Of The Art

H = {hG,LM,θ | G ∈ G, LM ∈ LM, θ ∈ R2} where hG,LM,θ : f → πE

• F

E D(G) R πF πE

5 / 15

Modelling: State Of The Art

H = {hG,LM,θ | G ∈ G, LM ∈ LM, θ ∈ R2} where hG,LM,θ : f → πE

• G(d)θ1 · LM(πE(d))θ2
• F

E D(G) R πF πE

5 / 15

Modelling: State Of The Art

H = {hG,LM,θ | G ∈ G, LM ∈ LM, θ ∈ R2} where hG,LM,θ : f → πE

• argmaxd : πF(d)=f G(d)θ1 · LM(πE(d))θ2
• F

E D(G) R πF πE

5 / 15

Synchronous Context-Free Grammar

π1: S → S X, S X π2: S → X, X π3: X → yu X 1 you X 2 , have X 2 with X 1 π4: X → X 1 de X 2 , the X 2 that X 1 π5: X → X zhiyi, one of X π6: X → Aozhou, Australia π7: X → Beihan, North Korea π8: X → shi, is π9: X → bangjiao, diplomatic relations π10: X → shaoshu guojia, few countries

6 / 15

Derivation

S, S

7 / 15

Derivation

S, S

π1

⇒ S X, S X

7 / 15

Derivation

S, S

π1

⇒ S X, S X

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

π8

⇒ Aozhou shi X, Australia is X

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

π8

⇒ Aozhou shi X, Australia is X

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

π8

⇒ Aozhou shi X, Australia is X

π5

⇒ Aozhou shi X zhiyi, Australia is one of X

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

π8

⇒ Aozhou shi X, Australia is X

π5

⇒ Aozhou shi X zhiyi, Australia is one of X

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

π8

⇒ Aozhou shi X, Australia is X

π5

⇒ Aozhou shi X zhiyi, Australia is one of X

π4

⇒ Aozhou shi X 1 de X 2 zhiyi, Australia is one of the X 2 that X 1

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

π8

⇒ Aozhou shi X, Australia is X

π5

⇒ Aozhou shi X zhiyi, Australia is one of X

π4

⇒ Aozhou shi X 1 de X 2 zhiyi, Australia is one of the X 2 that X 1

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

π8

⇒ Aozhou shi X, Australia is X

π5

⇒ Aozhou shi X zhiyi, Australia is one of X

π4

⇒ Aozhou shi X 1 de X 2 zhiyi, Australia is one of the X 2 that X 1

π3

⇒ Aozhou shi yu X 1 you X 0 de X 2 zhiyi, Australia is one of the X 2 that have X 0 with X 1

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

π8

⇒ Aozhou shi X, Australia is X

π5

⇒ Aozhou shi X zhiyi, Australia is one of X

π4

⇒ Aozhou shi X 1 de X 2 zhiyi, Australia is one of the X 2 that X 1

π3

⇒ Aozhou shi yu X 1 you X 0 de X 2 zhiyi, Australia is one of the X 2 that have X 0 with X 1

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

π8

⇒ Aozhou shi X, Australia is X

π5

⇒ Aozhou shi X zhiyi, Australia is one of X

π4

⇒ Aozhou shi X 1 de X 2 zhiyi, Australia is one of the X 2 that X 1

π3

⇒ Aozhou shi yu X 1 you X 0 de X 2 zhiyi, Australia is one of the X 2 that have X 0 with X 1

π7

⇒ Aozhou shi yu Beihan you X 0 de X 2 zhiyi, Australia is one of the X 2 that have X 0 with North Korea

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

π8

⇒ Aozhou shi X, Australia is X

π5

⇒ Aozhou shi X zhiyi, Australia is one of X

π4

⇒ Aozhou shi X 1 de X 2 zhiyi, Australia is one of the X 2 that X 1

π3

⇒ Aozhou shi yu X 1 you X 0 de X 2 zhiyi, Australia is one of the X 2 that have X 0 with X 1

π7

⇒ Aozhou shi yu Beihan you X 0 de X 2 zhiyi, Australia is one of the X 2 that have X 0 with North Korea

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

π8

⇒ Aozhou shi X, Australia is X

π5

⇒ Aozhou shi X zhiyi, Australia is one of X

π4

⇒ Aozhou shi X 1 de X 2 zhiyi, Australia is one of the X 2 that X 1

π3

⇒ Aozhou shi yu X 1 you X 0 de X 2 zhiyi, Australia is one of the X 2 that have X 0 with X 1

π7

⇒ Aozhou shi yu Beihan you X 0 de X 2 zhiyi, Australia is one of the X 2 that have X 0 with North Korea

π9

⇒ Aozhou shi yu Beihan you bangjiao de X 2 zhiyi, Australia is one of the X 2 that have diplomatic relations with . . .

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

π8

⇒ Aozhou shi X, Australia is X

π5

⇒ Aozhou shi X zhiyi, Australia is one of X

π4

⇒ Aozhou shi X 1 de X 2 zhiyi, Australia is one of the X 2 that X 1

π3

⇒ Aozhou shi yu X 1 you X 0 de X 2 zhiyi, Australia is one of the X 2 that have X 0 with X 1

π7

⇒ Aozhou shi yu Beihan you X 0 de X 2 zhiyi, Australia is one of the X 2 that have X 0 with North Korea

π9

⇒ Aozhou shi yu Beihan you bangjiao de X 2 zhiyi, Australia is one of the X 2 that have diplomatic relations with . . .

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

π8

⇒ Aozhou shi X, Australia is X

π5

⇒ Aozhou shi X zhiyi, Australia is one of X

π4

⇒ Aozhou shi X 1 de X 2 zhiyi, Australia is one of the X 2 that X 1

π3

⇒ Aozhou shi yu X 1 you X 0 de X 2 zhiyi, Australia is one of the X 2 that have X 0 with X 1

π7

⇒ Aozhou shi yu Beihan you X 0 de X 2 zhiyi, Australia is one of the X 2 that have X 0 with North Korea

π9

⇒ Aozhou shi yu Beihan you bangjiao de X 2 zhiyi, Australia is one of the X 2 that have diplomatic relations with . . .

π10

⇒ Aozhou shi yu Beihan you bangjiao de shaoshu guojia zhiyi, Australia is one of the few countries that have . . .

7 / 15

Derivation

S, S

π1

⇒ S X, S X

π1

⇒ S X 1 X 2 , S X 1 X 2

π2

⇒ X 0 X 1 X 2 , X 0 X 1 X 2

π6

⇒ Aozhou X 1 X 2 , Australia X 1 X 2

π8

⇒ Aozhou shi X, Australia is X

π5

⇒ Aozhou shi X zhiyi, Australia is one of X

π4

⇒ Aozhou shi X 1 de X 2 zhiyi, Australia is one of the X 2 that X 1

π3

⇒ Aozhou shi yu X 1 you X 0 de X 2 zhiyi, Australia is one of the X 2 that have X 0 with X 1

π7

⇒ Aozhou shi yu Beihan you X 0 de X 2 zhiyi, Australia is one of the X 2 that have X 0 with North Korea

π9

⇒ Aozhou shi yu Beihan you bangjiao de X 2 zhiyi, Australia is one of the X 2 that have diplomatic relations with . . .

π10

⇒ Aozhou shi yu Beihan you bangjiao de shaoshu guojia zhiyi, Australia is one of the few countries that have . . .

7 / 15

Advancing the State of the Art

grammar implementation reference SCFG Hiero (closed source) [CLM+05] Moses (C++) [KHB+07] Joshua (Java) [LCBD+09] cdec (C++) [DLG+10]

8 / 15

Advancing the State of the Art

grammar implementation reference SCFG Hiero (closed source) [CLM+05] Moses (C++) [KHB+07] Joshua (Java) [LCBD+09] cdec (C++) [DLG+10] XTOPs closed source [GKM08] STAG closed source [DK09] MBOT extension to Moses [Mal10]

8 / 15

Advancing the State of the Art

grammar implementation reference SCFG Hiero (closed source) [CLM+05] Moses (C++) [KHB+07] Joshua (Java) [LCBD+09] cdec (C++) [DLG+10] XTOPs closed source [GKM08] STAG closed source [DK09] MBOT extension to Moses [Mal10]

◮ monolithic algorithms, specifically implemented ◮ little standardisation, manual scripting ◮ lack of documentation

8 / 15

Outline

State of the Art Vanda: Aims and Solutions

9 / 15

Vanda Components

IRTGs formal framework + Vanda Toolbox in Haskell + Vanda Studio (hyper) workflow management

10 / 15

Vanda Components

IRTGs formal framework + Vanda Toolbox in Haskell + Vanda Studio (hyper) workflow management [KK11] flexible

10 / 15

Vanda Components

IRTGs formal framework + Vanda Toolbox in Haskell + Vanda Studio (hyper) workflow management [KK11] concise, yet clean flexible rapid prototyping

10 / 15

Vanda Components

IRTGs formal framework + Vanda Toolbox in Haskell + Vanda Studio (hyper) workflow management [KK11] concise, yet clean standardized procedures flexible rapid prototyping well documented universal

10 / 15

Formal Framework: IRTGs over (A1, A2)

R D(G) TΣ T∆1 T∆2 A1 A2

G

h1 h2 hA1 hA2 regular state behavior rules semantic domains

11 / 15

Formal Framework: IRTGs over (A1, A2)

R D(G) TΣ T∆1 T∆2 A1 A2

G

h1 h2 hA1 hA2 regular state behavior rules semantic domains Subsume: A1 A2

11 / 15

Formal Framework: IRTGs over (A1, A2)

R D(G) TΣ T∆1 T∆2 A1 A2

G

h1 h2 hA1 hA2 regular state behavior rules semantic domains Subsume: A1 A2 SCFG F, concatenation E, concatenation

11 / 15

Formal Framework: IRTGs over (A1, A2)

R D(G) TΣ T∆1 T∆2 A1 A2

G

h1 h2 hA1 hA2 regular state behavior rules semantic domains Subsume: A1 A2 lnXTOPs F, concatenation TE, top concatenation

11 / 15

Formal Framework: IRTGs over (A1, A2)

R D(G) TΣ T∆1 T∆2 A1 A2

G

h1 h2 hA1 hA2 regular state behavior rules semantic domains Subsume: A1 A2 STAG TF, top concatenation, TE, top concatenation, second-order subst. second-order subst.

11 / 15

Modularity

hG,LM,θ : f → πE

• argmaxd : πF(d)=f G(d)θ1 · LM(πE(d))θ2
• 12 / 15

Modularity

hG,LM,θ : f → πE

• argmaxd : πF(d)=f G(d)θ1 · LM(πE(d))θ2
• = πE
• argmaxd (= f )(πF(d)) · G(d)θ1 · LM(πE(d))θ2
• 12 / 15

Modularity

hG,LM,θ : f → πE

• argmaxd : πF(d)=f G(d)θ1 · LM(πE(d))θ2
• = πE
• argmaxd (= f )(πF(d)) · G(d)θ1 · LM(πE(d))θ2
• = πE
• argmaxd
• (= f ) ⊳ Gθ1 ⊲ LMθ2

(d)

• 12 / 15

Modularity

hG,LM,θ : f → πE

• argmaxd : πF(d)=f G(d)θ1 · LM(πE(d))θ2
• = πE
• argmaxd (= f )(πF(d)) · G(d)θ1 · LM(πE(d))θ2
• = πE
• argmaxd
• (= f ) ⊳ Gθ1 ⊲ LMθ2

(d)

• = πE
• argmaxd (= f ) ⊳ G θ1 ⊲ LMθ2(d)
• 12 / 15

Modularity

hG,LM,θ : f → πE

• argmaxd : πF(d)=f G(d)θ1 · LM(πE(d))θ2
• = πE
• argmaxd (= f )(πF(d)) · G(d)θ1 · LM(πE(d))θ2
• = πE
• argmaxd
• (= f ) ⊳ Gθ1 ⊲ LMθ2

(d)

• = πE
• argmaxd (= f ) ⊳ G θ1 ⊲ LMθ2(d)
• ϕ: supp(G ⊲ LM) → supp(G ⊲ LM)

G ⊲ LM = (G ⊲ LM) ◦ ϕ

12 / 15

Modularity

hG,LM,θ : f → πE

• argmaxd : πF(d)=f G(d)θ1 · LM(πE(d))θ2
• = πE
• argmaxd (= f )(πF(d)) · G(d)θ1 · LM(πE(d))θ2
• = πE
• argmaxd
• (= f ) ⊳ Gθ1 ⊲ LMθ2

(d)

• = πE
• argmaxd (= f ) ⊳ G θ1 ⊲ LMθ2(d)
• ϕ: supp(G ⊲ LM) → supp(G ⊲ LM)

G ⊲ LM = (G ⊲ LM) ◦ ϕ RTG ⊳ STAG ⊆ STAG ⊇ STAG ⊲ RTG [BNV11]

12 / 15

Modularity

hG,LM,θ : f → πE

• argmaxd : πF(d)=f G(d)θ1 · LM(πE(d))θ2
• = πE
• argmaxd (= f )(πF(d)) · G(d)θ1 · LM(πE(d))θ2
• = πE
• argmaxd
• (= f ) ⊳ Gθ1 ⊲ LMθ2

(d)

• = πE
• argmaxd (= f ) ⊳ G θ1 ⊲ LMθ2(d)
• 12 / 15

Current State

◮ Vanda Toolbox: small-scale data; state-split grammars, input

product, output product, n-best derivations, KA∗, string-to-tree rule extraction (GHKM), inside/outside EM

◮ Vanda Studio: proof-of-concept implementation (10K lines),

code generation for single workstation

13 / 15

Current State

◮ Vanda Toolbox: small-scale data; state-split grammars, input

product, output product, n-best derivations, KA∗, string-to-tree rule extraction (GHKM), inside/outside EM

◮ Vanda Studio: proof-of-concept implementation (10K lines),

code generation for single workstation I believe that this community should commit itself to achieving the goal, before this decade is out, of providing versatile off-the-shelf components for machine translation and using them successfully in a large-scale task.

13 / 15

References I

Matthias B¨ uchse, Mark-Jan Nederhof, and Heiko Vogler. Tree parsing with synchronous tree-adjoining grammars. In Proceedings of IWPT, 2011. David Chiang, Adam Lopez, Nitin Madnani, Christof Monz, Philip Resnik, and Michael Subotin. The Hiero machine translation system: extensions, evaluation, and analysis. In HLT ’05: Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, pages 779–786, Morristown, NJ, USA, 2005. Association for Computational Linguistics. Steve DeNeefe and Kevin Knight. Synchronous tree-adjoining machine translation. In EMNLP ’09: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 727–736, Morristown, NJ, USA, 2009. Association for Computational Linguistics. Chris Dyer, Adam Lopez, Juri Ganitkevitch, Jonathan Weese, Ferhan Ture, Phil Blunsom, Hendra Setiawan, Vladimir Eidelman, and Philip Resnik. cdec: A decoder, alignment, and learning framework for finite-state and context-free translation models. In Proceedings of the ACL 2010 System Demonstrations, pages 7–12, Uppsala, Sweden, July 2010. Association for Computational Linguistics. Jonathan Graehl, Kevin Knight, and Jonathan May. Training tree transducers.

• Comp. Ling., 34(3):391–427, 2008.

14 / 15

References II

Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondˇ rej Bojar, Alexandra Constantin, and Evan Herbst. Moses: open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, ACL ’07, pages 177–180, Stroudsburg, PA, USA, 2007. Association for Computational Linguistics. Alexander Koller and Marco Kuhlmann. A generalized view on parsing and translation. In Proceedings IWPT 2011, 2011. Zhifei Li, Chris Callison-Burch, Chris Dyer, Juri Ganitkevitch, Sanjeev Khudanpur, Lane Schwartz, Wren N. G. Thornton, Jonathan Weese, and Omar F. Zaidan. Joshua: an open source toolkit for parsing-based machine translation. In Proceedings of the Fourth Workshop on Statistical Machine Translation, StatMT ’09, pages 135–139, Stroudsburg, PA, USA, 2009. Association for Computational Linguistics. Andreas Maletti. Why synchronous tree substitution grammars? In Proc. 11th Conf. North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 2010.

15 / 15