weighted parsing for grammar based language models
play

Weighted parsing for grammar-based language models FSMNLP 2019 - PowerPoint PPT Presentation

Feb 23, 2024 β€’353 likes β€’1.61k views

Weighted parsing for grammar-based language models FSMNLP 2019 Richard Mrbitz Heiko Vogler 2019-09-25 The weighted parsing problem language e.g., English sentences ( ) syntactic object e.g., Fruit fmies like bananas ?

  1. Language algebras 𝑒 ∈ AST (𝐻) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 … Fruit Fruit 𝛽 … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) S β†’ 𝛽( NP , VP ) factors ( Fruit flies like bananas ) = { Fruit , like bananas , … } 7 / 21 βŸ¨π‘¦ 1 flies 𝑦 2 ⟩ 𝜌 𝛡 : T 𝑆 β†’ T 𝛡 interpretation of 𝛡 as operations on the set of syntactic objects β„’ = 𝛦 βˆ— (.) 𝛦 βˆ— : T 𝛡 (terms) β†’ 𝛦 βˆ— (syntactic objects)

  2. Language algebras ⟨ Fruit flies … ⟩ 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 … Fruit Fruit 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) S β†’ 𝛽( NP , VP ) factors ( Fruit flies like bananas ) = { Fruit , like bananas , … } 7 / 21 𝜌 𝛡 : T 𝑆 β†’ T 𝛡 interpretation of 𝛡 as operations on the set of syntactic objects β„’ = 𝛦 βˆ— (.) 𝛦 βˆ— : T 𝛡 (terms) β†’ 𝛦 βˆ— (syntactic objects)

  3. Language algebras ⟨ Fruit flies … ⟩ 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 … Fruit Fruit 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) S β†’ 𝛽( NP , VP ) factors ( Fruit flies like bananas ) = { Fruit , like bananas , … } 7 / 21 𝜌 𝛡 : T 𝑆 β†’ T 𝛡 interpretation of 𝛡 as operations on the set of syntactic objects β„’ = 𝛦 βˆ— (.) 𝛦 βˆ— : T 𝛡 (terms) β†’ 𝛦 βˆ— (syntactic objects)

  4. Language algebras 𝛽 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 … 𝛾 πœ€ 𝛿 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) S β†’ 𝛽( NP , VP ) factors ( Fruit flies like bananas ) = { Fruit , like bananas , … } 7 / 21 𝜌 𝛡 : T 𝑆 β†’ T 𝛡 (.) 𝛦 βˆ— : T 𝛡 β†’ 𝛦 βˆ— interpretation of 𝛡 as operations on the set of syntactic objects β„’ = 𝛦 βˆ— (.) 𝛦 βˆ— : T 𝛡 (terms) β†’ 𝛦 βˆ— (syntactic objects)

  5. Semirings Algebraic structure ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) βŠ— is used to evaluate an AST to a weight βŠ• accumulates the weights of several ASTs Examples ( 𝔺 , ∨, ∧, false , true ) the Boolean semiring with 𝔺 = { false , true } ( β„• ∞ , +, β‹…, 0, 1) the semiring of natural numbers ( β„• ∞ , min , +, ∞, 0) the tropical semiring ( ℝ 1 0 , max , β‹…, 0, 1) the Viterbi semiring Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 8 / 21

  6. Semirings Algebraic structure ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) βŠ— is used to evaluate an AST to a weight βŠ• accumulates the weights of several ASTs Examples ( 𝔺 , ∨, ∧, false , true ) the Boolean semiring with 𝔺 = { false , true } ( β„• ∞ , +, β‹…, 0, 1) the semiring of natural numbers ( β„• ∞ , min , +, ∞, 0) the tropical semiring ( ℝ 1 0 , max , β‹…, 0, 1) the Viterbi semiring Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 8 / 21

  7. 𝛻 βŠ— = { mul 𝕝 ( 𝕝 1 , … , 𝕝 𝑛 ) = 𝕝 βŠ— 𝕝 1 βŠ— β‹― βŠ— 𝕝 𝑛 with 𝛻 med = { del , ins , rep = , rep β‰  , nil } Multioperator monoids (M-monoids) mul 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) Minimum edit distance M-monoid ({{π‘œ} ∣ π‘œ ∈ β„• }, min ∘ βˆͺ, βˆ…, 𝛻 med ) 0 , max , 0, 𝛻 mul ) Viterbi M-monoid ( ℝ 1 Examples (𝑛) 𝕝 ∣ 𝕝 ∈ 𝕃 , 𝑛 ∈ β„• } Generalization of semirings (𝑛) M-monoid ( 𝕃 , βŠ• , πŸ™ , 𝛻 βŠ— ) where Semiring ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) ⟢ ( 𝕃 , βŠ• , πŸ™ , 𝛻) ⟢ ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) 9 / 21 binary βŠ— set of 𝑛 -ary operations 𝛻 (here: distributive)

  8. with 𝛻 med = { del , ins , rep = , rep β‰  , nil } Multioperator monoids (M-monoids) Generalization of semirings 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) Minimum edit distance M-monoid ({{π‘œ} ∣ π‘œ ∈ β„• }, min ∘ βˆͺ, βˆ…, 𝛻 med ) 0 , max , 0, 𝛻 mul ) Viterbi M-monoid ( ℝ 1 Examples (𝑛) mul ∣ 𝕝 ∈ 𝕃 , 𝑛 ∈ β„• } 𝕝 (𝑛) ⟢ ( 𝕃 , βŠ• , πŸ™ , 𝛻) ⟢ ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) 9 / 21 binary βŠ— set of 𝑛 -ary operations 𝛻 (here: distributive) Semiring ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) ⇝ M-monoid ( 𝕃 , βŠ• , πŸ™ , 𝛻 βŠ— ) where 𝛻 βŠ— = { mul 𝕝 ( 𝕝 1 , … , 𝕝 𝑛 ) = 𝕝 βŠ— 𝕝 1 βŠ— β‹― βŠ— 𝕝 𝑛

  9. Multioperator monoids (M-monoids) Generalization of semirings 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) Minimum edit distance M-monoid ({{π‘œ} ∣ π‘œ ∈ β„• }, min ∘ βˆͺ, βˆ…, 𝛻 med ) 0 , max , 0, 𝛻 mul ) Viterbi M-monoid ( ℝ 1 Examples (𝑛) mul ∣ 𝕝 ∈ 𝕃 , 𝑛 ∈ β„• } 𝕝 (𝑛) ⟢ ( 𝕃 , βŠ• , πŸ™ , 𝛻) ⟢ ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) 9 / 21 binary βŠ— set of 𝑛 -ary operations 𝛻 (here: distributive) Semiring ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) ⇝ M-monoid ( 𝕃 , βŠ• , πŸ™ , 𝛻 βŠ— ) where 𝛻 βŠ— = { mul 𝕝 ( 𝕝 1 , … , 𝕝 𝑛 ) = 𝕝 βŠ— 𝕝 1 βŠ— β‹― βŠ— 𝕝 𝑛 with 𝛻 med = { del , ins , rep = , rep β‰  , nil }

  10. 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra) 0.2 β‹… 𝕝 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) wt (𝑒) ∈ T 𝛻 wt … 1.0 Weight algebras S β†’ 𝛽( NP , VP ) (.) 𝛦 βˆ— Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 𝜌 𝛡 … 𝛾 πœ€ 𝛿 𝛽 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) 10 / 21

  11. 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra) 0.2 β‹… 𝕝 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) wt (𝑒) ∈ T 𝛻 wt … 1.0 Weight algebras S β†’ 𝛽( NP , VP ) (.) 𝛦 βˆ— Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 𝜌 𝛡 … 𝛾 πœ€ 𝛿 𝛽 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) 10 / 21

  12. 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra) Weight algebras S β†’ 𝛽( NP , VP ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) wt (𝑒) ∈ T 𝛻 wt … 1.0 0.2 β‹… 𝕝 1 10 / 21 (.) 𝛦 βˆ— Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 𝜌 𝛡 … 𝛾 πœ€ 𝛿 𝛽 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2

  13. Weight algebras (.) 𝛦 βˆ— 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) wt (𝑒) ∈ T 𝛻 wt … 1.0 0.2 β‹… 𝕝 1 S β†’ 𝛽( NP , VP ) 10 / 21 Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra)

  14. Weight algebras (.) 𝛦 βˆ— 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) wt (𝑒) ∈ T 𝛻 wt … 1.0 0.2 S β†’ 𝛽( NP , VP ) 10 / 21 Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra)

  15. Weight algebras S β†’ 𝛽( NP , VP ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) wt (𝑒) ∈ T 𝛻 wt 0.6 β‹… … 1.0 0.2 (.) 𝛦 βˆ— Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 10 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra)

  16. Weight algebras S β†’ 𝛽( NP , VP ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) wt (𝑒) ∈ T 𝛻 wt 0.6 β‹… … 1.0 0.2 0.12 β‹… … (.) 𝛦 βˆ— Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 𝜌 𝛡 … 𝛾 πœ€ 𝛿 𝛽 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) 10 / 21 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra)

  17. Weight algebras S β†’ 𝛽( NP , VP ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) 0 (.) ℝ 1 0.12 β‹… … wt (𝑒) ∈ T 𝛻 wt … 1.0 0.2 β‹… 𝕝 1 10 / 21 (.) 𝛦 βˆ— NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 Fruit flies … 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra)

  18. β„’ A wRTG-LM is a tuple ( (𝐻 = (𝑂, 𝛡, 𝐡 0 , 𝑆)) Weighted RTG-based language models ⏟ ⏟ ⏟ ⏟ ⏟ ⏟ ⏟ RTG , ⏟ language algebra ), ( 𝕃 , βŠ• , πŸ™ , 𝛻) ⏟ M-monoid , wt ⏟ wt : 𝑆 β†’ 𝛻 ) . Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 ⏟⏟ Defjnition (weighted RTG-based language model) S β†’ 𝛽( NP , VP ) 𝜌 𝛡 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 (.) 𝛦 βˆ— 0.2 β‹… 𝕝 1 1.0 … wt wt (𝑒) ∈ T 𝛻 0.12 β‹… … (.) ℝ 1 0 Fruit flies … 11 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2

  19. Weighted RTG-based language models language algebra ⏟ ⏟ ⏟ ⏟ ⏟ ⏟ ⏟ RTG , ⏟ ), S β†’ 𝛽( NP , VP ) ( 𝕃 , βŠ• , πŸ™ , 𝛻) ⏟ M-monoid , wt ⏟ wt : 𝑆 β†’ 𝛻 ) . Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 ⏟⏟ Defjnition (weighted RTG-based language model) (.) 𝛦 βˆ— 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 11 / 21 0.2 β‹… 𝕝 1 1.0 … wt wt (𝑒) ∈ T 𝛻 0.12 β‹… … (.) ℝ 1 0 Fruit flies … 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 β„’ A wRTG-LM is a tuple ( (𝐻 = (𝑂, 𝛡, 𝐡 0 , 𝑆))

  20. Outline 1 Weighted RTG-based language models 2 The weighted parsing problem 3 The weighted parsing algorithm Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 12 / 21

  21. The weighted parsing problem 0.2 β‹… 𝕝 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) 𝛦 βˆ— Fruit flies … 0 (.) ℝ 1 0.12 β‹… … wt (𝑒) ∈ T 𝛻 wt … 1.0 13 / 21 S β†’ 𝛽( NP , VP ) 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 𝜌 𝛡 … 𝛾 πœ€ 𝛿 𝛽 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2

  22. The weighted parsing problem S β†’ 𝛽( NP , VP ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) 𝛦 βˆ— 𝜌 𝛡 (.) 𝛦 βˆ— Fruit flies … 0 (.) ℝ 1 0.12 β‹… … wt (𝑒) ∈ T 𝛻 wt … 1.0 0.2 β‹… 𝕝 1 𝛿 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 πœ€ 𝛾 … 𝜌 𝛡 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 13 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 𝑒 β€² ∈ T 𝛡 𝑒 β€² ∈ AST (𝐻)

  23. The weighted parsing problem (.) 𝛦 βˆ— 0.12 β‹… … (.) ℝ 1 0 Fruit flies … (.) 𝛦 βˆ— 𝜌 𝛡 wt (𝑒 β€² ) ∈ T 𝛻 S β†’ 𝛽( NP , VP ) wt 0.0144 (.) ℝ 1 0 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 wt (𝑒) ∈ T 𝛻 wt … πœ€ NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 𝛾 … 𝜌 𝛡 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 0.2 β‹… 𝕝 1 1.0 13 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 𝑒 β€² ∈ T 𝛡 𝑒 β€² ∈ AST (𝐻)

  24. The weighted parsing problem wt (𝑒 β€² ) ∈ T 𝛻 (.) ℝ 1 0 Fruit flies … (.) 𝛦 βˆ— 𝜌 𝛡 (.) 𝛦 βˆ— wt S β†’ 𝛽( NP , VP ) 0.0144 (.) ℝ 1 0 ( ℝ 1 0 , max , 0, 𝛻 mul ) Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 0.12 β‹… … wt (𝑒) ∈ T 𝛻 wt 𝛾 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ … … 𝜌 𝛡 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 0.2 β‹… 𝕝 1 1.0 13 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 𝑒 β€² ∈ T 𝛡 𝑒 β€² ∈ AST (𝐻)

  25. The weighted parsing problem wt (𝑒 β€² ) ∈ T 𝛻 (.) ℝ 1 0 Fruit flies … (.) 𝛦 βˆ— 𝜌 𝛡 (.) 𝛦 βˆ— wt S β†’ 𝛽( NP , VP ) 0.0144 (.) ℝ 1 0 ( ℝ 1 0 , max , 0, 𝛻 mul ) Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 0.12 β‹… … wt (𝑒) ∈ T 𝛻 wt … NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 0.2 β‹… 𝕝 1 1.0 13 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 max { 𝑒 β€² ∈ T 𝛡 𝑒 β€² ∈ AST (𝐻)

  26. The weighted parsing problem wt (.) ℝ 1 0 Fruit flies … (.) 𝛦 βˆ— 𝜌 𝛡 (.) 𝛦 βˆ— wt (𝑒 β€² ) ∈ T 𝛻 0.0144 wt (𝑒) ∈ T 𝛻 (.) ℝ 1 0 ( ℝ 1 0 , max , 0, 𝛻 mul ) parse Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 S β†’ 𝛽( NP , VP ) 0.12 β‹… … wt … NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 0.2 β‹… 𝕝 1 1.0 13 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 max { 𝑒 β€² ∈ T 𝛡 𝑒 β€² ∈ AST (𝐻)

  27. The weighted parsing problem (.) ℝ 1 Fruit flies … (.) 𝛦 βˆ— 𝜌 𝛡 (.) 𝛦 βˆ— wt (𝑒 β€² ) ∈ T 𝛻 wt 0.0144 0 S β†’ 𝛽( NP , VP ) ( ℝ 1 0 , max , 0, 𝛻 mul ) parse parse (𝑏) = βˆ‘ βŠ• π‘’βˆˆ AST (𝐻,𝑏) wt (𝑒) Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 0 (.) ℝ 1 0.12 β‹… … wt (𝑒) ∈ T 𝛻 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 0.2 β‹… 𝕝 1 1.0 … wt 13 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 max { 𝑒 β€² ∈ T 𝛡 𝑒 β€² ∈ AST (𝐻)

  28. The weighted parsing problem Algebraic dynamic programming (Giegerich, Meyer, and Stefgen 2004) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) Shamir 1961) Reduct of a grammar and a syntactic object (cf. Bar-Hillel, Perles, and matrix chain multiplication minimum edit distance Parsing with superior grammars (Knuth 1977; Nederhof 2003) Examples π‘œ best derivation(s) best derivation(s) derivation forest probability of best derivation string probability recognition Semiring parsing (Goodman 1999) 14 / 21

  29. Outline 1 Weighted RTG-based language models 2 The weighted parsing problem 3 The weighted parsing algorithm Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 15 / 21

  30. ((𝐻, β„’ ), 𝕃 , wt ) - 𝑏 ∈ β„’ Weighted parsing algorithm Two-phase pipeline (Goodman 1999; Nederhof 2003) - wRTG-LM parse (𝑏) = βˆ‘ βŠ• π‘’βˆˆ AST (𝐻,𝑏) wt (𝑒) Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 16 / 21

  31. Weighted parsing algorithm Two-phase pipeline (Goodman 1999; Nederhof 2003) - wRTG-LM parse (𝑏) = βˆ‘ βŠ• π‘’βˆˆ AST (𝐻,𝑏) wt (𝑒) Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 16 / 21 ((𝐻, β„’ ), 𝕃 , wt ) - 𝑏 ∈ β„’

  32. Weighted parsing algorithm Two-phase pipeline (Goodman 1999; Nederhof 2003) - wRTG-LM parse (𝑏) = βˆ‘ βŠ• π‘’βˆˆ AST (𝐻,𝑏) wt (𝑒) canonical weighted deduction system Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 16 / 21 ((𝐻, β„’ ), 𝕃 , wt ) - 𝑏 ∈ β„’

  33. Weighted parsing algorithm Two-phase pipeline (Goodman 1999; Nederhof 2003) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) wRTG-LM system deduction weighted canonical wt (𝑒) π‘’βˆˆ AST (𝐻,𝑏) parse (𝑏) = βˆ‘ βŠ• - wRTG-LM 16 / 21 ((𝐻, β„’ ), 𝕃 , wt ) ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’

  34. Weighted parsing algorithm Two-phase pipeline (Goodman 1999; Nederhof 2003) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) ? = wt β€² (𝑒) π‘’βˆˆ AST (𝐻 β€² ) βˆ‘ βŠ• wRTG-LM system deduction weighted canonical wt (𝑒) π‘’βˆˆ AST (𝐻,𝑏) parse (𝑏) = βˆ‘ βŠ• - wRTG-LM 16 / 21 ((𝐻, β„’ ), 𝕃 , wt ) ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’

  35. Weighted parsing algorithm Two-phase pipeline (Goodman 1999; Nederhof 2003) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) algorithm computation value ? = wt β€² (𝑒) π‘’βˆˆ AST (𝐻 β€² ) βˆ‘ βŠ• 16 / 21 wRTG-LM system deduction weighted canonical wt (𝑒) π‘’βˆˆ AST (𝐻,𝑏) parse (𝑏) = βˆ‘ βŠ• - wRTG-LM ((𝐻, β„’ ), 𝕃 , wt ) ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’

  36. Weighted parsing algorithm π‘’βˆˆ AST (𝐻 β€² ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) ? = 0 ) π‘Š(𝐡 β€² algorithm computation value ? = wt β€² (𝑒) βˆ‘ βŠ• Two-phase pipeline (Goodman 1999; Nederhof 2003) wRTG-LM system deduction weighted canonical wt (𝑒) π‘’βˆˆ AST (𝐻,𝑏) parse (𝑏) = βˆ‘ βŠ• - wRTG-LM 16 / 21 ((𝐻, β„’ ), 𝕃 , wt ) ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’

  37. Weighted parsing algorithm π‘’βˆˆ AST (𝐻 β€² ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) weighted parsing algorithm ? = 0 ) π‘Š(𝐡 β€² algorithm computation value ? = wt β€² (𝑒) βˆ‘ βŠ• Two-phase pipeline (Goodman 1999; Nederhof 2003) wRTG-LM system deduction weighted canonical wt (𝑒) π‘’βˆˆ AST (𝐻,𝑏) parse (𝑏) = βˆ‘ βŠ• - wRTG-LM 16 / 21 ((𝐻, β„’ ), 𝕃 , wt ) ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’

  38. Canonical weighted deduction system Weight preserving 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) wt (𝑒) = wt β€² (πœ”(𝑒)) for every 𝑒 ∈ AST (𝐻, 𝑏) 2 Bijection πœ”: AST (𝐻, 𝑏) β†’ AST (𝐻 β€² ) 1 𝐡 β†’ 𝜏(𝐡 1 , … , 𝐡 𝑛 ) is a rule { [𝐡 , 𝑏 0 ] Parsing as deduction (Shieber, Schabes, and Pereira 1995) cwds 17 / 21 - wRTG-LM ((𝐻, β„’ ), 𝕃 , wt ) wRTG-LM ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’ [𝐡 1 , 𝑏 1 ] … [𝐡 𝑛 , 𝑏 𝑛 ] 𝑏 0 , 𝑏 1 , … , 𝑏 𝑛 ∈ factors (𝑏) 𝑏 0 = 𝜏(𝑏 1 , … , 𝑏 𝑛 )

  39. Canonical weighted deduction system Weight preserving 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) wt (𝑒) = wt β€² (πœ”(𝑒)) for every 𝑒 ∈ AST (𝐻, 𝑏) 2 Bijection πœ”: AST (𝐻, 𝑏) β†’ AST (𝐻 β€² ) 1 𝐡 β†’ 𝜏(𝐡 1 , … , 𝐡 𝑛 ) is a rule { [𝐡 , 𝑏 0 ] Parsing as deduction (Shieber, Schabes, and Pereira 1995) cwds 17 / 21 - wRTG-LM ((𝐻, β„’ ), 𝕃 , wt ) wRTG-LM ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’ [𝐡 1 , 𝑏 1 ] … [𝐡 𝑛 , 𝑏 𝑛 ] 𝑏 0 , 𝑏 1 , … , 𝑏 𝑛 ∈ factors (𝑏) 𝑏 0 = 𝜏(𝑏 1 , … , 𝑏 𝑛 )

  40. Canonical weighted deduction system Weight preserving 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) wt (𝑒) = wt β€² (πœ”(𝑒)) for every 𝑒 ∈ AST (𝐻, 𝑏) 2 Bijection πœ”: AST (𝐻, 𝑏) β†’ AST (𝐻 β€² ) 1 𝐡 β†’ 𝜏(𝐡 1 , … , 𝐡 𝑛 ) is a rule { [𝐡 , 𝑏 0 ] Parsing as deduction (Shieber, Schabes, and Pereira 1995) cwds 17 / 21 - wRTG-LM ((𝐻, β„’ ), 𝕃 , wt ) wRTG-LM ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’ [𝐡 1 , 𝑏 1 ] … [𝐡 𝑛 , 𝑏 𝑛 ] 𝑏 0 , 𝑏 1 , … , 𝑏 𝑛 ∈ factors (𝑏) 𝑏 0 = 𝜏(𝑏 1 , … , 𝑏 𝑛 )

  41. Canonical weighted deduction system Weight preserving 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) wt (𝑒) = wt β€² (πœ”(𝑒)) for every 𝑒 ∈ AST (𝐻, 𝑏) 2 Bijection πœ”: AST (𝐻, 𝑏) β†’ AST (𝐻 β€² ) 1 17 / 21 { [𝐡 , 𝑏 0 ] Parsing as deduction (Shieber, Schabes, and Pereira 1995) cwds - wRTG-LM ((𝐻, β„’ ), 𝕃 , wt ) wRTG-LM ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’ 𝐡 β†’ 𝜏(𝐡 1 , … , 𝐡 𝑛 ) is a rule [𝐡 1 , 𝑏 1 ] … [𝐡 𝑛 , 𝑏 𝑛 ] 𝑏 0 , 𝑏 1 , … , 𝑏 𝑛 ∈ factors (𝑏) 𝑏 0 = 𝜏(𝑏 1 , … , 𝑏 𝑛 )

  42. Canonical weighted deduction system Weight preserving 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) wt (𝑒) = wt β€² (πœ”(𝑒)) for every 𝑒 ∈ AST (𝐻, 𝑏) 2 Bijection πœ”: AST (𝐻, 𝑏) β†’ AST (𝐻 β€² ) 1 𝐡 β†’ 𝜏(𝐡 1 , … , 𝐡 𝑛 ) is a rule { [𝐡 , 𝑏 0 ] Parsing as deduction (Shieber, Schabes, and Pereira 1995) cwds 17 / 21 - wRTG-LM ((𝐻, β„’ ), 𝕃 , wt ) wRTG-LM ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’ [𝐡 1 , 𝑏 1 ] … [𝐡 𝑛 , 𝑏 𝑛 ] 𝑏 0 , 𝑏 1 , … , 𝑏 𝑛 ∈ factors (𝑏) 𝑏 0 = 𝜏(𝑏 1 , … , 𝑏 𝑛 )

  43. Canonical weighted deduction system Weight preserving 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) wt (𝑒) = wt β€² (πœ”(𝑒)) for every 𝑒 ∈ AST (𝐻, 𝑏) 2 Bijection πœ”: AST (𝐻, 𝑏) β†’ AST (𝐻 β€² ) 1 𝐡 β†’ 𝜏(𝐡 1 , … , 𝐡 𝑛 ) is a rule { [𝐡 , 𝑏 0 ] Parsing as deduction (Shieber, Schabes, and Pereira 1995) cwds 17 / 21 - wRTG-LM ((𝐻, β„’ ), 𝕃 , wt ) wRTG-LM ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’ [𝐡 1 , 𝑏 1 ] … [𝐡 𝑛 , 𝑏 𝑛 ] 𝑏 0 , 𝑏 1 , … , 𝑏 𝑛 ∈ factors (𝑏) 𝑏 0 = 𝜏(𝑏 1 , … , 𝑏 𝑛 )

  44. Weighted parsing algorithm π‘’βˆˆ AST (𝐻 β€² ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) weighted parsing algorithm ? = 0 ) π‘Š(𝐡 β€² algorithm computation value ? = wt β€² (𝑒) βˆ‘ βŠ• Two-phase pipeline (Goodman 1999; Nederhof 2003) wRTG-LM system deduction weighted canonical wt (𝑒) π‘’βˆˆ AST (𝐻,𝑏) parse (𝑏) = βˆ‘ βŠ• - wRTG-LM 18 / 21 ((𝐻, β„’ ), 𝕃 , wt ) ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’

  45. Value computation algorithm 5: 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 12: until changed = false π‘Š(𝐡) ← π‘Š new 11: changed ← true 10: 9: 8: 7: 6: changed ← false π‘Š(𝐡) ← πŸ™ 0 , 𝑆 β€² ) Output: π‘Š(𝐡 β€² 0 ) 2: 19 / 21 3: repeat 4: Input: a wRTG-LM ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , βŠ• , πŸ™ , 𝛻), wt β€² ) with 𝐻 β€² = (𝑂 β€² , 𝛡 β€² , 𝐡 β€² Variables: π‘Š: 𝑂 β€² β†’ 𝕃 , π‘Š new ∈ 𝕃 , changed ∈ 𝔺 1: for each 𝐡 ∈ 𝑂 β€² do for each 𝐡 ∈ 𝑂 β€² do π‘Š new ← πŸ™ for each 𝑠 = (𝐡 β†’ βŸ¨π‘¦ 1 … 𝑦 𝑛 ⟩(𝐡 1 , … , 𝐡 𝑛 )) in 𝑆 β€² do π‘Š new ← π‘Š new βŠ• wt β€² (𝑠)(π‘Š(𝐡 1 ), … , π‘Š(𝐡 𝑛 )) if π‘Š(𝐡) β‰  π‘Š new then

  46. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ … πœ€ πŸ™ πŸ™ ( B A S 𝜏 𝛾 𝛿 S 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ Value computation algorithm (example) πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ A πœ• 3 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt )

  47. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ … πœ€ πŸ™ πŸ™ ( B A S 𝜏 𝛾 𝛿 S 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ Value computation algorithm (example) πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ A πœ• 3 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt )

  48. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ … πœ€ πŸ™ πŸ™ ( B A S 𝜏 𝛾 𝛿 S 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ Value computation algorithm (example) πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ A πœ• 3 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt )

  49. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ … πœ€ πŸ™ πŸ™ ( B A S 𝜏 𝛾 𝛿 S 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ Value computation algorithm (example) πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ A πœ• 3 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt )

  50. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ … πœ€ πŸ™ πŸ™ ( B A S 𝜏 𝛾 𝛿 S 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ Value computation algorithm (example) πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ A πœ• 3 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt )

  51. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ … πœ€ πŸ™ πŸ™ ( B A S 𝜏 𝛾 𝛿 S 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ Value computation algorithm (example) πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ A πœ• 3 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt )

  52. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ … 𝜏 πŸ™ πŸ™ πŸ™ ( B A S S πœ€ 𝛿 𝛾 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ ) Value computation algorithm (example) πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) A βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S β†’ πœ€( B ) πœ• 3 βˆ’ βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt )

  53. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) = ( ) ↦ ( ) = ( ) ↦ … πœ€ πŸ™ πŸ™ ( B A S 𝜏 𝛾 𝛿 S πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ Value computation algorithm (example) πœ• 3 () 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ ) βˆ’ βˆ’ βˆ’ S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) A βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S β†’ πœ€( B ) πœ• 3 20 / 21 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ (

  54. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ↦ ( ) = ( ) ↦ … Value computation algorithm (example) 𝛿 πŸ™ ( B A S 𝜏 πœ€ 𝛾 S πŸ™ 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ 𝕝 1 πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) ) 𝕝 3 𝕝 2 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) πœ• 3 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S β†’ πœ€( B ) A 20 / 21 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = (

  55. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ↦ ( ) = ( ) ↦ … Value computation algorithm (example) 𝛿 πŸ™ ( B A S 𝜏 πœ€ 𝛾 S πŸ™ 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ 𝕝 1 πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) ) 𝕝 3 𝕝 2 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) πœ• 3 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S β†’ πœ€( B ) A 20 / 21 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = (

  56. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ↦ ( ) = ( ) ↦ … Value computation algorithm (example) 𝛿 πŸ™ ( B A S 𝜏 πœ€ 𝛾 S πŸ™ 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ 𝕝 1 πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) ) 𝕝 3 𝕝 2 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) πœ• 3 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S β†’ πœ€( B ) A 20 / 21 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = (

  57. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ↦ ( ) = ( ) ↦ … Value computation algorithm (example) 𝛿 πŸ™ ( B A S 𝜏 πœ€ 𝛾 S πŸ™ 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ 𝕝 1 πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) ) 𝕝 3 𝕝 2 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) πœ• 3 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S β†’ πœ€( B ) A 20 / 21 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = (

  58. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ↦ ( ) = ( ) ↦ … Value computation algorithm (example) 𝛿 πŸ™ ( B A S 𝜏 πœ€ 𝛾 S πŸ™ 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ 𝕝 1 πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) ) 𝕝 3 𝕝 2 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) πœ• 3 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S β†’ πœ€( B ) A 20 / 21 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = (

  59. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) = ( ) ↦ … Value computation algorithm (example) S πŸ™ ( B A S 𝜏 πœ€ 𝛿 𝛽 𝛾 πŸ™ B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ πŸ™ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 1 ) βŠ• πœ• 5 () πœ• 3 () 2 , 𝕝 β€² πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ βˆ’ βˆ’ 20 / 21 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) A βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S πœ• 5 β†’ πœ€( B ) B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 πœ• 3 βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ B βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = ( ) ↦ (

  60. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) Value computation algorithm (example) S πŸ™ ( B A S 𝜏 πœ€ 𝛿 𝛽 𝛾 πŸ™ B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ πŸ™ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) ↦ … ) 3 𝕝 β€² 2 𝕝 β€² 𝕝 β€² πœ• 3 () 1 ) βŠ• πœ• 5 () 2 , 𝕝 β€² πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ βˆ’ βˆ’ 20 / 21 β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ πœ• 5 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ β†’ πœ€( B ) A πœ• 4 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = ( ) ↦ ( ) = (

  61. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) Value computation algorithm (example) S πŸ™ ( B A S 𝜏 πœ€ 𝛿 𝛽 𝛾 πŸ™ B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ πŸ™ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) ↦ … ) 3 𝕝 β€² 2 𝕝 β€² 𝕝 β€² πœ• 3 () 1 ) βŠ• πœ• 5 () 2 , 𝕝 β€² πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ βˆ’ βˆ’ 20 / 21 β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ πœ• 5 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ β†’ πœ€( B ) A πœ• 4 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = ( ) ↦ ( ) = (

  62. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) Value computation algorithm (example) S πŸ™ ( B A S 𝜏 πœ€ 𝛿 𝛽 𝛾 πŸ™ B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ πŸ™ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) ↦ … ) 3 𝕝 β€² 2 𝕝 β€² 𝕝 β€² πœ• 3 () 1 ) βŠ• πœ• 5 () 2 , 𝕝 β€² πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ βˆ’ βˆ’ 20 / 21 β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ πœ• 5 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ β†’ πœ€( B ) A πœ• 4 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = ( ) ↦ ( ) = (

  63. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) Value computation algorithm (example) S πŸ™ ( B A S 𝜏 πœ€ 𝛿 𝛽 𝛾 πŸ™ B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ πŸ™ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) ↦ … ) 3 𝕝 β€² 2 𝕝 β€² 𝕝 β€² πœ• 3 () 1 ) βŠ• πœ• 5 () 2 , 𝕝 β€² πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ βˆ’ βˆ’ 20 / 21 β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ πœ• 5 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ β†’ πœ€( B ) A πœ• 4 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = ( ) ↦ ( ) = (

  64. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) Value computation algorithm (example) S πŸ™ ( B A S 𝜏 πœ€ 𝛿 𝛽 𝛾 πŸ™ B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ πŸ™ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) ↦ … ) 3 𝕝 β€² 2 𝕝 β€² 𝕝 β€² πœ• 3 () 1 ) βŠ• πœ• 5 () 2 , 𝕝 β€² πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ βˆ’ βˆ’ 20 / 21 β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ πœ• 5 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ β†’ πœ€( B ) A πœ• 4 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = ( ) ↦ ( ) = (

  65. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) Value computation algorithm (example) 𝛾 ( B A S 𝜏 πœ€ 𝛿 S B 𝛽 πŸ™ A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πŸ™ πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) πŸ™ 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 5 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () βˆ’ βˆ’ B β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ 20 / 21 A πœ• 3 β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  66. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B πŸ™ A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( πŸ™ B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πŸ™ πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) πœ• 5 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B πœ• 4 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  67. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B πŸ™ A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( πŸ™ B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πŸ™ πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) πœ• 5 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B πœ• 4 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  68. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B πŸ™ A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( πŸ™ B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πŸ™ πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) πœ• 5 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B πœ• 4 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  69. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B πŸ™ A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( πŸ™ B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πŸ™ πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) πœ• 5 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B πœ• 4 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  70. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B πŸ™ A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( πŸ™ B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πŸ™ πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 0.1 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B 0.7β‹… 𝕝 1 β‹… 𝕝 2 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  71. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B 0 A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( 0 B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 0.1 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B 0.7β‹… 𝕝 1 β‹… 𝕝 2 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  72. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B 0 A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( 0 B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () 0.8 β‹… 0 max 0.1 β‹… 0 0.1 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B 0.7β‹… 𝕝 1 β‹… 𝕝 2 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  73. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B 0 A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( 0 B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () 0 0.1 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B 0.7β‹… 𝕝 1 β‹… 𝕝 2 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  74. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B 0 A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( 0 B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () 0.5 0 0.1 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B 0.7β‹… 𝕝 1 β‹… 𝕝 2 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  75. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B 0 A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( 0 B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 0.7 β‹… 0.5 β‹… 0 max 0.1 0.5 0 0.1 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B 0.7β‹… 𝕝 1 β‹… 𝕝 2 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  76. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B 0 A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( 0 B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 0.1 0.5 0 0.1 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B 0.7β‹… 𝕝 1 β‹… 𝕝 2 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  77. Value computation algorithm (example) B S 𝜏 πœ€ 𝛿 S 𝛾 𝛽 A B β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A ( β†’ 𝜏( A , S ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 0.1 0.5 0 0 0 0.1 B βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.7β‹… 𝕝 1 β‹… 𝕝 2 A β†’ 𝛽 B βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) ↦ ( ) = ( ) ↦ …

  78. Value computation algorithm (example) B S 𝜏 πœ€ 𝛿 S 𝛾 𝛽 A B β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A ( β†’ 𝜏( A , S ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () 0.8 β‹… 0.5 max 0.1 β‹… 0.1 0.1 0.5 0 0 0 0.1 B βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.7β‹… 𝕝 1 β‹… 𝕝 2 A β†’ 𝛽 B βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) ↦ ( ) = ( ) ↦ …

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

TDT4205 Lecture 07 2 Parsing by recursive descent Take this grammar which models

TDT4205 Lecture 07 2 Parsing by recursive descent Take this grammar which models

1 Top-down parsing and LL(1) parser construction TDT4205 Lecture 07 2 Parsing by recursive descent Take this grammar which models ifs and whiles: P iCtSz | iCtSeSz | wCdSz C c S s Lets parse the

606 views β€’ 29 slides
Identifying Grammar Rules for Language Education with Dependency Parsing in German Eleni

Identifying Grammar Rules for Language Education with Dependency Parsing in German Eleni

Identifying Grammar Rules for Language Education with Dependency Parsing in German Eleni Metheniti, Pomi Park, Kristina Kolesova, Gnter Neumann August 27, 2019 Depling SyntaxFest Why identify grammar rules? Our larger mission: find

523 views β€’ 29 slides
CSE 447/547 Natural Language Processing Winter 2018 Dependency Parsing And Other Grammar

CSE 447/547 Natural Language Processing Winter 2018 Dependency Parsing And Other Grammar

CSE 447/547 Natural Language Processing Winter 2018 Dependency Parsing And Other Grammar Formalisms Yejin Choi - University of Washington Dependency Grammar For each word, find one parent. Child Parent A child is dependent on the

733 views β€’ 56 slides
CSE 490 U Natural Language Processing Spring 2016 Dependency Parsing And Other Grammar

CSE 490 U Natural Language Processing Spring 2016 Dependency Parsing And Other Grammar

CSE 490 U Natural Language Processing Spring 2016 Dependency Parsing And Other Grammar Formalisms Yejin Choi - University of Washington Dependency Grammar For each word, find one parent. Child Parent A child is dependent on the

814 views β€’ 55 slides
Tree-Adjoining Grammar Parsing and Vector Representations of Supertags Jungo Kasai Yale

Tree-Adjoining Grammar Parsing and Vector Representations of Supertags Jungo Kasai Yale

Background and Motivations Supertagging Models Parsing Models Vector Representations of Supertags Ongoing TAG Parsing W Tree-Adjoining Grammar Parsing and Vector Representations of Supertags Jungo Kasai Yale University December 14, 2017

799 views β€’ 62 slides
CSE 517 Natural Language Processing Winter 2017 Dependency Parsing And Other Grammar Formalisms

CSE 517 Natural Language Processing Winter 2017 Dependency Parsing And Other Grammar Formalisms

CSE 517 Natural Language Processing Winter 2017 Dependency Parsing And Other Grammar Formalisms Yejin Choi - University of Washington Dependency Grammar For each word, find one parent. Child Parent A child is dependent on the

1.83k views β€’ 55 slides
Natural Language Processing Lecture 13: More on CFG Parsing Probabilistjc/Weighted Parsing

Natural Language Processing Lecture 13: More on CFG Parsing Probabilistjc/Weighted Parsing

Natural Language Processing Lecture 13: More on CFG Parsing Probabilistjc/Weighted Parsing Example: ambiguous parse Probabilistjc CFG Ambiguous parse w/probabilitjes 0.05 0.05 0.20 0.10 0.30 0.20 0.20 0.30 0.20 0.20 0.60 0.60 0.75

1.05k views β€’ 68 slides
1 Determinism and Parsing The parsing problem is, given a string w and a context-free grammar G ,

1 Determinism and Parsing The parsing problem is, given a string w and a context-free grammar G ,

1 Determinism and Parsing The parsing problem is, given a string w and a context-free grammar G , to decide if w L ( G ), and if so, to construct a derivation or a parse tree for w . Parsing is studied in courses in compilers. To be efficient

519 views β€’ 12 slides
Natural Language Processing Other Syntactic Models Parsing IV Dan Klein UC Berkeley Dependency

Natural Language Processing Other Syntactic Models Parsing IV Dan Klein UC Berkeley Dependency

Natural Language Processing Other Syntactic Models Parsing IV Dan Klein UC Berkeley Dependency Parsing Dependency Parsing Pure dependency parsing is only cubic [Eisner 99] Lexicalized parsers can be seen as producing dependency trees

314 views β€’ 5 slides
Parsing beyond context-free grammar: necessarily adjacent. Range Concatenation Grammar Parsing

Parsing beyond context-free grammar: necessarily adjacent. Range Concatenation Grammar Parsing

Kallmeyer/Maier ESSLLI 2008 Kallmeyer/Maier ESSLLI 2008 Range Concatenation Grammar The idea behind range concatenation grammar (RCG) is comparable to the idea behind MCFG. Predicate-rewriting clauses describe ranges which are not Parsing

396 views β€’ 10 slides
Lagrangian Based Approaches for Lexicalized Tree Adjoining Grammar Parsing Caio Corro

Lagrangian Based Approaches for Lexicalized Tree Adjoining Grammar Parsing Caio Corro

Lagrangian Based Approaches for Lexicalized Tree Adjoining Grammar Parsing Caio Corro Supervision: Adeline Nazarenko & Joseph Le Roux 9 march 2018 1 / 51 Syntax: description of structures in natural languages She walks the woman dog

1.77k views β€’ 162 slides
Advanced Natural Language Processing Guest Lecture: Modeling Human Parsing Frank Keller

Advanced Natural Language Processing Guest Lecture: Modeling Human Parsing Frank Keller

Introduction Prediction and Grammar Predictive Parsing Evaluation Advanced Natural Language Processing Guest Lecture: Modeling Human Parsing Frank Keller Institute for Language, Cognition and Computation School of Informatics, University of

683 views β€’ 47 slides
CSE 3341: Principles of Programming Languages Recursive Descent Parsing Jeremy Morris 1

CSE 3341: Principles of Programming Languages Recursive Descent Parsing Jeremy Morris 1

CSE 3341: Principles of Programming Languages Recursive Descent Parsing Jeremy Morris 1 Parsing A grammar is a generator for a language The rules tell us how to create strings in the language A parser is a recognizer for a language

798 views β€’ 32 slides
Top-Down Parsing Top-Down Parsing #1 Extra Credit Question Given this grammar G: E

Top-Down Parsing Top-Down Parsing #1 Extra Credit Question Given this grammar G: E

Top-Down Parsing Top-Down Parsing #1 Extra Credit Question Given this grammar G: E E + T E T T T * int T int T ( E ) Is the string int * (int + int) in L(G)? Give a derivation or prove that it

1.08k views β€’ 61 slides
Statistical Natural Language Parsing Gerald Penn [based on slides by Christopher Manning]

Statistical Natural Language Parsing Gerald Penn [based on slides by Christopher Manning]

Statistical Natural Language Parsing Gerald Penn [based on slides by Christopher Manning] Parsing examples Fed raises interest rates Parsing in the early 1990s The parsers produced detailed, linguistically rich representations

849 views β€’ 46 slides
Efficient parsing with a large-scale unification-based grammar Lessons from a multi-year,

Efficient parsing with a large-scale unification-based grammar Lessons from a multi-year,

0. Efficient parsing with a large-scale unification-based grammar Lessons from a multi-year, multi-team endeavour Liviu Ciortuz Department of Computer Science University of Iasi, Romania ALEAR Workshop, FP7 E.U. Project Humbold

947 views β€’ 76 slides
INTERNATIONAL SEM INAR ON HORTICULTURE TO SUPPORT FOOD SECURITY 2010 SCHEDULE of PARALLEL SESSION

INTERNATIONAL SEM INAR ON HORTICULTURE TO SUPPORT FOOD SECURITY 2010 SCHEDULE of PARALLEL SESSION

INTERNATIONAL SEM INAR ON HORTICULTURE TO SUPPORT FOOD SECURITY 2010 SCHEDULE of PARALLEL SESSION Group A: Horticultural Biology and Physiology FIRST AUTHOR's TIM E NO TITLES SPEAKERS INSTITUTION Tuesday - J une 22, 2010 Quality variation

326 views β€’ 7 slides
The Digital Revolution: The move from Teaching to Learning Revolution Compare Education Values

The Digital Revolution: The move from Teaching to Learning Revolution Compare Education Values

The Digital Revolution: The move from Teaching to Learning Revolution Compare Education Values 1976 2012 V alues The purpose of education Expectations of teaching Assessment of learning The

1.78k views β€’ 33 slides
PHP Angel R. Roman Keyla Perez Velez Michael Voss Julian Richen Group 12 Questions to be

PHP Angel R. Roman Keyla Perez Velez Michael Voss Julian Richen Group 12 Questions to be

PHP Angel R. Roman Keyla Perez Velez Michael Voss Julian Richen Group 12 Questions to be addressed + History + Development + Versioning + Compatibility + Usage Introduction PHP is a recursive acronym for "PHP: Hypertext

752 views β€’ 31 slides
Evergreen Inventory From Start to Finish Introduction Presenters CUTPL Information

Evergreen Inventory From Start to Finish Introduction Presenters CUTPL Information

Evergreen Inventory From Start to Finish Introduction Presenters CUTPL Information Technology Manager Andrew Baker CUTPL Circulation Manager Teresa Hudson Thank you to our Library Director, Colleen McCarty for being our

904 views β€’ 62 slides
PostGIS Feature Frenzy!!! Wednesday, September 16, 15 frenzy because state of postgis

PostGIS Feature Frenzy!!! Wednesday, September 16, 15 frenzy because state of postgis

P AUL R AMSEY PRESENTS .... PostGIS Feature Frenzy!!! Wednesday, September 16, 15 frenzy because state of postgis talks only covered leading edge, nothing covering all the good features already there frenzy because our standard 20-25

1.68k views β€’ 142 slides
HANNAH HOWARD #ABOUTME Personal hannah@carbon Anecdote: I have @techgirlwonder ve.com a

HANNAH HOWARD #ABOUTME Personal hannah@carbon Anecdote: I have @techgirlwonder ve.com a

HANNAH HOWARD #ABOUTME Personal hannah@carbon Anecdote: I have @techgirlwonder ve.com a dog she/her REACTIVE PROGRAMMING: A Better Way to Write Frontend Applications 1. PROBLEM STATEMENT WHAT IS A COMPUTER PROGRAM? A computer

923 views β€’ 81 slides
GMO and its impact on our modern society By Jaeseung Shin, Jevon Suharnoko What is GMO?

GMO and its impact on our modern society By Jaeseung Shin, Jevon Suharnoko What is GMO?

GMO and its impact on our modern society By Jaeseung Shin, Jevon Suharnoko What is GMO? Definition: Genetic Engineering Selective Breeding Three aspects of GMOs Biological Economic Used in plants to boost yield, resistant

140 views β€’ 10 slides
2016 MERGER BY INCORPORATION of Centrale del Latte di Firenze, Pistoia e Livorno S,p,A, into

2016 MERGER BY INCORPORATION of Centrale del Latte di Firenze, Pistoia e Livorno S,p,A, into

PRESENTATION FOR ANALYSTS Star Conference 2017 Milan, 21 March 2017 2016 MERGER BY INCORPORATION of Centrale del Latte di Firenze, Pistoia e Livorno S,p,A, into Centrale del Latte di Torino & C, S,p,A, FY 2016 was a moment of significant

908 views β€’ 44 slides

More recommend