natural language generation
play

Natural Language Generation . .. . . .. .. . .. . . . .. - PowerPoint PPT Presentation

Mar 23, 2024 •500 likes •1.65k views

. .. . . .. . . .. . . .. . . .. . . . . Ondej Duek Ondej Duek 1/ 40 May 14 th , 2015 Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics for Spoken Dialogue

  1. • Paiva&Evans : linguistic features annotated in corpus generated with • PERSONAGE-PE : personality traits connected to linguistic features via . . . .. . . .. .. . .. . . .. . . . . . . .. . . .. . Example NLG Systems Sentence planning Trainable Sentence Planning: Parameter Optimization Examples many parameter settings, correlation analysis machine learning 9/ 40 Ondřej Dušek .. .. . . . . .. . . .. . . .. . . .. . . .. .. . .. . . .. . . .. . Natural Language Generation . .. . . .. . . • Requires a flexible handcrafed planner • No overgeneration • Adjusting its parameters “somehow”

  2. . . .. . . .. . . .. . . .. . . .. . .. . Sentence planning Ondřej Dušek 9/ 40 machine learning many parameter settings, correlation analysis Examples Trainable Sentence Planning: Parameter Optimization Example NLG Systems . . .. . . .. . . .. .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . .. . Natural Language Generation . .. . . .. . . .. • Requires a flexible handcrafed planner • No overgeneration • Adjusting its parameters “somehow” • Paiva&Evans : linguistic features annotated in corpus generated with • PERSONAGE-PE : personality traits connected to linguistic features via

  3. • General purpose • Functional Unification . .. . . .. . . . . . .. .. . .. . .. .. . KPML Ondřej Dušek 10/ 40 Grammar FUF/SURGE Grammar multilingual Grammar-based Realizers (90's): KPML , FUF/SURGE .. Surface Realization Example NLG Systems . .. . . . . . . . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . . .. .. Natural Language Generation . . .. . . .. . (EXAMPLE :NAME EX-SET-1 :TARGETFORM "It is raining cats and dogs." :LOGICALFORM • General purpose, (A / AMBIENT-PROCESS :LEX RAIN :TENSE PRESENT-CONTINUOUS :ACTEE (C / OBJECT :LEX CATS-AND-DOGS :NUMBER MASS)) ) • Systemic Functional

  4. . . . . .. . . .. . . .. .. . .. . . .. . . KPML Ondřej Dušek 10/ 40 Grammar FUF/SURGE Grammar multilingual Grammar-based Realizers (90's): KPML , FUF/SURGE .. Surface Realization Example NLG Systems . .. . . .. . . . . . .. . .. .. . . .. . . .. . . .. . . .. . . .. .. . . Natural Language Generation . . . .. . . . .. (EXAMPLE :NAME EX-SET-1 :TARGETFORM "It is raining cats and dogs." :LOGICALFORM • General purpose, (A / AMBIENT-PROCESS :LEX RAIN :TENSE PRESENT-CONTINUOUS :ACTEE (C / OBJECT :LEX CATS-AND-DOGS :NUMBER MASS)) ) • Systemic Functional • General purpose • Functional Unification

  5. . . .. . . .. . . .. . . .. . . .. . .. . Surface Realization Ondřej Dušek 11/ 40 enhancements Grammar multi-lingual Grammar-based Realizer: OpenCCG Example NLG Systems . . .. . . .. . .. . .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . .. . . .. . . .. . . .. Natural Language Generation • General purpose, • Combinatory Categorial • Used in several projects • With statistical

  6. . . .. . . .. .. . .. . . .. . . .. . .. . . . .. . . .. . Example NLG Systems Surface Realization Procedural Realizer: SimpleNLG other languages (procedural) 12/ 40 Ondřej Dušek . . .. .. . . .. . . .. . . . . . .. . . .. . . . .. . . . .. Natural Language Generation . . . . .. . .. .. Lexicon lexicon = new XMLLexicon("my-lexicon.xml"); • General purpose NLGFactory nlgFactory = new NLGFactory(lexicon); Realiser realiser = new Realiser(lexicon); • English, adapted to several SPhraseSpec p = nlgFactory.createClause(); p.setSubject("Mary"); p.setVerb("chase"); • Java implementation p.setObject("the monkey"); p.setFeature(Feature.TENSE, Tense.PAST); String output = realiser.realiseSentence(p); System.out.println(output); >>> Mary chased the monkey.

  7. • Ranking according to: • n -gram models ( NITROGEN, HALOGEN ) • Tree models (XTAG grammar – FERGUS ) • Predicted Text-to-Speech quality ( Nakatsu and White ) • Personality traits (extraversion, agreeableness… – CRAG ) • Provides variance, but at a greater computational cost . .. . . .. . . .. . . . .. . . .. .. . . .. . . .. . Example NLG Systems Surface Realization Trainable Realizers: Overgenerate and Rank + alignment (repeating words uttered by dialogue counterpart) 13/ 40 Ondřej Dušek .. . . . . . .. . . .. . . .. . . .. . . . .. . . .. . . .. . . .. . Natural Language Generation .. .. . . .. . . • Require a handcrafued realizer, e.g. CCG realizer • Input underspecified → more outputs possible • Overgenerate • Then use a statistical reranker

  8. • Provides variance, but at a greater computational cost . . .. .. . .. . . .. . . .. . . .. .. . . . . .. . . .. . Example NLG Systems Surface Realization Trainable Realizers: Overgenerate and Rank + alignment (repeating words uttered by dialogue counterpart) 13/ 40 Ondřej Dušek . . .. .. . . .. . . .. . . .. . . . . . .. . . . .. . . . .. Natural Language Generation . .. . . .. . . .. • Require a handcrafued realizer, e.g. CCG realizer • Input underspecified → more outputs possible • Overgenerate • Then use a statistical reranker • Ranking according to: • n -gram models ( NITROGEN, HALOGEN ) • Tree models (XTAG grammar – FERGUS ) • Predicted Text-to-Speech quality ( Nakatsu and White ) • Personality traits (extraversion, agreeableness… – CRAG )

  9. . . . .. .. . .. . . .. . . .. . . .. . .. .. . . .. . . .. . Example NLG Systems Surface Realization Trainable Realizers: Overgenerate and Rank + alignment (repeating words uttered by dialogue counterpart) 13/ 40 Ondřej Dušek . . . .. . . .. . . .. . . .. . . .. . . . . . .. . . . .. . Natural Language Generation .. . . .. .. . . • Require a handcrafued realizer, e.g. CCG realizer • Input underspecified → more outputs possible • Overgenerate • Then use a statistical reranker • Ranking according to: • n -gram models ( NITROGEN, HALOGEN ) • Tree models (XTAG grammar – FERGUS ) • Predicted Text-to-Speech quality ( Nakatsu and White ) • Personality traits (extraversion, agreeableness… – CRAG ) • Provides variance, but at a greater computational cost

  10. . .. .. .. . . .. . . .. . . .. . . . .. . .. . . .. . . .. . Example NLG Systems Surface Realization Trainable Realizers: Syntax-Based 14/ 40 Ondřej Dušek . . . .. . . .. . . .. . . .. . . .. . . . . . .. . . . . .. .. . . .. . . .. Natural Language Generation • StuMaBa : general realizer based on SVMs • Pipeline: ↓ Deep syntax/semantics ↓ surface syntax ↓ linearization ↓ morphologization

  11. • Most common, also in commercial NLG systems • Simple, straightforward, reliable (custom-tailored for domain) • Lack generality and variation, difficult to maintain • Enhancements for more complex utterances: rules . . . .. . . .. . . .. .. . .. . . .. . . .. . . .. . Example NLG Systems Holistic NLG Approaches Holistic NLG Holistic NLG Template-based systems 15/ 40 Ondřej Dušek . .. . . . . .. . . .. . . .. . . .. . . .. . . .. .. . . .. . . .. Natural Language Generation . . .. . . .. . . • Only one stage – no distinction • “Good enough” for limited domains, also in SDS

  12. . . .. . . .. . . .. . . .. . . .. . .. . . . .. . . .. . Example NLG Systems Holistic NLG Approaches Holistic NLG Holistic NLG Template-based systems 15/ 40 Ondřej Dušek . .. .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . . . .. .. . . .. . . .. Natural Language Generation • Only one stage – no distinction • “Good enough” for limited domains, also in SDS • Most common, also in commercial NLG systems • Simple, straightforward, reliable (custom-tailored for domain) • Lack generality and variation, difficult to maintain • Enhancements for more complex utterances: rules

  13. . . .. . . .. . . .. . . .. . . .. . .. . Holistic NLG Approaches Ondřej Dušek 16/ 40 Facebook templates domain) Alex (English restaurant Example: Templates Example NLG Systems . . .. . . .. . . .. .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . .. . Natural Language Generation . . . .. . . .. .. • Just filling variables into slots • Possibly a few enhancements, e. g. articles inform(pricerange="{pricerange}"): 'It is in the {pricerange} price range.' affirm()&inform(task="find") &inform(pricerange="{pricerange}"): 'Ok, you are looking for something in the' + ' {pricerange} price range.' affirm()&inform(area="{area}"): 'Ok, you want something in the {area} area.' affirm()&inform(food="{food}") &inform(pricerange="{pricerange}"): 'Ok, you want something with the {food} food' + ' in the {pricerange} price range.' inform(food="None"): 'I do not have any information' + ' about the type of food.'

  14. • BAGEL : Bayesian networks • semantic stacks, ordering • Angeli et al. : log-linear model • records • WASP • noisy channel, similar to MT .. . . .. . . . .. . . . .. . . . .. .. . . . . Example NLG Systems Holistic NLG Approaches Statistical Holistic NLG (typically: MR + sentence + alignment) Examples fields templates 1 : Synchronous CFGs 17/ 40 Ondřej Dušek .. .. . .. .. . . .. . . . .. . .. . . .. . . . . .. .. . . .. . . .. . . . . . .. . . .. . Natural Language Generation • Limited domain • Based on supervised learning • Typically: phrase-based

  15. . . .. .. . .. . . .. . . .. . . .. . .. . . . .. . . .. . Example NLG Systems Holistic NLG Approaches Statistical Holistic NLG (typically: MR + sentence + alignment) Examples 17/ 40 Ondřej Dušek . . .. .. . . .. . . .. . . .. . . . . . .. . . . .. . . . .. Natural Language Generation . . . . .. .. . .. • Limited domain • Based on supervised learning • Typically: phrase-based • BAGEL : Bayesian networks • semantic stacks, ordering • Angeli et al. : log-linear model • records ↘ fields ↘ templates • WASP − 1 : Synchronous CFGs • noisy channel, similar to MT

  16. • Our generator learns alignments jointly • (with sentence planning) • training from pairs: MR + sentence . .. . . .. . . .. . . . . . .. .. .. .. . .. . . .. . Our System Overview Our experiments: Two-Step NLG for SDS Learning from unaligned data a) requires detailed alignments of MR elements and words/phrases b) uses a separate alignment step 18/ 40 Ondřej Dušek . . . . . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . .. .. . . .. . . .. . . Natural Language Generation • Typical NLG training: MR inform(name=X, type=placetoeat, eattype=restaurant, area=riverside, food=Italian) alignment X is an italian restaurant in the riverside area . text

  17. . .. . .. .. . . .. . . .. . . .. . . . . Our experiments: Two-Step NLG for SDS Ondřej Dušek 18/ 40 b) uses a separate alignment step of MR elements and words/phrases a) requires detailed alignments Learning from unaligned data Overview . Our System . .. . . .. .. . . . . .. .. . . .. . . .. . . .. . . .. . . . . . .. . .. .. . . .. . . .. . . Natural Language Generation • Typical NLG training: • Our generator learns alignments jointly • (with sentence planning) • training from pairs: MR + sentence MR inform(name=X, type=placetoeat, eattype=restaurant, area=riverside, food=Italian) X is an italian restaurant in the riverside area . text

  18. • Step 1. – sentence planning • statistical, our main focus • Sentence plan : deep-syntax dependency trees • based on TectoMT 's t-layer, but very simplified • two attributes per tree node: t-lemma + formeme • using surface word order • Step 2. – surface realization • reusing Treex / TectoMT English synthesis (rule-based) • Output : plain text sentence . .. . . .. . . .. . . . .. . . .. . . .. . . .. . Our System Workflow / data formats Overall workflow of our generator 19/ 40 Ondřej Dušek .. . .. .. .. . . .. . . . . . .. . . .. . . . . .. . . .. . . .. . . .. . . .. . . .. . Natural Language Generation • Input : a MR • here – dialogue acts: “inform” + slot-value pairs • other formats possible

  19. • Sentence plan : deep-syntax dependency trees • based on TectoMT 's t-layer, but very simplified • two attributes per tree node: t-lemma + formeme • using surface word order • Step 2. – surface realization • reusing Treex / TectoMT English synthesis (rule-based) • Output : plain text sentence . .. . . .. . . .. . . . .. .. . .. .. . . .. . . .. . Our System Workflow / data formats Overall workflow of our generator 19/ 40 Ondřej Dušek . . . . . . .. . . .. . . .. . . .. . . . .. . . .. . . .. . . .. . Natural Language Generation .. .. . . .. . . • Input : a MR • here – dialogue acts: “inform” + slot-value pairs • other formats possible • Step 1. – sentence planning • statistical, our main focus

  20. • Output : plain text sentence . .. . .. . .. .. . . .. . . .. . . . . .. .. . . .. . . .. . Our System Workflow / data formats Overall workflow of our generator 19/ 40 Ondřej Dušek . . . .. . . .. . . .. . . .. . . .. . . . . . .. . . . .. . Natural Language Generation .. . . .. .. . . • Input : a MR • here – dialogue acts: “inform” + slot-value pairs • other formats possible • Step 1. – sentence planning • statistical, our main focus • Sentence plan : deep-syntax dependency trees • based on TectoMT 's t-layer, but very simplified • two attributes per tree node: t-lemma + formeme • using surface word order • Step 2. – surface realization • reusing Treex / TectoMT English synthesis (rule-based)

  21. . .. . .. . .. .. . . .. . . .. . . . .. . .. . . .. . . .. . Our System Workflow / data formats Overall workflow of our generator 19/ 40 Ondřej Dušek . . . .. . . .. . . .. . . .. . . .. . . . . . .. . . . .. . Natural Language Generation .. . . .. . .. . • Input : a MR • here – dialogue acts: “inform” + slot-value pairs • other formats possible • Step 1. – sentence planning • statistical, our main focus • Sentence plan : deep-syntax dependency trees • based on TectoMT 's t-layer, but very simplified • two attributes per tree node: t-lemma + formeme • using surface word order • Step 2. – surface realization • reusing Treex / TectoMT English synthesis (rule-based) • Output : plain text sentence

  22. . .. . .. . . .. .. . .. . . .. . . . .. . .. . . .. . . .. . Our System Workflow / data formats Data structures used 20/ 40 Ondřej Dušek . . . .. . . .. . . .. . . .. . . .. . . . . . .. . . . .. . Natural Language Generation .. . . .. .. . . inform(name=X, type=placetoeat, eattype=restaurant, area=riverside, food=Italian) t-tree be v:fin X-name restaurant n:subj n:obj italian area adj:attr n:in+X riverside n:attr X is an italian restaurant in the riverside area .

  23. • The 2nd step – rule based – can ensure grammatical • or at least it's more straightforward to fix when it doesn't • The realizer is (relatively) easy to implement and • + why not use it if we have it already in Treex / TectoMT . . .. . . .. . . .. .. . . .. . . .. . . .. . . .. . Our System Two-step architecture Why we keep the two-step approach correctness domain-independent 21/ 40 Ondřej Dušek . .. . . . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . . .. Natural Language Generation .. . .. . . .. . . • It makes the 1st – statistical – task simpler • no need to worry about morphology • this will be more important for Czech (and similar)

  24. • The realizer is (relatively) easy to implement and • + why not use it if we have it already in Treex / TectoMT . . .. . . .. . . .. . . .. . . . . .. . . .. . . .. . Our System Two-step architecture Why we keep the two-step approach correctness domain-independent 21/ 40 Ondřej Dušek .. .. . . .. . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. Natural Language Generation . . .. . . .. . . • It makes the 1st – statistical – task simpler • no need to worry about morphology • this will be more important for Czech (and similar) • The 2nd step – rule based – can ensure grammatical • or at least it's more straightforward to fix when it doesn't

  25. . . .. . . .. . . .. . . .. . . .. . .. . . . .. . . .. . Our System Two-step architecture Why we keep the two-step approach correctness domain-independent 21/ 40 Ondřej Dušek . .. .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . . .. Natural Language Generation . .. . . .. . . .. • It makes the 1st – statistical – task simpler • no need to worry about morphology • this will be more important for Czech (and similar) • The 2nd step – rule based – can ensure grammatical • or at least it's more straightforward to fix when it doesn't • The realizer is (relatively) easy to implement and • + why not use it if we have it already in Treex / TectoMT

  26. • but we can do it easily using Treex • automatic annotation is good enough . .. . . .. . . . . . .. . . .. .. .. . .. . . .. . . .. . Our System Two-step architecture Downside of the two-step approach – t-layer analysis implemented for several languages 22/ 40 Ondřej Dušek . . .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . .. . . .. . . .. . . .. Natural Language Generation • We need to analyze training sentences into deep trees

  27. . . . .. . . .. . . .. . . .. . . .. . .. .. . . .. . . .. . Our System Two-step architecture Downside of the two-step approach – t-layer analysis implemented for several languages 22/ 40 Ondřej Dušek .. . . .. . . .. . . .. . . .. . . .. . . . . . .. . . .. . . .. . . .. . . .. Natural Language Generation • We need to analyze training sentences into deep trees • but we can do it easily using Treex • automatic annotation is good enough

  28. • A*-style search • incrementally finding the path • from an empty tree • to a full sentence plan tree which contains all information • using open_set , close_set – heaps sorted by score . . .. . . .. . . .. . . .. . . .. .. . . .. . . .. . Our System Sentence planner Sentence planner – overall – churning out more and more sentence plan trees 23/ 40 Ondřej Dušek . .. . . . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . . .. Natural Language Generation .. . .. . . .. . . • Two main components: • candidate generator : • scorer /ranker for the candidates

  29. • using open_set , close_set – heaps sorted by score . . .. . . .. . . .. . . .. . . .. .. . . . . .. . . .. . Our System Sentence planner Sentence planner – overall – churning out more and more sentence plan trees 23/ 40 Ondřej Dušek . .. .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . . .. Natural Language Generation . .. . . .. . . .. • Two main components: • candidate generator : • scorer /ranker for the candidates • A*-style search • incrementally finding the path • from an empty tree • to a full sentence plan tree which contains all information

  30. . . . .. . . .. . . .. . . .. . . .. . .. .. . . .. . . .. . Our System Sentence planner Sentence planner – overall – churning out more and more sentence plan trees 23/ 40 Ondřej Dušek . .. . .. . . .. . . .. . . .. . . . . . .. . . .. . . . .. . Natural Language Generation .. . . .. . . .. • Two main components: • candidate generator : • scorer /ranker for the candidates • A*-style search • incrementally finding the path • from an empty tree • to a full sentence plan tree which contains all information • using open_set , close_set – heaps sorted by score

  31. • Loop: open_set close_set • viable trees, C + some node(s) • C may be empty open_set 4. check if top score( open_set ) top score( close_set ) • Stop if: a) close_set has better top score than open_set .. . . .. . . . .. Our System . .. . . .. . . Sentence planner Sentence planner – workflow 1. get top-scoring C . put C 2. C candidate generator successors( C ) 3. score C C C put C for d consecutive iterations b) there's nothing lefu on the open list (unlikely) 24/ 40 Ondřej Dušek . .. .. . .. . . .. . . .. . .. . . . .. . . .. . . . .. . . . .. . . .. . . .. . . .. . . .. . . .. . Natural Language Generation • Init: open_set = {empty tree}, close_set = ∅

  32. • viable trees, C + some node(s) • C may be empty open_set 4. check if top score( open_set ) top score( close_set ) • Stop if: a) close_set has better top score than open_set . .. . . .. . . . .. . . .. . . .. . Sentence planner .. . Our System . Sentence planner – workflow 2. C candidate generator successors( C ) 3. score C C C put C for d consecutive iterations b) there's nothing lefu on the open list (unlikely) 24/ 40 Ondřej Dušek .. . .. . . . .. . . .. . .. .. . . .. . . .. . . . Natural Language Generation . . . .. . . .. . . .. . .. . . . .. . . .. • Init: open_set = {empty tree}, close_set = ∅ • Loop: 1. get top-scoring C ← open_set put C → close_set

  33. open_set 4. check if top score( open_set ) top score( close_set ) • Stop if: a) close_set has better top score than open_set . . .. . . .. . . .. . .. . .. .. . . .. . . . . Our System Sentence planner Sentence planner – workflow 3. score C C C put C for d consecutive iterations b) there's nothing lefu on the open list (unlikely) 24/ 40 Ondřej Dušek .. . . .. .. . . .. . . . . . .. . . .. . . . .. .. . . . .. . . .. . . .. . .. . . . .. . Natural Language Generation • Init: open_set = {empty tree}, close_set = ∅ • Loop: 1. get top-scoring C ← open_set put C → close_set 2. C = candidate generator successors( C ) • viable trees, C + some node(s) • C may be empty

  34. 4. check if top score( open_set ) top score( close_set ) • Stop if: a) close_set has better top score than open_set . . .. . . .. . . .. .. . . .. . . .. . . .. . . .. . Our System Sentence planner Sentence planner – workflow for d consecutive iterations b) there's nothing lefu on the open list (unlikely) 24/ 40 Ondřej Dušek . .. . . . . .. . . .. . . .. . . .. . . .. . . .. .. . . .. . . .. . Natural Language Generation . .. . . .. . . • Init: open_set = {empty tree}, close_set = ∅ • Loop: 1. get top-scoring C ← open_set put C → close_set 2. C = candidate generator successors( C ) • viable trees, C + some node(s) • C may be empty 3. score C ′ ∀ C ′ ∈ C put C ′ → open_set

  35. . . .. . . .. . . .. . . .. . . .. . .. . . . .. . . .. . Our System Sentence planner Sentence planner – workflow for d consecutive iterations b) there's nothing lefu on the open list (unlikely) 24/ 40 Ondřej Dušek . .. .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . . .. Natural Language Generation . .. . . .. . . .. • Init: open_set = {empty tree}, close_set = ∅ • Loop: 1. get top-scoring C ← open_set put C → close_set 2. C = candidate generator successors( C ) • viable trees, C + some node(s) • C may be empty 3. score C ′ ∀ C ′ ∈ C put C ′ → open_set 4. check if top score( open_set ) > top score( close_set ) • Stop if: a) close_set has better top score than open_set

  36. . . . .. . . .. . . .. . . .. . . .. . .. .. . . .. . . .. . Our System Sentence planner Candidate generator by adding 1 node (at every possible place) 25/ 40 Ondřej Dušek . .. . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . .. . . . .. . . .. Natural Language Generation • Given a candidate plan tree, generate its successors

  37. . . .. . .. .. . . .. . . .. . . .. . .. . . . .. . . .. . Our System Sentence planner Candidate generator by adding 1 node (at every possible place) 25/ 40 Ondřej Dušek Natural Language Generation . . .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . .. . . .. . .. . . .. . • Given a candidate plan tree, generate its successors t-tree t-tree t-tree t-tree be recommend serve v:fin v:fin v:fin

  38. . . .. . .. .. . . .. . . .. . . .. . .. . . . .. . . .. . Our System Sentence planner Candidate generator by adding 1 node (at every possible place) 25/ 40 Ondřej Dušek Natural Language Generation . . .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . .. . . .. . .. . . .. . • Given a candidate plan tree, generate its successors t-tree t-tree t-tree t-tree be recommend serve v:fin v:fin v:fin

  39. . . .. . . .. . . .. . . .. . . .. . .. . . . .. . . .. . Our System Sentence planner Candidate generator by adding 1 node (at every possible place) 25/ 40 Ondřej Dušek Natural Language Generation . .. .. .. . . .. . . .. . . .. . . .. . . . . . . . .. . .. . . .. . . .. . . .. • Given a candidate plan tree, generate its successors t-tree t-tree t-tree t-tree be be be v:fin v:fin v:fin be v:fin restaurant X-name restaurant n:obj n:subj n:subj

  40. . . .. . . .. . . .. . . .. . . .. . .. . . . .. . . .. . Our System Sentence planner Candidate generator by adding 1 node (at every possible place) 25/ 40 Ondřej Dušek Natural Language Generation . .. .. .. . . .. . . .. . . .. . . .. . . . . . . . .. . .. . . .. . . .. . . .. • Given a candidate plan tree, generate its successors t-tree t-tree t-tree t-tree be be be v:fin v:fin v:fin be v:fin restaurant X-name restaurant n:obj n:subj n:subj

  41. . . .. . . .. . . .. . . .. . . .. . .. . . . .. . . .. . Our System Sentence planner Candidate generator by adding 1 node (at every possible place) 25/ 40 Ondřej Dušek Natural Language Generation . .. .. .. . . .. . . .. . . .. . . .. . . . . . . . .. . . . .. .. . . .. . . .. • Given a candidate plan tree, generate its successors t-tree t-tree t-tree be be be v:fin v:fin v:fin X-name restaurant X-name bar X-name n:subj n:obj n:subj n:obj n:subj

  42. • Limiting by things seen in training data: • + at depth levels • + given input MR • nodes seen with current slot-values • required slot-values for each node .. . . .. . . . . . . .. . . .. . .. . . .. .. Our System Sentence planner Candidate generator – limiting the space 1. t-lemma + formeme combination 2. parent–child combination 3. number of children 4. tree size 5. “weak” compatibility with input MR: 6. “strong” compatibility with input MR: (minimum seen in training data) 26/ 40 Ondřej Dušek . .. . .. . .. . . .. . . . . . .. . . .. . . . .. . . .. . . .. . . .. . .. . . . .. . . .. Natural Language Generation • Number of candidates very high even for small domains • We need to lower the number of “possible” successors

  43. . .. .. . . .. . . . . . .. . . .. . . .. .. 3. number of children Ondřej Dušek 26/ 40 (minimum seen in training data) 6. “strong” compatibility with input MR: 5. “weak” compatibility with input MR: 4. tree size 2. parent–child combination . 1. t-lemma + formeme combination Candidate generator – limiting the space Sentence planner Our System . .. . . .. . . . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . . .. .. . . .. . .. . . Natural Language Generation • Number of candidates very high even for small domains • We need to lower the number of “possible” successors • Limiting by things seen in training data: • + at depth levels • + given input MR • nodes seen with current slot-values • required slot-values for each node

  44. • score • Training: • given m , generate the best tree t top with current weights • update weights if t top t gold (gold-standard) • Update: w feat t gold m feat t top m .. . . .. . . . .. . . . .. . . . .. . . .. .. Our System Sentence planner Scorer Basic perceptron scorer w feat t m w 27/ 40 Ondřej Dušek . .. . .. . .. . . .. . . . .. . .. . . .. . . . . . . .. . . .. . . .. . .. . . . .. . . .. Natural Language Generation • a function: sentence plan tree t , MR m → real-valued score • describes the fitness of t for m

  45. • Training: • given m , generate the best tree t top with current weights • update weights if t top t gold (gold-standard) • Update: w feat t gold m feat t top m . . .. . . .. . . .. .. . . .. . . . . .. . . .. . Our System Sentence planner Scorer Basic perceptron scorer w 27/ 40 Ondřej Dušek .. . .. .. .. . . .. . . . . . .. . . .. . . . . .. . . .. . . .. . . .. . . .. . . .. . Natural Language Generation • a function: sentence plan tree t , MR m → real-valued score • describes the fitness of t for m • score = w ⊤ · feat ( t , m )

  46. • Update: w feat t gold m feat t top m . . .. . . .. . . .. . . .. .. . . . .. . . .. . . .. . Our System Sentence planner Scorer Basic perceptron scorer w 27/ 40 Ondřej Dušek .. . . . . .. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . .. . . . .. . . Natural Language Generation • a function: sentence plan tree t , MR m → real-valued score • describes the fitness of t for m • score = w ⊤ · feat ( t , m ) • Training: • given m , generate the best tree t top with current weights • update weights if t top ̸ = t gold (gold-standard)

  47. . . . .. . . .. . . .. . . .. . . .. . .. .. . . .. . . .. . Our System Sentence planner Scorer Basic perceptron scorer 27/ 40 Ondřej Dušek . .. . .. . . .. . . .. . . .. . . . . . .. . . .. . . . .. . Natural Language Generation .. . . .. . . .. • a function: sentence plan tree t , MR m → real-valued score • describes the fitness of t for m • score = w ⊤ · feat ( t , m ) • Training: • given m , generate the best tree t top with current weights • update weights if t top ̸ = t gold (gold-standard) • Update: w = w + α · ( feat ( t gold , m ) − feat ( t top , m ))

  48. • + demoting subtrees of wrong generation outputs • Update: find common subtree, start from it and update using gold t i . .. . . .. . . .. . . . . . .. . .. .. . .. . . .. . Our System Sentence planner Differing subtree updates • promoting subtrees of gold-standard trees pairs of subtrees t i top 28/ 40 Ondřej Dušek . .. . . . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . . .. . .. .. . . .. . . Natural Language Generation • Features are global → bigger trees score better • need to promote “promising” incomplete trees

  49. • Update: find common subtree, start from it and update using gold t i . . .. . . .. . . .. . . .. . . . . .. . . .. . . .. . Our System Sentence planner Differing subtree updates pairs of subtrees t i top 28/ 40 Ondřej Dušek .. .. . . . . .. . . .. . . .. . . .. . . .. .. . .. . . .. . . .. Natural Language Generation . . .. . . .. . . • Features are global → bigger trees score better • need to promote “promising” incomplete trees • → promoting subtrees of gold-standard trees • + demoting subtrees of wrong generation outputs

  50. . . .. . . .. . . .. . . .. . . .. . .. . . . .. . . .. . Our System Sentence planner Differing subtree updates pairs of subtrees t i top 28/ 40 Ondřej Dušek . .. .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . . . .. .. . . .. . . .. Natural Language Generation • Features are global → bigger trees score better • need to promote “promising” incomplete trees • → promoting subtrees of gold-standard trees • + demoting subtrees of wrong generation outputs • Update: find common subtree, start from it and update using gold , t i

  51. . . .. . . .. . . .. . . .. . . .. .. .. . Sentence planner Natural Language Generation Ondřej Dušek 28/ 40 top pairs of subtrees t i Differing subtree updates Our System . . .. . . .. . . . .. . . . . . .. . . .. . . .. . . . .. . . . .. . . .. .. . .. . . .. . . .. • Features are global → bigger trees score better • need to promote “promising” incomplete trees • → promoting subtrees of gold-standard trees • + demoting subtrees of wrong generation outputs • Update: find common subtree, start from it and update using gold , t i Gold standard ( t gold ): Top generated t top : t-tree t-tree be be v:fin v:fin X restaurant restaurant X n:subj n:obj n:obj n:subj range n:in+X cheap italian adj:attr adj:attr price n:attr moderate adj:attr

  52. . .. .. . . .. . . .. . . .. . . .. . .. . Sentence planner Natural Language Generation Ondřej Dušek 28/ 40 top pairs of subtrees t i Differing subtree updates Our System . . .. . . .. . . . .. . .. . . . .. . . .. . . .. . . .. . . . . .. . . .. .. . .. . . .. . . . • Features are global → bigger trees score better • need to promote “promising” incomplete trees • → promoting subtrees of gold-standard trees • + demoting subtrees of wrong generation outputs • Update: find common subtree, start from it and update using gold , t i Gold standard ( t gold ): Top generated t top : t-tree t-tree Common be subtree be v:fin v:fin ( t c ) X restaurant restaurant X n:subj n:obj n:obj n:subj range n:in+X cheap italian adj:attr adj:attr price n:attr moderate adj:attr

  53. . . .. . . .. . . .. . . .. . . .. . .. . Sentence planner Natural Language Generation Ondřej Dušek 28/ 40 top pairs of subtrees t i Differing subtree updates Our System . . .. . . .. . . .. .. .. . . .. . . .. . . .. . . .. . . . . . .. .. . . . . .. . . .. . . .. . • Features are global → bigger trees score better • need to promote “promising” incomplete trees • → promoting subtrees of gold-standard trees • + demoting subtrees of wrong generation outputs • Update: find common subtree, start from it and update using gold , t i Gold standard ( t gold ): Top generated t top : t-tree t-tree be be v:fin v:fin X restaurant restaurant X n:subj n:obj n:obj n:subj range n:in+X cheap italian adj:attr adj:attr price n:attr t 1 top t 1 gold Differing subtrees for update moderate adj:attr

  54. . . .. . . .. . . .. . . .. . . .. .. .. . Sentence planner Natural Language Generation Ondřej Dušek 28/ 40 top pairs of subtrees t i Differing subtree updates Our System . . .. . . .. . . . .. .. . . .. . . .. . . .. . . .. . . . . . . . . .. . . .. . . .. . . .. .. • Features are global → bigger trees score better • need to promote “promising” incomplete trees • → promoting subtrees of gold-standard trees • + demoting subtrees of wrong generation outputs • Update: find common subtree, start from it and update using gold , t i Gold standard ( t gold ): Top generated t top : t-tree t-tree be be v:fin v:fin X restaurant restaurant X n:subj n:obj n:obj n:subj range n:in+X cheap italian adj:attr adj:attr price n:attr + regular full update moderate adj:attr

  55. • Using expected number of children E c n of a node • Future promise: max 0 E c n • over the whole tree • + multiplied by feature sum • + weighted • used on the open_set , not close_set • not for perceptron updates, not for stopping generation . .. . . .. . . . .. . . .. . . . .. . . .. .. Our System Sentence planner Future promise estimate “how many children are missing to meet the expectation” fc n t c n 29/ 40 Ondřej Dušek . . .. .. . .. . . .. . . . . . .. . . .. . . . .. . . .. . . .. . . .. . . .. . . .. . . .. Natural Language Generation • Further boost for incomplete trees

  56. • Future promise: max 0 E c n • over the whole tree • + multiplied by feature sum • + weighted • used on the open_set , not close_set • not for perceptron updates, not for stopping generation . .. . . .. . . . .. . . .. . . . .. .. . . .. . Our System Sentence planner Future promise estimate “how many children are missing to meet the expectation” fc n t c n 29/ 40 Ondřej Dušek .. . . .. .. . . .. . . . .. . .. . . .. . . . . .. .. . . .. . . .. . . . . . .. . . .. . Natural Language Generation • Further boost for incomplete trees • Using expected number of children E c ( n ) of a node

  57. • over the whole tree • + multiplied by feature sum • + weighted • used on the open_set , not close_set • not for perceptron updates, not for stopping generation . . .. . . .. . . .. . . .. .. . .. .. . . .. . . .. . Our System Sentence planner Future promise estimate “how many children are missing to meet the expectation” 29/ 40 Ondřej Dušek . . . . . . .. . . .. . . .. . . .. . . .. . . .. .. . . .. . . .. . .. . . . .. . . Natural Language Generation • Further boost for incomplete trees • Using expected number of children E c ( n ) of a node • Future promise: ∑ fc = max { 0 , E c ( n ) − c ( n ) } n ∈ t

  58. • used on the open_set , not close_set • not for perceptron updates, not for stopping generation . .. . . .. . . . . . .. .. . .. .. .. . . . . .. . . .. . Our System Sentence planner Future promise estimate “how many children are missing to meet the expectation” 29/ 40 Ondřej Dušek . . .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . .. . Natural Language Generation . .. . . .. . . .. • Further boost for incomplete trees • Using expected number of children E c ( n ) of a node • Future promise: ∑ fc = max { 0 , E c ( n ) − c ( n ) } n ∈ t • over the whole tree • + multiplied by feature sum • + weighted

  59. . . . .. . .. .. . . .. . . .. . . .. . .. .. . . .. . . .. . Our System Sentence planner Future promise estimate “how many children are missing to meet the expectation” 29/ 40 Ondřej Dušek . . . .. . . .. . . .. . . .. . . .. . . . . . .. . . . .. . Natural Language Generation .. . . .. . .. . • Further boost for incomplete trees • Using expected number of children E c ( n ) of a node • Future promise: ∑ fc = max { 0 , E c ( n ) − c ( n ) } n ∈ t • over the whole tree • + multiplied by feature sum • + weighted • used on the open_set , not close_set • not for perceptron updates, not for stopping generation

  60. • Mostly simple, single-purpose, rule-based modules (blocks) • Word inflection: statistical ( Flect ) • Gradual transformation of deep trees into surface dependency • Surface trees are then simply linearized • Works OK: analysis . . .. . . .. . . .. .. . . .. . . .. . . .. . . .. . Our System Surface realizer Surface realizer overview trees synthesis on our data = 89.79% BLEU 30/ 40 Ondřej Dušek . .. . . . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . . .. . .. .. . . .. . . Natural Language Generation • English synthesis pipeline from Treex / TectoMT • domain-independent

  61. • Gradual transformation of deep trees into surface dependency • Surface trees are then simply linearized • Works OK: analysis . . .. . . .. . . .. . . .. . . . . .. . . .. . . .. . Our System Surface realizer Surface realizer overview trees synthesis on our data = 89.79% BLEU 30/ 40 Ondřej Dušek .. .. . . . . .. . . .. . . .. . . .. . . .. .. . .. . . .. . . .. . Natural Language Generation . .. . . .. . . • English synthesis pipeline from Treex / TectoMT • domain-independent • Mostly simple, single-purpose, rule-based modules (blocks) • Word inflection: statistical ( Flect )

  62. • Works OK: analysis . . .. . . .. . . .. . . .. . . .. . .. . . . .. . . .. . Our System Surface realizer Surface realizer overview trees synthesis on our data = 89.79% BLEU 30/ 40 Ondřej Dušek . .. .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . .. . Natural Language Generation . .. . . .. . . .. • English synthesis pipeline from Treex / TectoMT • domain-independent • Mostly simple, single-purpose, rule-based modules (blocks) • Word inflection: statistical ( Flect ) • Gradual transformation of deep trees into surface dependency • Surface trees are then simply linearized

  63. . . . .. . . .. . . .. . . .. . . .. . .. .. . . .. . . .. . Our System Surface realizer Surface realizer overview trees 30/ 40 Ondřej Dušek . .. . .. . . .. . . .. . . .. . . . . . .. . . .. . . . . .. .. . . .. . . .. Natural Language Generation • English synthesis pipeline from Treex / TectoMT • domain-independent • Mostly simple, single-purpose, rule-based modules (blocks) • Word inflection: statistical ( Flect ) • Gradual transformation of deep trees into surface dependency • Surface trees are then simply linearized • Works OK: analysis → synthesis on our data = 89.79% BLEU

  64. • Copy the deep tree (sentence plan) • Determine morphological agreement • Add prepositions and conjunctions • Add articles • Compound verb forms (add auxiliaries) • Punctuation • Word inflection • Capitalization t-tree zone=en_gen jump v:fin cat window n:subj n:through+X . .. . . . . .. . . .. . Surface realization example .. . . .. . Our System Surface realizer . 31/ 40 Ondřej Dušek Natural Language Generation .. . .. . . . .. . . .. . .. .. . . .. . . .. . . .. . . .. . . .. . . .. . . . . .. . . .. . . .. . • Realizer steps (simplified):

  65. • Determine morphological agreement • Add prepositions and conjunctions • Add articles • Compound verb forms (add auxiliaries) • Punctuation • Word inflection • Capitalization . . .. . . .. . . .. .. . .. .. . . . . .. . . .. . Our System Surface realizer Surface realization example 31/ 40 Ondřej Dušek Natural Language Generation . . .. .. .. . . .. . . . . . .. . . .. . . . . .. .. . .. . . .. . . . . . . .. . . .. • Realizer steps (simplified): • Copy the deep tree (sentence plan) t-tree zone=en_gen jump v:fin cat window n:subj n:through+X

  66. • Add prepositions and conjunctions • Add articles • Compound verb forms (add auxiliaries) • Punctuation • Word inflection • Capitalization . .. . . .. . . .. . . .. .. . . .. .. . . .. . . .. . Our System Surface realizer Surface realization example 31/ 40 Ondřej Dušek Natural Language Generation . . . . . . . . . .. . . .. . . .. . . .. .. . . .. . . .. . .. .. . . . .. . . .. . • Realizer steps (simplified): • Copy the deep tree (sentence plan) • Determine morphological agreement t-tree zone=en_gen jump v:fin cat window n:subj n:through+X

  67. • Add articles • Compound verb forms (add auxiliaries) • Punctuation • Word inflection • Capitalization . .. . . .. . . .. . . .. .. . . .. .. . . .. . . .. . Our System Surface realizer Surface realization example 31/ 40 Ondřej Dušek Natural Language Generation . . . . . . . . . .. . . .. . . .. . . .. .. . . .. . . .. . . .. .. . .. . . .. . . • Realizer steps (simplified): • Copy the deep tree (sentence plan) • Determine morphological agreement • Add prepositions and conjunctions t-tree zone=en_gen jump v:fin cat window n:subj n:through+X

  68. • Compound verb forms (add auxiliaries) • Punctuation • Word inflection • Capitalization . . .. . . .. . .. .. . . .. . . . . .. . . .. . . .. . Our System Surface realizer Surface realization example 31/ 40 Ondřej Dušek Natural Language Generation .. . . . . . .. .. . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . . .. . . .. • Realizer steps (simplified): • Copy the deep tree (sentence plan) • Determine morphological agreement • Add prepositions and conjunctions • Add articles t-tree zone=en_gen jump v:fin cat window n:subj n:through+X

  69. • Punctuation • Word inflection • Capitalization . . .. . . .. . . .. . .. .. . . . . .. . . .. . . .. . Our System Surface realizer Surface realization example 31/ 40 Ondřej Dušek Natural Language Generation .. . . . . . .. . .. .. . . .. . . .. . . .. . . . .. . . .. .. . . . .. . . .. . . • Realizer steps (simplified): • Copy the deep tree (sentence plan) • Determine morphological agreement • Add prepositions and conjunctions • Add articles • Compound verb forms (add auxiliaries) t-tree zone=en_gen jump v:fin cat window n:subj n:through+X

  70. • Word inflection • Capitalization . .. . . .. . . .. . . .. . . .. .. .. . . . . .. . . .. . Our System Surface realizer Surface realization example 31/ 40 Ondřej Dušek Natural Language Generation . . .. .. . . .. . . .. . . . . . .. . . .. . . .. .. . . . . .. . . .. . . .. . • Realizer steps (simplified): • Copy the deep tree (sentence plan) • Determine morphological agreement • Add prepositions and conjunctions • Add articles • Compound verb forms (add auxiliaries) • Punctuation t-tree zone=en_gen jump v:fin cat window n:subj n:through+X

  71. • Capitalization . . .. . . .. .. . .. . . .. . . .. .. . . . . .. . . .. . Our System Surface realizer Surface realization example 31/ 40 Ondřej Dušek Natural Language Generation . . .. .. . . .. . . .. . . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . • Realizer steps (simplified): • Copy the deep tree (sentence plan) • Determine morphological agreement • Add prepositions and conjunctions • Add articles • Compound verb forms (add auxiliaries) • Punctuation • Word inflection t-tree zone=en_gen jump v:fin cat window n:subj n:through+X

  72. . . . .. . . .. .. . .. . . .. . . .. . .. .. . . .. . . .. . Our System Surface realizer Surface realization example 31/ 40 Ondřej Dušek Natural Language Generation . . . .. . . .. . . .. . . .. . . .. . . . . .. .. . .. . . . . . .. . . .. . • Realizer steps (simplified): • Copy the deep tree (sentence plan) • Determine morphological agreement • Add prepositions and conjunctions • Add articles • Compound verb forms (add auxiliaries) • Punctuation • Word inflection • Capitalization t-tree zone=en_gen jump v:fin cat window n:subj n:through+X

  73. • Alignment provided, but we don't use it • “Non-enumerable” information replaced by “X” symbol • restaurant names, postcodes, phone numbers etc. . .. . . .. . . . . . .. . . .. .. .. . . . . .. . . .. . Our System Experiments Experiments – data set 32/ 40 Ondřej Dušek .. . .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . .. . . .. . . .. . . .. Natural Language Generation • Restaurant recommendations from the BAGEL generator • restaurant location, food type, etc. • 404 utterances for 202 input dialogue acts (DAs) • two paraphrases for each DA

  74. • “Non-enumerable” information replaced by “X” symbol • restaurant names, postcodes, phone numbers etc. . .. . . .. . . . . . .. . . .. .. .. . . . . .. . . .. . Our System Experiments Experiments – data set 32/ 40 Ondřej Dušek .. . .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . . . .. .. . . .. . . .. Natural Language Generation • Restaurant recommendations from the BAGEL generator • restaurant location, food type, etc. • 404 utterances for 202 input dialogue acts (DAs) • two paraphrases for each DA • Alignment provided, but we don't use it

  75. . .. .. .. . . .. . . .. . . .. . . . .. . .. . . .. . . .. . Our System Experiments Experiments – data set 32/ 40 Ondřej Dušek . . . .. . . .. . . .. . . .. . . . . . .. . . .. . . . . .. .. . . .. . . .. Natural Language Generation • Restaurant recommendations from the BAGEL generator • restaurant location, food type, etc. • 404 utterances for 202 input dialogue acts (DAs) • two paraphrases for each DA • Alignment provided, but we don't use it • “Non-enumerable” information replaced by “X” symbol • restaurant names, postcodes, phone numbers etc.

  76. • Basic feature types: • tree properties (size, depth…) • tree + input DA (nodes per slot-value pair…) • node features • input DA feautres (slots, values, pairs of slots) • node + input DA features • repeat features (repeated nodes/slots/values) • dependency features (parent-child) • siblings features (+DA) • bigram features (+DA) • Typical case: counts over whole tree • normalized .. . .. . . . . . .. .. . . . . . .. . . .. .. Our System Experiments Experiments – features 33/ 40 Ondřej Dušek . . .. .. . .. . . .. . . . . . .. . . .. . . . .. . . .. . . .. . . .. . . .. . . .. . . .. Natural Language Generation • Tailored for the input MR format

  77. • Typical case: counts over whole tree • normalized . .. . . .. . . . . . .. .. . .. .. .. . . . . .. . . .. . Our System Experiments Experiments – features 33/ 40 Ondřej Dušek . . .. .. . . .. . . .. . . .. . . .. . . . . . . .. . . . .. Natural Language Generation . .. . . .. . . .. • Tailored for the input MR format • Basic feature types: • tree properties (size, depth…) • tree + input DA (nodes per slot-value pair…) • node features • input DA feautres (slots, values, pairs of slots) • node + input DA features • repeat features (repeated nodes/slots/values) • dependency features (parent-child) • siblings features (+DA) • bigram features (+DA)

  78. . .. . .. . .. .. . . .. . . .. . . . .. . .. . . .. . . .. . Our System Experiments Experiments – features 33/ 40 Ondřej Dušek . . . .. . . .. . . .. . . . . . .. . . .. . . .. . . . .. . Natural Language Generation .. . . .. . .. . • Tailored for the input MR format • Basic feature types: • tree properties (size, depth…) • tree + input DA (nodes per slot-value pair…) • node features • input DA feautres (slots, values, pairs of slots) • node + input DA features • repeat features (repeated nodes/slots/values) • dependency features (parent-child) • siblings features (+DA) • bigram features (+DA) • Typical case: counts over whole tree • normalized

  79. • less than BAGEL 's ~ 67% BLEU • But: • we do not use alignments • our generator has to know when to stop (whether all . .. .. . . .. . . .. . . Our System . . .. . . . Setup Results 4.876 Ondřej Dušek 34/ 40 information is already included) 5.231 59.89 + future promise 58.70 Results + diff-tree updates 4.643 54.24 basic perceptron NIST BLEU . .. .. . . .. . . .. . . .. . .. .. . . .. . . .. . . . Natural Language Generation . . . . .. . . .. . .. . . . .. . . .. . .. • Using 10-fold cross-validation, measuring BLEU/NIST • training DAs never used for testing • using 2 paraphrases for BLEU/NIST measurements

  80. • less than BAGEL 's ~ 67% BLEU • But: • we do not use alignments • our generator has to know when to stop (whether all . .. .. . . .. . . .. . . Our System . . .. . . . Setup Results 4.876 Ondřej Dušek 34/ 40 information is already included) 5.231 59.89 + future promise 58.70 Results + diff-tree updates 4.643 54.24 basic perceptron NIST BLEU . .. .. . . .. . . .. . . .. . .. .. . . .. . . .. . . . Natural Language Generation . . . . .. . . .. . .. . . . .. . . .. . .. • Using 10-fold cross-validation, measuring BLEU/NIST • training DAs never used for testing • using 2 paraphrases for BLEU/NIST measurements

  81. • But: • we do not use alignments • our generator has to know when to stop (whether all . .. .. . . .. . . .. . . Our System . . .. . . . Results Results 4.876 Ondřej Dušek 34/ 40 information is already included) 5.231 59.89 + future promise 58.70 . + diff-tree updates 4.643 54.24 basic perceptron NIST BLEU Setup .. .. . . . . .. . . .. . .. .. . . .. . . .. . . .. Natural Language Generation . . . . .. . . .. . . .. . . .. . . .. . . .. • Using 10-fold cross-validation, measuring BLEU/NIST • training DAs never used for testing • using 2 paraphrases for BLEU/NIST measurements • less than BAGEL 's ~ 67% BLEU

  82. . . . . .. . . .. . .. . . . .. . . .. . .. Our System .. 58.70 Ondřej Dušek 34/ 40 information is already included) 5.231 59.89 + future promise 4.876 + diff-tree updates Results 4.643 54.24 basic perceptron NIST BLEU Setup Results . .. . .. . .. . . .. . . . . . .. . . .. . . . Natural Language Generation .. . .. . . .. . . .. . . .. . . .. . . .. . • Using 10-fold cross-validation, measuring BLEU/NIST • training DAs never used for testing • using 2 paraphrases for BLEU/NIST measurements • less than BAGEL 's ~ 67% BLEU • But: • we do not use alignments • our generator has to know when to stop (whether all

  83. . Our System Generated X is a French and continental restaurant near X. Reference food=Continental, food=French) inform(name=X-name, type=placetoeat, eattype=restaurant, near=X-near, Input DA Example Outputs Results . Input DA .. . . .. . . .. . . X is a French and continental restaurant near X. inform(name=X-name, type=placetoeat, area=riverside, near=X-near, . X is a restaurant in the X area. Ondřej Dušek 35/ 40 X is a French restaurant in the riverside area which serves French food. Generated X is a French restaurant on the riverside. Reference area=riverside, food=French) inform(name=X-name, type=placetoeat, eattype=restaurant, Input DA Generated eattype=restaurant) X is a moderately priced restaurant in X. Reference pricerange=moderate, eattype=restaurant) inform(name=X-name, type=placetoeat, area=X-area, Input DA X is a restaurant in the riverside area near X. Generated X restaurant is near X on the riverside. Reference .. . .. . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . .. . . . .. . . .. . . .. . .. .. . . .. . . .. . . Natural Language Generation

  84. . Our System Generated X is a French and continental restaurant near X. Reference food=Continental, food=French) inform(name=X-name, type=placetoeat, eattype=restaurant, near=X-near, Input DA Example Outputs Results . Input DA .. . . .. . . .. . . X is a French and continental restaurant near X. inform(name=X-name, type=placetoeat, area=riverside, near=X-near, . X is a restaurant in the X area. Ondřej Dušek 35/ 40 X is a French restaurant in the riverside area which serves French food. Generated X is a French restaurant on the riverside. Reference area=riverside, food=French) inform(name=X-name, type=placetoeat, eattype=restaurant, Input DA Generated eattype=restaurant) X is a moderately priced restaurant in X. Reference pricerange=moderate, eattype=restaurant) inform(name=X-name, type=placetoeat, area=X-area, Input DA X is a restaurant in the riverside area near X. Generated X restaurant is near X on the riverside. Reference .. . .. . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . .. . . . .. . . .. . . .. . .. .. . . .. . . .. . . Natural Language Generation

  85. . Our System Generated X is a French and continental restaurant near X. Reference food=Continental, food=French) inform(name=X-name, type=placetoeat, eattype=restaurant, near=X-near, Input DA Example Outputs Results . Input DA .. . . .. . . .. . . X is a French and continental restaurant near X. inform(name=X-name, type=placetoeat, area=riverside, near=X-near, . X is a restaurant in the X area. Ondřej Dušek 35/ 40 X is a French restaurant in the riverside area which serves French food. Generated X is a French restaurant on the riverside. Reference area=riverside, food=French) inform(name=X-name, type=placetoeat, eattype=restaurant, Input DA Generated eattype=restaurant) X is a moderately priced restaurant in X. Reference pricerange=moderate, eattype=restaurant) inform(name=X-name, type=placetoeat, area=X-area, Input DA X is a restaurant in the riverside area near X. Generated X restaurant is near X on the riverside. Reference .. . .. . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . .. . . . .. . . .. . . .. . .. .. . . .. . . .. . . Natural Language Generation

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Outline of todays lecture Overview of Natural Language Generation Components of Natural

Outline of todays lecture Overview of Natural Language Generation Components of Natural

Natural Language Processing Outline of todays lecture Overview of Natural Language Generation Components of Natural Language Generation systems Data for NNs via classical realization Referring expressions Natural Language Processing

586 views • 37 slides
Natural Language Generation Survey in the State of the Art of Natural Topic Coverage

Natural Language Generation Survey in the State of the Art of Natural Topic Coverage

Natural Language Generation Survey in the State of the Art of Natural Topic Coverage Language Generation by Gatt and Krahmer Intro and NLG tasks ->Tianqi Wu NLG Architecture and Approaches ->Jianing Zhou Style Variation

1.02k views • 99 slides
Natural Language Processing 1 Lecture 11: Language generation and summarisation Katia Shutova

Natural Language Processing 1 Lecture 11: Language generation and summarisation Katia Shutova

Natural Language Processing 1 Natural Language Processing 1 Lecture 11: Language generation and summarisation Katia Shutova ILLC University of Amsterdam 6 December 2018 Natural Language Processing 1 Language generation Language generation

828 views • 50 slides
Let the AI do the Talk Adventures with Natural Language Generation @MarcoBonzanini PyParis 2018

Let the AI do the Talk Adventures with Natural Language Generation @MarcoBonzanini PyParis 2018

Let the AI do the Talk Adventures with Natural Language Generation @MarcoBonzanini PyParis 2018 PyData London Conference 12-14 July 2019 @PyDataLondon NATURAL LANGUAGE GENERATION Natural Language Processing 5 Natural Language Processing

1.43k views • 84 slides
STS for NLG Christian Chiarcos chiarcos@uni-potsdam.de Natural Language Generation Natural

STS for NLG Christian Chiarcos chiarcos@uni-potsdam.de Natural Language Generation Natural

STS for NLG Christian Chiarcos chiarcos@uni-potsdam.de Natural Language Generation Natural Language Generation (NLG) (...) is a subfield of artificial intelligence and computational linguistics that is concerned with building computer

557 views • 10 slides
Natural Language Processing 1 Lecture 10: Language generation and summarisation Katia Shutova

Natural Language Processing 1 Lecture 10: Language generation and summarisation Katia Shutova

Natural Language Processing 1 Natural Language Processing 1 Lecture 10: Language generation and summarisation Katia Shutova ILLC University of Amsterdam 2 December 2019 1 / 51 Natural Language Processing 1 Language generation Language

830 views • 54 slides
Natural Language Generation Andrea Zugarini SAILab December 5th, 2019 LabMeeting, December 5th

Natural Language Generation Andrea Zugarini SAILab December 5th, 2019 LabMeeting, December 5th

Natural Language Generation Andrea Zugarini SAILab December 5th, 2019 LabMeeting, December 5th - 2019 Natural Language Generation Natural Language Generation Natural Language Generation is the problem of generating text automatically.

514 views • 16 slides
Natural Language Generation AN OVERVIEW What is NL Generation? a definition, the roots, and

Natural Language Generation AN OVERVIEW What is NL Generation? a definition, the roots, and

Natural Language Generation An Overview Stephan Busemann DFKI GmbH Saarbrcken, Germany Stephan.Busemann@dfki.de Acknowledgement: Part of this presentation was inspired by Robert Dales and Ehud Reiters tutorial on Applied NL Generation at

430 views • 31 slides
Natural Language Generation and Dialog System Evaluation EE596/LING580 -- Conversational

Natural Language Generation and Dialog System Evaluation EE596/LING580 -- Conversational

Natural Language Generation and Dialog System Evaluation EE596/LING580 -- Conversational Artificial Intelligence Hao Cheng University of Washington Conv AI System Diagram 1 Natural Language Generation 2 NLG Approaches Template

782 views • 43 slides
Building Adaptable and Scalable Natural Language Generation Systems Yannis Konstas Natural

Building Adaptable and Scalable Natural Language Generation Systems Yannis Konstas Natural

Building Adaptable and Scalable Natural Language Generation Systems Yannis Konstas Natural Language Generation is everywhere (Machine Translation)

1.78k views • 174 slides
Natural Language Generation (Not Only) in Dialogue Systems Ond rej Du sek Institute of

Natural Language Generation (Not Only) in Dialogue Systems Ond rej Du sek Institute of

Overview Content Planning Surface Realization Natural Language Generation (Not Only) in Dialogue Systems Ond rej Du sek Institute of Formal and Applied Linguistics Charles University in Prague May 22, 2013 Ond rej Du sek

366 views • 24 slides
Natural Language Processing with Deep Learning CS224N/Ling284 Lecture 15: Natural Language

Natural Language Processing with Deep Learning CS224N/Ling284 Lecture 15: Natural Language

Natural Language Processing with Deep Learning CS224N/Ling284 Lecture 15: Natural Language Generation Christopher Manning Announcements Thank you for all your hard work! We know Assignment 5 was tough and a real challenge to do

1.24k views • 87 slides
Intro to Natural Language Generation Ehud Reiter (Abdn Uni and Arria/Data2text) Background read:

Intro to Natural Language Generation Ehud Reiter (Abdn Uni and Arria/Data2text) Background read:

Intro to Natural Language Generation Ehud Reiter (Abdn Uni and Arria/Data2text) Background read: Reiter and Dale, Building Natural Language Generation Systems Ehud Reiter, Computing Science, University of Aberdeen 1 What is NLG? NLG

873 views • 56 slides
Roadmapping for Natural Language Generation Robert Dale rdale@ ics.mq.edu.au www.clt.mq.edu.au

Roadmapping for Natural Language Generation Robert Dale rdale@ ics.mq.edu.au www.clt.mq.edu.au

Roadmapping for Natural Language Generation Robert Dale rdale@ ics.mq.edu.au www.clt.mq.edu.au LREC 2004-05-29 1 Underlying Premise The problem: Current NLG research delivers solutions that are looking for problems The disconnect:

406 views • 14 slides
Language Technology II: Natural Language Dialogue Verbal Output Generation in Dialogue

Language Technology II: Natural Language Dialogue Verbal Output Generation in Dialogue

Language Technology II: Natural Language Dialogue Verbal Output Generation in Dialogue Systems Ivana Kruijff-Korbayov ivana.kruijff@dfki.de Dialog System: Basic Architecture Input ASR Interpretation Dialogue

581 views • 39 slides
Natural Language Understanding We want to communicate with computers using natural language

Natural Language Understanding We want to communicate with computers using natural language

Natural Language Understanding We want to communicate with computers using natural language (spoken and written). unstructured natural language allow any statements, but make mistakes or failure. controlled natural language only

698 views • 20 slides
Peninsula Clean Energy Board of Directors Meeting August 22, 2019 Agenda Call to order /

Peninsula Clean Energy Board of Directors Meeting August 22, 2019 Agenda Call to order /

Peninsula Clean Energy Board of Directors Meeting August 22, 2019 Agenda Call to order / Roll call Public Comment Action to set the agenda and approve consent items 2 Regular Agenda 1. Chair Report (Discussion) 3 Regular

911 views • 69 slides
STANDARDIZING QUALITY ASSESSMENT FOR THE MULTILINGUAL WEB Leonid

STANDARDIZING QUALITY ASSESSMENT FOR THE MULTILINGUAL WEB Leonid

STANDARDIZING QUALITY ASSESSMENT FOR THE MULTILINGUAL WEB Leonid Glazychev, Ph.D., CEO Logrus Interna4onal Corpora4on (C) 2015 Logrus Interna4onal ASTM STANDARD

684 views • 19 slides
On the Impact of Isolation Costs on Locality-aware Cloud Scheduling Ankit Bhardwaj, Meghana G

On the Impact of Isolation Costs on Locality-aware Cloud Scheduling Ankit Bhardwaj, Meghana G

On the Impact of Isolation Costs on Locality-aware Cloud Scheduling Ankit Bhardwaj, Meghana G Gupta , Ryan Stutsman University of Utah Scalable Computer Systems Lab www.utah.systems Code Isolation-cost Aware Scheduling Cloud N Networking P

539 views • 19 slides
Holistic Aggregates in a Networked World: Distributed Tracking of Approximate Quantiles Graham

Holistic Aggregates in a Networked World: Distributed Tracking of Approximate Quantiles Graham

Holistic Aggregates in a Networked World: Distributed Tracking of Approximate Quantiles Graham Cormode Minos Garofalakis cormode@bell-labs.com minos@acm.org S. Muthukrishnan Rajeev Rastogi muthu@cs.rutgers.edu rastogi@bell-labs.com

547 views • 23 slides
Breaking reCAPTCHA: A Holistic Approach via Shape Recognition IFIP SEC 2011 Paul Baecher, Niklas

Breaking reCAPTCHA: A Holistic Approach via Shape Recognition IFIP SEC 2011 Paul Baecher, Niklas

Breaking reCAPTCHA: A Holistic Approach via Shape Recognition IFIP SEC 2011 Paul Baecher, Niklas B uscher, Marc Fischlin, Benjamin Milde Darmstadt University of Technology, supported by DFG Heisenberg and Emmy Noether Programmes

466 views • 27 slides
A Holistic Approach to enhance safety and address the carriage of non-compliant Dangerous Goods

A Holistic Approach to enhance safety and address the carriage of non-compliant Dangerous Goods

A Holistic Approach to enhance safety and address the carriage of non-compliant Dangerous Goods _______________ IVODGA Panel October 6, 2020 NCB Introduction Disturbing Trends Root of Problem Agenda Industry

445 views • 19 slides
Decentralized Control A Case Study of Russia Reethika Ramesh , R. Sundara Raman, M. Bernhard, V.

Decentralized Control A Case Study of Russia Reethika Ramesh , R. Sundara Raman, M. Bernhard, V.

Decentralized Control A Case Study of Russia Reethika Ramesh , R. Sundara Raman, M. Bernhard, V. Ongkowikaya, L. Evdokimov, A. Edmundson, S. Sprecher, M. Ikram, and R. Ensafi 24 February 2020 Centralized Censorship Conventionally,

724 views • 35 slides
High High Per erfor ormance mance Dummy ummy Fill Ins Fill nser ertion ion wit ith h

High High Per erfor ormance mance Dummy ummy Fill Ins Fill nser ertion ion wit ith h

High High Per erfor ormance mance Dummy ummy Fill Ins Fill nser ertion ion wit ith h Coupling oupling and and Unif Unifor ormit mity Cons onstraint aints Yibo Lin, Bei Yu, David Z. Pan Electrical and Computer Engineering

478 views • 28 slides

More recommend