Concept-to-text Generation via Discriminative Reranking Ioannis - - PowerPoint PPT Presentation

▶

concept to text generation via discriminative reranking

Concept-to-text Generation via Discriminative Reranking Ioannis - - PowerPoint PPT Presentation

Feb 27, 2024 463 likes •1.17k views

Concept-to-text Generation via Discriminative Reranking Ioannis Konstas and Mirella Lapata School of Informatics Institute for Language, Cognition and Computation University of Edinburgh ACL 2012, Jeju Island Konstas, Lapata (ILCC)

slide-1

SLIDE 1

Concept-to-text Generation via Discriminative Reranking

Ioannis Konstas and Mirella Lapata

School of Informatics Institute for Language, Cognition and Computation University of Edinburgh

ACL 2012, Jeju Island

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 1 / 26

slide-2

SLIDE 2

Introduction

Introduction

Concept-to-text generation refers to the task of automatically producing textual output from nonlinguistic input (Reiter and Dale, 2000)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 2 / 26

slide-3

SLIDE 3

Introduction

Introduction

Concept-to-text generation refers to the task of automatically producing textual output from nonlinguistic input (Reiter and Dale, 2000)

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Condition arg1 arg2 type arrival_time 1600 < Search type what query flight Give me the flights leaving Edinburgh July seventh coming back to Jeju Island before 4pm

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 2 / 26

slide-4

SLIDE 4

Introduction

Traditional NLG Pipeline

Content Selection Surface Realisation Text Input Data Communicative Goal

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 3 / 26

slide-5

SLIDE 5

Introduction

Traditional NLG Pipeline

Content Selection Surface Realisation Text Input Data Communicative Goal

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 4 / 26

slide-6

SLIDE 6

Introduction

Our Approach

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Condition arg1 arg2 type arrival_time 1600 < Search type what query flight Give me the flights leaving Edinburgh July seventh coming back to Jeju Island before 4pm Please show me the flights from Edinburgh on July seventh to Jeju Island before 16:00

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 5 / 26

slide-7

SLIDE 7

Introduction

Our Approach

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Condition arg1 arg2 type arrival_time 1600 < Search type what query flight Give me the flights leaving Edinburgh July seventh coming back to Jeju Island before 4pm Please show me the flights from Edinburgh on July seventh to Jeju Island before 16:00

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 5 / 26

slide-8

SLIDE 8

Introduction

Our Approach

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Condition arg1 arg2 type arrival_time 1600 < Search type what query flight Give me the flights leaving Edinburgh July seventh coming back to Jeju Island before 4pm Please show me the flights from Edinburgh on July seventh to Jeju Island before 16:00

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 5 / 26

slide-9

SLIDE 9

Introduction

Our Approach

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Condition arg1 arg2 type arrival_time 1600 < Search type what query flight Give me the flights leaving Edinburgh July seventh coming back to Jeju Island before 4pm Please show me the flights from Edinburgh on July seventh to Jeju Island before 16:00

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 5 / 26

slide-10

SLIDE 10

Introduction

Our Approach

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Condition arg1 arg2 type arrival_time 1600 < Search type what query flight Give me the flights leaving Edinburgh July seventh coming back to Jeju Island before 4pm Please show me the flights from Edinburgh on July seventh to Jeju Island before 16:00

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 5 / 26

slide-11

SLIDE 11

Introduction

Our Approach

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Condition arg1 arg2 type arrival_time 1600 < Search type what query flight Give me the flights leaving Edinburgh July seventh coming back to Jeju Island before 4pm Please show me the flights from Edinburgh on July seventh to Jeju Island before 16:00

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 5 / 26

slide-12

SLIDE 12

Introduction

Our Approach

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Condition arg1 arg2 type arrival_time 1600 < Search type what query flight Give me the flights leaving Edinburgh July seventh coming back to Jeju Island before 4pm Please show me the flights from Edinburgh on July seventh to Jeju Island before 16:00

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 5 / 26

slide-13

SLIDE 13

Introduction

Our Approach

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Condition arg1 arg2 type arrival_time 1600 < Search type what query flight Give me the flights leaving Edinburgh July seventh coming back to Jeju Island before 4pm Please show me the flights from Edinburgh on July seventh to Jeju Island before 16:00

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 5 / 26

slide-14

SLIDE 14

Introduction

Our Approach

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Condition arg1 arg2 type arrival_time 1600 < Search type what query flight Give me the flights leaving Edinburgh July seventh coming back to Jeju Island before 4pm Please show me the flights from Edinburgh on July seventh to Jeju Island before 16:00

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 5 / 26

slide-15

SLIDE 15

Introduction

Joint Discriminative Reranking with Hypergraphs

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 6 / 26

slide-16

SLIDE 16

Introduction

Joint Discriminative Reranking with Hypergraphs

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Search type what query flight

S

...

S

...

S

...

S

...

S

...

? ? ?

Give me the flights leaving Edinburgh July seventh coming back to Jeju Island

Training S → R(start) R(ri.t)→FS(rj, start)R(rj.t) R(ri.t)→FS(rj, start) FS(r, r.fi)→F(r, r.fj)FS(r, r.fj) FS(r, r.fi)→F(r, r.fj) F(r, r.f )→W(r, r.f )F(r, r.f ) F(r, r.f )→W(r, r.f ) W(r, r.f )→α W(r, r.f )→g(f .v)

FS0,1(query1,start) FS0,2(query1,start) F0,1(query1,type) F0,1(query1,what) F0,2(query1,type) F0,2(query1,what) FS1,2(query1,start) f (e) : Φ = (Φ1, . . . , Φm) Train using Perceptron

FS0,5(search1.t,start)

 

show me ⋆ the flights [type what] show me ⋆ what flights [type what] show me ⋆ all flights [type what] · · ·

 

F0,2(search1.t,type)

 

show me [type] show the [type] what are [type] · · ·

 

W4,5(search1.t,what)

 

flights [∅] flight [∅] airline [∅] · · ·

 

W0,1(search1.t,type)

 

show [∅] me [∅] what [∅] · · ·

 

W1,2(search1.t,type)

 

show [∅] me [∅] what [∅] · · ·

  k-best decoding via reranking Testing

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 6 / 26

slide-17

SLIDE 17

Introduction

Joint Discriminative Reranking with Hypergraphs

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Search type what query flight

S

...

S

...

S

...

S

...

S

...

? ? ?

Give me the flights leaving Edinburgh July seventh coming back to Jeju Island

Training S → R(start) R(ri.t)→FS(rj, start)R(rj.t) R(ri.t)→FS(rj, start) FS(r, r.fi)→F(r, r.fj)FS(r, r.fj) FS(r, r.fi)→F(r, r.fj) F(r, r.f )→W(r, r.f )F(r, r.f ) F(r, r.f )→W(r, r.f ) W(r, r.f )→α W(r, r.f )→g(f .v)

FS0,1(query1,start) FS0,2(query1,start) F0,1(query1,type) F0,1(query1,what) F0,2(query1,type) F0,2(query1,what) FS1,2(query1,start) f (e) : Φ = (Φ1, . . . , Φm) Train using Perceptron

FS0,5(search1.t,start)

 

show me ⋆ the flights [type what] show me ⋆ what flights [type what] show me ⋆ all flights [type what] · · ·

 

F0,2(search1.t,type)

 

show me [type] show the [type] what are [type] · · ·

 

W4,5(search1.t,what)

 

flights [∅] flight [∅] airline [∅] · · ·

 

W0,1(search1.t,type)

 

show [∅] me [∅] what [∅] · · ·

 

W1,2(search1.t,type)

 

show [∅] me [∅] what [∅] · · ·

  k-best decoding via reranking Testing

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 6 / 26

slide-18

SLIDE 18

Introduction

Joint Discriminative Reranking with Hypergraphs

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Search type what query flight

S

...

S

...

S

...

S

...

S

...

? ? ?

Give me the flights leaving Edinburgh July seventh coming back to Jeju Island

Training S → R(start) R(ri.t)→FS(rj, start)R(rj.t) R(ri.t)→FS(rj, start) FS(r, r.fi)→F(r, r.fj)FS(r, r.fj) FS(r, r.fi)→F(r, r.fj) F(r, r.f )→W(r, r.f )F(r, r.f ) F(r, r.f )→W(r, r.f ) W(r, r.f )→α W(r, r.f )→g(f .v)

FS0,1(query1,start) FS0,2(query1,start) F0,1(query1,type) F0,1(query1,what) F0,2(query1,type) F0,2(query1,what) FS1,2(query1,start) f (e) : Φ = (Φ1, . . . , Φm) Train using Perceptron

FS0,5(search1.t,start)

 

show me ⋆ the flights [type what] show me ⋆ what flights [type what] show me ⋆ all flights [type what] · · ·

 

F0,2(search1.t,type)

 

show me [type] show the [type] what are [type] · · ·

 

W4,5(search1.t,what)

 

flights [∅] flight [∅] airline [∅] · · ·

 

W0,1(search1.t,type)

 

show [∅] me [∅] what [∅] · · ·

 

W1,2(search1.t,type)

 

show [∅] me [∅] what [∅] · · ·

  k-best decoding via reranking Testing

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 6 / 26

slide-19

SLIDE 19

Introduction

Joint Discriminative Reranking with Hypergraphs

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Search type what query flight

S

...

S

...

S

...

S

...

S

...

? ? ?

Give me the flights leaving Edinburgh July seventh coming back to Jeju Island

Training S → R(start) R(ri.t)→FS(rj, start)R(rj.t) R(ri.t)→FS(rj, start) FS(r, r.fi)→F(r, r.fj)FS(r, r.fj) FS(r, r.fi)→F(r, r.fj) F(r, r.f )→W(r, r.f )F(r, r.f ) F(r, r.f )→W(r, r.f ) W(r, r.f )→α W(r, r.f )→g(f .v)

FS0,1(query1,start) FS0,2(query1,start) F0,1(query1,type) F0,1(query1,what) F0,2(query1,type) F0,2(query1,what) FS1,2(query1,start) f (e) : Φ = (Φ1, . . . , Φm) Train using Perceptron

FS0,5(search1.t,start)

 

show me ⋆ the flights [type what] show me ⋆ what flights [type what] show me ⋆ all flights [type what] · · ·

 

F0,2(search1.t,type)

 

show me [type] show the [type] what are [type] · · ·

 

W4,5(search1.t,what)

 

flights [∅] flight [∅] airline [∅] · · ·

 

W0,1(search1.t,type)

 

show [∅] me [∅] what [∅] · · ·

 

W1,2(search1.t,type)

 

show [∅] me [∅] what [∅] · · ·

  k-best decoding via reranking Testing

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 6 / 26

slide-20

SLIDE 20

Introduction

Related Work

Angeli et al., 2010 Unified content selection and surface realisation Obtain alignments from Liang et al. (2009) Sequence of discriminative (log-linear) local decisions (records - fields - templates)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 7 / 26

slide-21

SLIDE 21

Introduction

Related Work

Angeli et al., 2010 Unified content selection and surface realisation Obtain alignments from Liang et al. (2009) Sequence of discriminative (log-linear) local decisions (records - fields - templates) Our approach Joint model allows for more global decisions Hypergraphs are a compact representation and allow for efficient inference (k-best decoding via cube pruning) Discriminative reranking reranks k-best trees at all internal nodes

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 7 / 26

slide-22

SLIDE 22

Model

Input

Input: database records d Output: words w corresponding to some records of d Each record r ∈ d has a type r.t and fields f Fields have values f .v and types f .t (integer, categorical) Day day of week dep/ar/ret sunday departure thursday return leaving on sunday

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 8 / 26

slide-23

SLIDE 23

Model

Input

Input: database records d Output: words w corresponding to some records of d Each record r ∈ d has a type r.t and fields f Fields have values f .v and types f .t (integer, categorical) Day day of week dep/ar/ret sunday departure thursday return leaving on sunday

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 8 / 26

slide-24

SLIDE 24

Model

Input

Input: database records d Output: words w corresponding to some records of d Each record r ∈ d has a type r.t and fields f Fields have values f .v and types f .t (integer, categorical) Day day of week dep/ar/ret sunday departure thursday return leaving on sunday

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 8 / 26

slide-25

SLIDE 25

Model

Input

Input: database records d Output: words w corresponding to some records of d Each record r ∈ d has a type r.t and fields f Fields have values f .v and types f .t (integer, categorical) Day day of week dep/ar/ret sunday departure thursday return leaving on sunday

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 8 / 26

slide-26

SLIDE 26

Model

Input

Input: database records d Output: words w corresponding to some records of d Each record r ∈ d has a type r.t and fields f Fields have values f .v and types f .t (integer, categorical) Day day of week dep/ar/ret sunday departure thursday return leaving on sunday

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 8 / 26

slide-27

SLIDE 27

Model

Grammar Definition

1 S → R(start) 2 R(ri.t)→FS(rj, start)R(rj.t) 3 R(ri.t)→FS(rj, start) 4 FS(r, r.fi)→F(r, r.fj)FS(r, r.fj) 5 FS(r, r.fi)→F(r, r.fj) 6 F(r, r.f )→W(r, r.f )F(r, r.f ) 7 F(r, r.f )→W(r, r.f ) 8 W(r, r.f )→α 9 W(r, r.f )→g(f .v) Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 9 / 26

slide-28

SLIDE 28

Model

Grammar Definition

R(search1.t) → FS(flight1, start)R(flight1.t)

1 S → R(start) 2 R(ri.t)→FS(rj, start)R(rj.t) 3 R(ri.t)→FS(rj, start) 4 FS(r, r.fi)→F(r, r.fj)FS(r, r.fj) 5 FS(r, r.fi)→F(r, r.fj) 6 F(r, r.f )→W(r, r.f )F(r, r.f ) 7 F(r, r.f )→W(r, r.f ) 8 W(r, r.f )→α 9 W(r, r.f )→g(f .v) Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 9 / 26

slide-29

SLIDE 29

Model

Grammar Definition

FS(flight1, from) → F(flight1, to)FS(flight1, to)

1 S → R(start) 2 R(ri.t)→FS(rj, start)R(rj.t) 3 R(ri.t)→FS(rj, start) 4 FS(r, r.fi)→F(r, r.fj)FS(r, r.fj) 5 FS(r, r.fi)→F(r, r.fj) 6 F(r, r.f )→W(r, r.f )F(r, r.f ) 7 F(r, r.f )→W(r, r.f ) 8 W(r, r.f )→α 9 W(r, r.f )→g(f .v) Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 9 / 26

slide-30

SLIDE 30

Model

Grammar Definition

F(search1, what) → W(search1, what)F(search1, what)

1 S → R(start) 2 R(ri.t)→FS(rj, start)R(rj.t) 3 R(ri.t)→FS(rj, start) 4 FS(r, r.fi)→F(r, r.fj)FS(r, r.fj) 5 FS(r, r.fi)→F(r, r.fj) 6 F(r, r.f )→W(r, r.f )F(r, r.f ) 7 F(r, r.f )→W(r, r.f ) 8 W(r, r.f )→α 9 W(r, r.f )→g(f .v) Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 9 / 26

slide-31

SLIDE 31

Model

Grammar Definition

W(search1, type) → show [type.v = ‘query’]

1 S → R(start) 2 R(ri.t)→FS(rj, start)R(rj.t) 3 R(ri.t)→FS(rj, start) 4 FS(r, r.fi)→F(r, r.fj)FS(r, r.fj) 5 FS(r, r.fi)→F(r, r.fj) 6 F(r, r.f )→W(r, r.f )F(r, r.f ) 7 F(r, r.f )→W(r, r.f ) 8 W(r, r.f )→α 9 W(r, r.f )→g(f .v) Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 9 / 26

slide-32

SLIDE 32

Model

Hypergraph Construction

Map standard weighted CYK algorithm to hypergraph H : N, E, t, R R(search1.t) FS(flight1.t,start) R(flight1.t) f (e) =f (FS5,7(flight1.t, start)) ⊗ f (R7,9(flight1.t))⊗ w(R(search1.t) → FS(flight1, start) R(flight1.t)) R(ri.t)→FS(rj, start)R(rj.t)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 10 / 26

slide-33

SLIDE 33

Model

Hypergraph Construction

Map standard weighted CYK algorithm to hypergraph H : N, E, t, R R(search1.t) FS(flight1.t,start) R(flight1.t) f (e) =f (FS5,7(flight1.t, start)) ⊗ f (R7,9(flight1.t))⊗ w(R(search1.t) → FS(flight1, start) R(flight1.t)) R(ri.t)→FS(rj, start)R(rj.t)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 10 / 26

slide-34

SLIDE 34

Model

Hypergraph Construction

Map standard weighted CYK algorithm to hypergraph H : N, E, t, R R(search1.t) FS(flight1.t,start) R(flight1.t) f (e) =f (FS5,7(flight1.t, start)) ⊗ f (R7,9(flight1.t))⊗ w(R(search1.t) → FS(flight1, start) R(flight1.t)) R(ri.t)→FS(rj, start)R(rj.t)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 10 / 26

slide-35

SLIDE 35

Model

Hypergraph Construction

Map standard weighted CYK algorithm to hypergraph H : N, E, t, R R(search1.t) FS(flight1.t,start) R(flight1.t) f (e) =f (FS5,7(flight1.t, start)) ⊗ f (R7,9(flight1.t))⊗ w(R(search1.t) → FS(flight1, start) R(flight1.t)) R(ri.t)→FS(rj, start)R(rj.t)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 10 / 26

slide-36

SLIDE 36

Model

Hypergraph Construction

Map standard weighted CYK algorithm to hypergraph H : N, E, t, R R(search1.t) FS(flight1.t,start) R(flight1.t) f (e) =f (FS5,7(flight1.t, start)) ⊗ f (R7,9(flight1.t))⊗ w(R(search1.t) → FS(flight1, start) R(flight1.t)) R(ri.t)→FS(rj, start)R(rj.t)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 10 / 26

slide-37

SLIDE 37

Model

Hypergraph Construction

Map standard weighted CYK algorithm to hypergraph H : N, E, t, R R(search1.t) FS(flight1.t,start) R(flight1.t) f (e) =f (FS5,7(flight1.t, start)) ⊗ f (R7,9(flight1.t))⊗ w(R(search1.t) → FS(flight1, start) R(flight1.t)) R(ri.t)→FS(rj, start)R(rj.t)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 10 / 26

slide-38

SLIDE 38

Model

Hypergraph Reranking

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Search type what query flight

S

...

S

...

S

...

S

...

S

...

? ? ?

Give me the flights leaving Edinburgh July seventh coming back to Jeju Island

Hidden correspondence h between database d and words w

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 11 / 26

slide-39

SLIDE 39

Model

Hypergraph Reranking

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Search type what query flight

S

...

S

...

S

...

S

...

S

...

? ? ?

Give me the flights leaving Edinburgh July seventh coming back to Jeju Island

Hidden correspondence h between database d and words w (^ w, ^ h) = arg max

w,h α · Φ(d, w, h)

Φ = (Φ1, . . . , Φm) : high dimensional feature representation α : weight vector Learn α with averaged structured perceptron (Collins, 2002)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 11 / 26

slide-40

SLIDE 40

Model

Hypergraph Reranking

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Search type what query flight

S

...

S

...

S

...

S

...

S

...

? ? ?

Give me the flights leaving Edinburgh July seventh coming back to Jeju Island

Hidden correspondence h between database d and words w (^ w, ^ h) = arg max

w,h α · Φ(d, w, h)

Φ = (Φ1, . . . , Φm) : high dimensional feature representation α : weight vector Learn α with averaged structured perceptron (Collins, 2002)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 11 / 26

slide-41

SLIDE 41

Model

Hypergraph Reranking

Flight from to edinburgh jeju Day Number number dep/ar 7 departure Month month dep/ar july departure Search type what query flight

S

...

S

...

S

...

S

...

S

...

? ? ?

Give me the flights leaving Edinburgh July seventh coming back to Jeju Island

Hidden correspondence h between database d and words w (^ w, ^ h) = arg max

w,h α · Φ(d, w, h)

Φ = (Φ1, . . . , Φm) : high dimensional feature representation α : weight vector Learn α with averaged structured perceptron (Collins, 2002)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 11 / 26

slide-42

SLIDE 42

Model

Baseline Features

Baseline Model Feature (local) : Log score of unsupervised generative decoder (Konstas and Lapata, 2012a)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 12 / 26

slide-43

SLIDE 43

Model

Baseline Features

Baseline Model Feature (local) : Log score of unsupervised generative decoder (Konstas and Lapata, 2012a) Alignment Features (local) : Count of each PCFG rule

R(search1.t) FS(flight1.t,start) R(flight1.t) R(ri.t)→FS(rj, start)R(rj.t)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 12 / 26

slide-44

SLIDE 44

Model

Lexical Features

Word Bigrams/Trigrams (non-local) Number of Words per Field (local) Consecutive Word/Bigram/Trigram (non-local)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 13 / 26

slide-45

SLIDE 45

Model

Lexical Features

Word Bigrams/Trigrams (non-local) Number of Words per Field (local) Consecutive Word/Bigram/Trigram (non-local)

FS0,3(search1.t,start) W0,1(search1.t,type) · · · W1,3(search1.t,what)

    

show me what · · ·

         

me the me flights the flights · · ·

    

<show me the>, <show me flights>, etc.

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 13 / 26

slide-46

SLIDE 46

Model

Structural Features

Field Bigrams/Trigrams (non-local) Number of Fields per Record (local) Fields with no Value (local)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 14 / 26

slide-47

SLIDE 47

Model

Structural Features

Field Bigrams/Trigrams (non-local) Number of Fields per Record (local) Fields with no Value (local)

FS2,6(flight1.t,start) F2,4(flight1.t,from) FS4,6(flight1.t,from) F4,6(flight1.t,to) ǫ <from to> | flight

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 14 / 26

slide-48

SLIDE 48

Model

k-best Decoding

Bottom-up Viterbi search

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 15 / 26

slide-49

SLIDE 49

Model

k-best Decoding

Bottom-up Viterbi search Keep k-best derivations at each node, cube pruning (Chiang, 2007)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 15 / 26

slide-50

SLIDE 50

Model

k-best Decoding

Bottom-up Viterbi search Keep k-best derivations at each node, cube pruning (Chiang, 2007) Score of j-th derivation: α · ΦL(e) + α · ΦN(< e, j >)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 15 / 26

slide-51

SLIDE 51

Model

k-best Decoding

Bottom-up Viterbi search Keep k-best derivations at each node, cube pruning (Chiang, 2007) Score of j-th derivation: α · ΦL(e) + α · ΦN(< e, j >) Nodes in hypergraph augmented with lexical and structural sub-strings (Huang and Chiang, 2007) e.g. R2,8(flight1.t)<one way⋆to Seoul; direction from⋆stop-over to>

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 15 / 26

slide-52

SLIDE 52

Model

k-best Decoding

Bottom-up Viterbi search Keep k-best derivations at each node, cube pruning (Chiang, 2007) Score of j-th derivation: α · ΦL(e) + α · ΦN(< e, j >) Nodes in hypergraph augmented with lexical and structural sub-strings (Huang and Chiang, 2007) e.g. R2,8(flight1.t)<one way⋆to Seoul; direction from⋆stop-over to> Root node: most grammatical and semantically correct derivation

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 15 / 26

slide-53

SLIDE 53

Model

k-best Decoding

FS0,5(search1.t,start)

 

show me ⋆ the flights [type what] show me ⋆ what flights [type what] show me ⋆ all flights [type what] · · ·

 

F0,2(search1.t,type)

 

show me [type] show the [type] what are [type] · · ·

 

W4,5(search1.t,what)

 

flights [∅] flight [∅] airline [∅] · · ·

 

W0,1(search1.t,type)

 

show [∅] me [∅] what [∅] · · ·

 

W1,2(search1.t,type)

 

show [∅] me [∅] what [∅] · · ·

 

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 16 / 26

slide-54

SLIDE 54

Model

k-best Decoding

FS0,5(search1.t,start)

 

show me ⋆ the flights [type what] show me ⋆ what flights [type what] show me ⋆ all flights [type what] · · ·

 

F0,2(search1.t,type)

 

show me [type] show the [type] what are [type] · · ·

 

W4,5(search1.t,what)

 

flights [∅] flight [∅] airline [∅] · · ·

 

W0,1(search1.t,type)

 

show [∅] me [∅] what [∅] · · ·

 

W1,2(search1.t,type)

 

show [∅] me [∅] what [∅] · · ·

 

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 16 / 26

slide-55

SLIDE 55

Model

k-best Decoding

FS0,5(search1.t,start)

 

show me ⋆ the flights [type what] show me ⋆ what flights [type what] show me ⋆ all flights [type what] · · ·

 

F0,2(search1.t,type)

 

show me [type] show the [type] what are [type] · · ·

 

W4,5(search1.t,what)

 

flights [∅] flight [∅] airline [∅] · · ·

 

W0,1(search1.t,type)

 

show [∅] me [∅] what [∅] · · ·

 

W1,2(search1.t,type)

 

show [∅] me [∅] what [∅] · · ·

 

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 16 / 26

slide-56

SLIDE 56

Results

Experimental Setup

Data Atis : mapping from λ-version (Zettlemoyer and Collins, 2007) Model parameters estimated on dev set (k-best, n-grams)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 17 / 26

slide-57

SLIDE 57

Results

Experimental Setup

Data Atis : mapping from λ-version (Zettlemoyer and Collins, 2007) Model parameters estimated on dev set (k-best, n-grams) Evaluation Automatic evaluation: BLEU-4, METEOR Human evaluation (MTurk): fluency, semantic correctness

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 17 / 26

slide-58

SLIDE 58

Results

Experimental Setup

Data Atis : mapping from λ-version (Zettlemoyer and Collins, 2007) Model parameters estimated on dev set (k-best, n-grams) Evaluation Automatic evaluation: BLEU-4, METEOR Human evaluation (MTurk): fluency, semantic correctness System Comparison Baseline: 1-best+base+align k-best (+Lexical): k-best+base+align+lex k-best (+Structural): k-best+base+align+lex+str Angeli et al. (2010)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 17 / 26

slide-59

SLIDE 59

Results

Results: Automatic Evaluation

System BLEU METEOR 1-best+base+align 21.93 34.01 k-best+base+align+lex 28.66 45.18 k-best+base+align+lex+str 30.62 46.07 Angeli 26.77 42.41

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 18 / 26

slide-60

SLIDE 60

Results

Results: Automatic Evaluation

System BLEU METEOR 1-best+base+align 21.93 34.01 k-best+base+align+lex 28.66 45.18 k-best+base+align+lex+str 30.62 46.07 Angeli 26.77 42.41

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 18 / 26

slide-61

SLIDE 61

Results

Results: Human Evaluation

System Fluency SemCor 1-Best 2.70 3.05 k-Best 4.02 4.04 Angeli 3.74 3.17 Human 4.18 4.02

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 19 / 26

slide-62

SLIDE 62

Results

Results: Human Evaluation

System Fluency SemCor 1-Best 2.70 3.05 k-Best 4.02 4.04 Angeli 3.74 3.17 Human 4.18 4.02 k-Best significantly better than 1-best and Angeli (α < 0.01) k-Best and Human are not significantly different

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 19 / 26

slide-63

SLIDE 63

Results

Output

Flight from to phoenix milwaukee Time when dep/ar evening departure Day day dep/ar wednesday departure Search type what query flight 1-Best: On Wednesday evening from from Phoenix to Milwaukee on

Wednesday evening

k-Best: Please list the flights from Phoenix to Milwaukee on

Wednesday evening

Angeli: Show me the flights from Phoenix to Milwaukee on Wednesday

evening flights from Phoenix to Milwaukee

Human: Give me the flights from Phoenix to Milwaukee on Wednesday

evening

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 20 / 26

slide-64

SLIDE 64

Results

Conclusions

Generation as parsing problem using the hypergraph framework Discriminative reranking using the structured perceptron Introduced local and non-local features Performance better than state-of-the-art Future work: dependency relations, discourse

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 21 / 26

slide-65

SLIDE 65

Results

Demo

Flight Booking System - lowJet

Scenario 1 Imagine you are in Los Angeles for the week- end and you would like to fly directly back home to Denver the following Monday. Try to find the earliest flight. Scenario 2 You have won a free one-way vacation from Miami to Washington DC. Book the cheapest tickets for next Thursday morning. Alternatively, just find the earliest itinerary.

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 22 / 26

slide-66

SLIDE 66

Results

Thank you

Questions ?

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 23 / 26

slide-67

SLIDE 67

Oracle Derivation

Oracle derivation (w∗, h+) Use the existing decoder but observe the training text. w∗: gold standard text h+: best latent configuration

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 24 / 26

slide-68

SLIDE 68

Human Evaluation (Mturk)

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 25 / 26

slide-69

SLIDE 69

Determining Text Length

Train a linear regression model Idea: The more records and fields that have values in the database → the more facts need to be uttered Input to the model: Flattened version of the database input, i.e. each feature is a record-field pair Feature values: Values vs Counts of Fields

Konstas, Lapata (ILCC) Concept-to-text Generation ACL 2012, Jeju Island 26 / 26