A Hierarchical Encoder-Decoder for Paragraph Summarization Farzaneh - - PowerPoint PPT Presentation

SLIDE 1

A Hierarchical Encoder-Decoder for Paragraph Summarization

Farzaneh Mahdisoltani

Department of Computer Science University of Toronto

CSC2541, Fall 2016

Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 1 / 8

SLIDE 2

Paragraph Summarization

Summarization Phenomena: ¯ Deletion Paraphrase Generalization

Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 2 / 8

SLIDE 3

Related Work

Extractive Summarization: Stitching words and phrases from the source

Dorr et al. (2003), Cohn et al. (2008), Woodsend et al. (2010)

Abstractive Summarization: Generating richer summaries based on understanding the input

Attention based: Rush et al. (2015) Statistical machine translation for headline generation: Banko et. al.(2000) RNN with attention: Chopra et al.(2016) Encoder-Decoder: Hu et al.(2015)

Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 3 / 8

SLIDE 4

Hierarchichal Encoder-Decoder Model for Summarization

S1 S2 S3 S4 S5 S6 Sentence to Vector enc(P) Paragraph enc(S1) enc(S2) enc(S3) enc(S4) enc(S5) enc(S6) Paragraph to Vector Sentence Level Decoder enc(O1) enc(O2) enc(O3) Word Level Decoder O1 O2 O3 Summary Abstract Concepts Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 4 / 8

SLIDE 5

Hierarchichal Encoder-Decoder Model for Summarization

Objective Function:

t P(ot w|o<t w , os, hp)

Encoder Decoder Word Level Sentence Level h1

i

h2

i

h3

i

h4

i

si wi1 wi2 wi3 wi4 h1

i

h1

j

h2

j

h3

j

h4

j

h5

j

vj1 vj2 vj3 vj4 vj5 yj h1

s

h2

s

start d1

s

• 1

s

d1

w

d2

w

d3

w

• 1

w

• 2

w

• 3

d

• 1

w

• 2

w

h1

i

h1

j

h2

j

h3

j

h4

j

h5

j

wj1 wj2 wj3 wj4 wj5 sj p

Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 5 / 8

SLIDE 6

Selecting the Correct Summary from Candidates

S1 S2 S3 S4 S5 S6 Sentence to Vector enc(P) Paragraph enc(S1) enc(S2) enc(S3) enc(S4) enc(S5) enc(S6) Paragrap h to Vector enc(O1) enc(O2) enc(O3) O1 O2 O3 Sentence to Vector enc(S1) Paragrap h to Vector enc(O1) enc(O2) enc(O3) O1 O2 O3 Sentence to Vector enc(S2) Paragrap h to Vector enc(O1) enc(O2) enc(O3) O1 O2 O3 Sentence to Vector enc(S3) Paragrap h to Vector enc(O1) enc(O2) enc(O3) O1 O2 O3 Sentence to Vector enc(S4) Paragrap h to Vector Candidates Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 6 / 8

SLIDE 7

Preliminary Results

Paragraph Representation Method Accuracy Random 25% Word2Vec 65% Skip-Thought Vectors 62% Bag of Words 67% RNN Encoder 57% Hierarchichal Encoder 54%

Table: Training the encoder for multiple choice question

Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 7 / 8

SLIDE 8

Thank You

Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 8 / 8