Towards Automatically Extracting Story Graphs from Natural Language - - PowerPoint PPT Presentation

towards automatically extracting story graphs from
SMART_READER_LITE
LIVE PREVIEW

Towards Automatically Extracting Story Graphs from Natural Language - - PowerPoint PPT Presentation

February 4th 2017 AAAI W17: What's Next for AI in Games? Towards Automatically Extracting Story Graphs from Natural Language Stories Josep Valls-Vargas 1 , Jichen Zhu 2 and Santiago Ontan 1 1 Computer Science, 2 Digital Media Drexel


slide-1
SLIDE 1

Josep Valls-Vargas1, Jichen Zhu2 and Santiago Ontañón1

1Computer Science, 2Digital Media

Drexel University

Towards Automatically Extracting Story Graphs from Natural Language Stories

February 4th 2017 – AAAI W17: What's Next for AI in Games?

slide-2
SLIDE 2

Outline

  • Introduction & Motivation
  • Story Graphs
  • Extracting Story Graphs
  • Using Story Graphs

2

slide-3
SLIDE 3

Outline

  • Introduction & Motivation
  • Story Graphs
  • Extracting Story Graphs
  • Using Story Graphs

3

slide-4
SLIDE 4

Introduction

Narratology Artificial Intelligence Natural Language Processing

4

Computational Narrative

slide-5
SLIDE 5

Authorial Bottleneck Problem

5

Opiate [Fairclough 2007]

slide-6
SLIDE 6

Authorial Bottleneck Problem

6

Opiate [Fairclough 2007]

Narrative Function Sequences Characters, Attitudes, … Locations, Props, …

  • Input required by OPIATE
slide-7
SLIDE 7

Automated Narrative Information Extraction

7

Opiate [Fairclough 2007]

Narrative Function Sequences Characters, Attitudes, … Locations, Props, …

  • Input required by OPIATE

Once upon a time, Bonji ran into Lili, Mimo and Bibi, three friends who lived in a hut. In a field nearby lived Snomm who had a Magic

  • Mirror. Past the field and further into the woods lived Blobar. In the
  • ther side of the woods there was a little town where Sergeant Lip

and Corporal Foot lived. They stole the Magic Mirror. [...]

slide-8
SLIDE 8

Automated Narrative Information Extraction

8

Opiate [Fairclough 2007]

Narrative Function Sequences Characters, Attitudes, … Locations, Props, …

  • Input required by OPIATE

Once upon a time, Bonji ran into Lili, Mimo and Bibi, three friends who lived in a hut. In a field nearby lived Snomm who had a Magic

  • Mirror. Past the field and further into the woods lived Blobar. In the
  • ther side of the woods there was a little town where Sergeant Lip

and Corporal Foot lived. They stole the Magic Mirror. [...]

slide-9
SLIDE 9

Outline

  • Introduction & Motivation
  • Story Graphs
  • Extracting Story Graphs
  • Using Story Graphs

9

slide-10
SLIDE 10

Story Graphs

10

Plot Graphs [Li et al. 2013] Social Networks [Elson 2010]

slide-11
SLIDE 11

Story Graphs

11

Once upon a time, Bonji ran into Lili, Mimo and Bibi, three friends who lived in a hut. In a field nearby lived Snomm who had a Magic Mirror. Past the field and further into the woods lived Blobar. In the

  • ther side of the woods there

was a little town where Sergeant Lip and Corporal Foot lived. They stole the Magic Mirror. [...]

slide-12
SLIDE 12

Story Graphs

12

Game Forge [Hartsook et al. 2011], Opiate [Fairclough 2007], Prom Week [McCoy et al. 2011]

slide-13
SLIDE 13

Outline

  • Introduction & Motivation
  • Story Graphs
  • Extracting Story Graphs
  • Using Story Graphs

13

slide-14
SLIDE 14

Extracting Story Graphs

  • Voz

14

Extraction Enrichment Classification Compilation Story Graph Verb Extraction Mention Extraction Feature-Vector Assembly Natural Language Processing Role Identification Coreference Resolution Mention Classification

G = hE, Li V = {v1, ..., vw} E A ⊆ E

A 7! R

Cases Cases Commonsense Knowledge

E V = {v1, ..., vw} E 7! C G = hE, Li

slide-15
SLIDE 15

Dataset

15

  • A. Afanasyev [Finlayson 2011] [Malec 2010]

21 Stories 4,791 Mentions 1,586 Verbs

slide-16
SLIDE 16
  • Mentions
  • Syntactic parse tree
  • Verbs
  • Part-of-speech tags
  • Verb Arguments
  • Typed dependencies

16

Information Extraction

  • Mentions
  • Recall 1.000
  • Precision 0.893
  • Verbs
  • Recall 0.842
  • Precision 1.000
  • Verb Arguments
  • Recall 0.204
  • Precision 0.260
slide-17
SLIDE 17
  • Additional Information
  • WordNet
  • ConceptNet
  • Gazetteers

17

Enrichment of Extracted Information

  • Coreference Resolution
  • C/Gr = 1.07
  • Gr/C = 6.00
slide-18
SLIDE 18
  • Instance Based
  • Weighted continuous Jaccard distance
  • One-story-out protocol
  • Majority Voting
  • Coreference information

18

Mention Classification

(Entities)

slide-19
SLIDE 19

Mention Classification

(Entities)

  • Character/Non-character
  • Precision 0.929
  • Recall 0.934
  • Type (14+1 classes from Chatman’s taxonomy)
  • Precision 0.567
  • Recall of 0.507
  • Roles
  • Precision 0.425
  • Recall of 0.661

19

slide-20
SLIDE 20

Mention Classification

(Entities)

  • Character/Non-character
  • Precision 0.929
  • Recall 0.934
  • Type (14+1 classes from Chatman’s taxonomy)
  • Precision 0.567
  • Recall of 0.507
  • Roles
  • Precision 0.425
  • Recall of 0.661

20

Male/Female/Magical Beings, Locations, Props, Happenings, …

slide-21
SLIDE 21

Story Graph Compilation

  • Character

interactions

  • Character mentions

as nodes

  • Verbs as edges
  • Other nodes
  • Locations
  • Objects

21

slide-22
SLIDE 22

22

slide-23
SLIDE 23

Outline

  • Introduction & Motivation
  • Story Graphs
  • Extracting Story Graphs
  • Using Story Graphs

23

slide-24
SLIDE 24

Story Chronology

24

slide-25
SLIDE 25

Environment & Spatial Relations

25

slide-26
SLIDE 26

Conclusions

  • Mentions
  • Recall 100%
  • Verb Arguments
  • F 0.23
  • Character/Non-character
  • F 0.93
  • Type
  • F 0.52
  • Coreference Resolution
  • C/Gr = 1.07
  • Gr/C = 6.00

26

slide-27
SLIDE 27

Future Work

  • Improve the quality of extracted story graphs
  • Map story graphs to the input of computational

narrative system

27

slide-28
SLIDE 28

Thanks

28

slide-29
SLIDE 29

Josep Valls-Vargas1, Jichen Zhu2 and Santiago Ontañón1

1Computer Science, 2Digital Media

Drexel University

Towards Automatically Extracting Story Graphs from Natural Language Stories

February 4th 2017 – AAAI W17: What's Next for AI in Games?

slide-30
SLIDE 30

Backup Slides

30

slide-31
SLIDE 31

Story Generation

31

Plot Graphs [Li et al. 2013]

slide-32
SLIDE 32

Study of Literature

32

ProppASM [Finlayson 2011], Social Networks [Elson 2010]

slide-33
SLIDE 33

Sentiment

33

slide-34
SLIDE 34

34

  • 14+1 classes derived Chatman’s existents
  • Micro-averaged accuracy: 0.537

Classification

slide-35
SLIDE 35

Automated Narrative Information Extraction

35

One day, somewhere near Kiev, a dragon appeared, who demanded heavy tribute from the people. He demanded every time to eat a fair maiden: and at last the turn came to the Tsarevna, the princess. But the dragon would not eat her, she was too beautiful. He dragged her into his den and made her his wife. [...] When she wrote a letter to her father and mother she used to tie it to the neck of her little dog. [...] The Tsarevna got every day on more intimate terms with her dragon in order to discover who was stronger. At last he owned that Nikita, the tanner at Kiev, was the stronger. [...] The Tsarevna at once wrote to her father [...] So the Tsar looked for Nikita, and went to him himself to beg him to release the land from the cruelty of the dragon and redeem the princess. [...]

slide-36
SLIDE 36

Automated Narrative Information Extraction

  • Characters

36

One day, somewhere near Kiev, a dragon appeared, who demanded heavy tribute from the people. He demanded every time to eat a fair maiden: and at last the turn came to the Tsarevna, the princess. But the dragon would not eat her, she was too beautiful. He dragged her into his den and made her his wife. [...] When she wrote a letter to her father and mother she used to tie it to the neck of her little dog. [...] The Tsarevna got every day on more intimate terms with her dragon in order to discover who was stronger. At last he owned that Nikita, the tanner at Kiev, was the stronger. [...] The Tsarevna at once wrote to her father [...] So the Tsar looked for Nikita, and went to him himself to beg him to release the land from the cruelty of the dragon and redeem the princess. [...]

slide-37
SLIDE 37

Automated Narrative Information Extraction

  • Coreference

37

One day, somewhere near Kiev, a dragon appeared, who demanded heavy tribute from the people. He demanded every time to eat a fair maiden: and at last the turn came to the Tsarevna, the princess. But the dragon would not eat her, she was too beautiful. He dragged her into his den and made her his wife. [...] When she wrote a letter to her father and mother she used to tie it to the neck of her little dog. [...] The Tsarevna got every day on more intimate terms with her dragon in order to discover who was stronger. At last he owned that Nikita, the tanner at Kiev, was the stronger. [...] The Tsarevna at once wrote to her father [...] So the Tsar looked for Nikita, and went to him himself to beg him to release the land from the cruelty of the dragon and redeem the princess. [...]

slide-38
SLIDE 38

Automated Narrative Information Extraction

  • Character Roles

38

One day, somewhere near Kiev, a dragon appeared, who demanded heavy tribute from the people. He demanded every time to eat a fair maiden: and at last the turn came to the Tsarevna, the princess. But the dragon would not eat her, she was too beautiful. He dragged her into his den and made her his wife. [...] When she wrote a letter to her father and mother she used to tie it to the neck of her little dog. [...] The Tsarevna got every day on more intimate terms with her dragon in order to discover who was stronger. At last he owned that Nikita, the tanner at Kiev, was the stronger. [...] The Tsarevna at once wrote to her father [...] So the Tsar looked for Nikita, and went to him himself to beg him to release the land from the cruelty of the dragon and redeem the princess. [...] Villain Hero Sought for person

slide-39
SLIDE 39

Automated Narrative Information Extraction

  • Narrative Functions

39

One day, somewhere near Kiev, a dragon appeared, who demanded heavy tribute from the people. He demanded every time to eat a fair maiden: and at last the turn came to the Tsarevna, the princess. But the dragon would not eat her, she was too beautiful. He dragged her into his den and made her his wife. [...] When she wrote a letter to her father and mother she used to tie it to the neck of her little dog. [...] The Tsarevna got every day on more intimate terms with her dragon in order to discover who was stronger. At last he owned that Nikita, the tanner at Kiev, was the stronger. [...] The Tsarevna at once wrote to her father [...] So the Tsar looked for Nikita, and went to him himself to beg him to release the land from the cruelty of the dragon and redeem the princess. [...] A: Villainy ↑: Departure

slide-40
SLIDE 40

Future Work

Bridging the Gap

40

Scenes & Narrative Functions Characters & Roles Locations & Props

Riu [Ontañón and Zhu 2009, 2014]

Annotated Text Coreference