February 4th 2017 – AAAI W17: What's Next for AI in Games? Towards Automatically Extracting Story Graphs from Natural Language Stories Josep Valls-Vargas 1 , Jichen Zhu 2 and Santiago Ontañón 1 1 Computer Science, 2 Digital Media Drexel University
Outline • Introduction & Motivation • Story Graphs • Extracting Story Graphs • Using Story Graphs 2
Outline • Introduction & Motivation • Story Graphs • Extracting Story Graphs • Using Story Graphs 3
Introduction Narratology Computational Narrative Natural Artificial Language Intelligence Processing 4
Authorial Bottleneck Problem Opiate [Fairclough 2007] 5
Authorial Bottleneck Problem • Input required by OPIATE Characters, Locations, Narrative Function Attitudes, … Props, … Sequences Opiate [Fairclough 2007] 6
Automated Narrative Information Extraction • Input required by OPIATE Characters, Locations, Narrative Function Attitudes, … Props, … Sequences Once upon a time, Bonji ran into Lili, Mimo and Bibi, three friends who lived in a hut. In a field nearby lived Snomm who had a Magic Mirror. Past the field and further into the woods lived Blobar. In the other side of the woods there was a little town where Sergeant Lip and Corporal Foot lived. They stole the Magic Mirror. [...] Opiate [Fairclough 2007] 7
Automated Narrative Information Extraction • Input required by OPIATE Characters, Locations, Narrative Function Attitudes, … Props, … Sequences Once upon a time, Bonji ran into Lili, Mimo and Bibi, three friends who lived in a hut. In a field nearby lived Snomm who had a Magic Mirror. Past the field and further into the woods lived Blobar. In the other side of the woods there was a little town where Sergeant Lip and Corporal Foot lived. They stole the Magic Mirror. [...] Opiate [Fairclough 2007] 8
Outline • Introduction & Motivation • Story Graphs • Extracting Story Graphs • Using Story Graphs 9
Story Graphs Plot Graphs [Li et al. 2013] Social Networks [Elson 2010] 10
Story Graphs Once upon a time, Bonji ran into Lili, Mimo and Bibi, three friends who lived in a hut . In a field nearby lived Snomm who had a Magic Mirror . Past the field and further into the woods lived Blobar . In the other side of the woods there was a little town where Sergeant Lip and Corporal Foot lived. They stole the Magic Mirror . [...] 11
Story Graphs Game Forge [Hartsook et al. 2011], Opiate [Fairclough 2007], Prom Week [McCoy et al. 2011] 12
Outline • Introduction & Motivation • Story Graphs • Extracting Story Graphs • Using Story Graphs 13
Extracting Story Graphs • Voz G = h E, L i G = h E, L i E Mention Coreference Commonsense Extraction Resolution Knowledge E A ⊆ E A 7! R Natural Language Feature-Vector Mention Role Story Processing Assembly Classification Identification Graph V = { v 1 , ..., v w } Verb Cases Cases Extraction E 7! C V = { v 1 , ..., v w } Extraction Enrichment Classification Compilation 14
Dataset 21 Stories 4,791 Mentions A. Afanasyev [Finlayson 2011] [Malec 2010] 1,586 Verbs 15
Information Extraction • Mentions o Recall 1.000 • Mentions o Precision 0.893 o Syntactic parse tree • Verbs • Verbs o Recall 0.842 o Part-of-speech tags o Precision 1.000 • Verb Arguments • Verb Arguments o Typed dependencies o Recall 0.204 o Precision 0.260 16
Enrichment of Extracted Information • Additional Information • Coreference Resolution o WordNet o C/Gr = 1.07 o ConceptNet o Gr/C = 6.00 o Gazetteers 17
Mention Classification (Entities) • Instance Based o Weighted continuous Jaccard distance o One-story-out protocol • Majority Voting o Coreference information 18
Mention Classification (Entities) • Character/Non-character o Precision 0.929 o Recall 0.934 • Type (14+1 classes from Chatman’s taxonomy) o Precision 0.567 o Recall of 0.507 • Roles o Precision 0.425 o Recall of 0.661 19
Mention Classification (Entities) • Character/Non-character o Precision 0.929 o Recall 0.934 • Type (14+1 classes from Chatman’s taxonomy) o Precision 0.567 o Recall of 0.507 • Roles o Precision 0.425 Male/Female/Magical Beings, o Recall of 0.661 Locations, Props, Happenings, … 20
Story Graph Compilation • Character interactions Character mentions o as nodes Verbs as edges o • Other nodes Locations o Objects o … o 21
22
Outline • Introduction & Motivation • Story Graphs • Extracting Story Graphs • Using Story Graphs 23
Story Chronology 24
Environment & Spatial Relations 25
Conclusions • Mentions o Recall 100% • Verb Arguments o F 0.23 • Character/Non-character o F 0.93 • Type o F 0.52 • Coreference Resolution o C/Gr = 1.07 o Gr/C = 6.00 26
Future Work • Improve the quality of extracted story graphs • Map story graphs to the input of computational narrative system 27
Thanks 28
February 4th 2017 – AAAI W17: What's Next for AI in Games? Towards Automatically Extracting Story Graphs from Natural Language Stories Josep Valls-Vargas 1 , Jichen Zhu 2 and Santiago Ontañón 1 1 Computer Science, 2 Digital Media Drexel University
Backup Slides 30
Story Generation Plot Graphs [Li et al. 2013] 31
Study of Literature ProppASM [Finlayson 2011 ], Social Networks [Elson 2010] 32
Sentiment 33
Classification • 14+1 classes derived Chatman’s existents Micro-averaged accuracy: 0.537 • 34
Automated Narrative Information Extraction One day, somewhere near Kiev, a dragon appeared, who demanded heavy tribute from the people. He demanded every time to eat a fair maiden: and at last the turn came to the Tsarevna, the princess. But the dragon would not eat her, she was too beautiful. He dragged her into his den and made her his wife. [...] When she wrote a letter to her father and mother she used to tie it to the neck of her little dog. [...] The Tsarevna got every day on more intimate terms with her dragon in order to discover who was stronger. At last he owned that Nikita, the tanner at Kiev, was the stronger. [...] The Tsarevna at once wrote to her father [...] So the Tsar looked for Nikita, and went to him himself to beg him to release the land from the cruelty of the dragon and redeem the princess. [...] 35
Automated Narrative Information Extraction • Characters One day, somewhere near Kiev, a dragon appeared, who demanded heavy tribute from the people. He demanded every time to eat a fair maiden: and at last the turn came to the Tsarevna, the princess. But the dragon would not eat her, she was too beautiful. He dragged her into his den and made her his wife. [...] When she wrote a letter to her father and mother she used to tie it to the neck of her little dog. [...] The Tsarevna got every day on more intimate terms with her dragon in order to discover who was stronger. At last he owned that Nikita, the tanner at Kiev, was the stronger. [...] The Tsarevna at once wrote to her father [...] So the Tsar looked for Nikita, and went to him himself to beg him to release the land from the cruelty of the dragon and redeem the princess. [...] 36
Automated Narrative Information Extraction • Coreference One day, somewhere near Kiev, a dragon appeared, who demanded heavy tribute from the people. He demanded every time to eat a fair maiden: and at last the turn came to the Tsarevna, the princess. But the dragon would not eat her, she was too beautiful. He dragged her into his den and made her his wife. [...] When she wrote a letter to her father and mother she used to tie it to the neck of her little dog. [...] The Tsarevna got every day on more intimate terms with her dragon in order to discover who was stronger. At last he owned that Nikita, the tanner at Kiev, was the stronger. [...] The Tsarevna at once wrote to her father [...] So the Tsar looked for Nikita, and went to him himself to beg him to release the land from the cruelty of the dragon and redeem the princess. [...] 37
Automated Narrative Information Extraction • Character Roles Villain One day, somewhere near Kiev, a dragon appeared, who demanded heavy tribute from the people. He demanded every time to eat a fair maiden: and at last the turn came to the Tsarevna, the princess. But the dragon would not eat her, she was too beautiful. He dragged her Sought into his den and made her his wife. [...] When she wrote a letter to for her father and mother she used to tie it to the neck of her little dog. person [...] The Tsarevna got every day on more intimate terms with her dragon in order to discover who was stronger. At last he owned that Nikita, the tanner at Kiev, was the stronger. [...] The Tsarevna at once wrote to her father [...] So the Tsar looked for Nikita, and went to him himself to beg him to release the land from the cruelty of the dragon and redeem the princess. [...] Hero 38
Recommend
More recommend
