Answering Complex Questions by Joining Multi-Document Evidence with - - PowerPoint PPT Presentation

▶
answering complex questions by joining multi document
SMART_READER_LITE
LIVE PREVIEW

Answering Complex Questions by Joining Multi-Document Evidence with - - PowerPoint PPT Presentation

Aug 25, 2023 238 likes •855 views

Answering Complex Questions by Joining Multi-Document Evidence with Quasi Knowledge Graphs Xiaolu Lu 1 , Soumajit Pramanik 2 , Rishiraj Saha Roy 2 , Abdalghani Abujabal 3 , Yafang Wang 4 and Gerhard Weikum 2 1 RMIT University, 2 Max Planck

slide-1
SLIDE 1

Answering Complex Questions by Joining Multi-Document Evidence with Quasi Knowledge Graphs

Xiaolu Lu1, Soumajit Pramanik2, Rishiraj Saha Roy2, Abdalghani Abujabal3, Yafang Wang4 and Gerhard Weikum2

1 RMIT University, 2 Max Planck Institute for Informatics, 3 Amazon Alexa, 4 Ant Financial

SIGIR 2019, Paris

slide-2
SLIDE 2

2

Question Answering: Vital for Search

★ Direct answers to questions ★ Saves time and effort ★ Natural in voice UI ★ Fact-centric, entity-seeking ★ Relies on knowledge graphs

slide-3
SLIDE 3

3

Question Answering: Vital for Search

★ Direct answers to questions ★ Saves time and effort ★ Natural in voice UI ★ Fact-centric, entity-seeking ★ Relies on knowledge graphs

slide-4
SLIDE 4

4

Question Answering: Vital for Search

★ Direct answers to questions ★ Saves time and effort ★ Natural in voice UI ★ Fact-centric, entity-seeking ★ Relies on knowledge graphs

slide-5
SLIDE 5

5

Question Answering: Vital for Search

★ Direct answers to questions ★ Saves time and effort ★ Natural in voice UI ★ Fact-centric, entity-seeking ★ Relies on knowledge graphs

slide-6
SLIDE 6

6

Question Answering: Vital for Search

★ Direct answers to questions ★ Saves time and effort ★ Natural in voice UI ★ Fact-centric, entity-seeking ★ Relies on knowledge graphs

slide-7
SLIDE 7

7

Question Answering: Vital for Search

★ Direct answers to questions ★ Saves time and effort ★ Natural in voice UI ★ Fact-centric, entity-seeking ★ Relies on knowledge graphs

slide-8
SLIDE 8

8

Direct Answers Today

★ Works for simple questions ★ Misses additional conditions ★ Depends upon KG completeness

slide-9
SLIDE 9

9

What’s new?

★ Entity answers to complex questions directly from text ★ Unsupervised method using Group Steiner Trees ★ Handles ungrammatical constructs ★ Robust to high degrees of noise ★ Provides explanatory evidence

slide-10
SLIDE 10

10

Approach Outline

1. Construct noisy quasi-KG on-the-fly 2. Augment with alignments and types 3. Spot question cornerstones 4. Compute Group Steiner Tree 5. Rank answer candidates

QUEST: Question Answering with Steiner Trees

Graph construction Graph algorithm

slide-11
SLIDE 11

11

Multi-document Evidence

Question: Which Nolan films won an Oscar but missed a Golden Globe? The 2011 Oscar award just announced that Inception is the winner of the Best Sound Editing award. Other winners of the day... Nolan directed the movie Inception and other science thrillers... Inception narrowly lost to The Social Network for Best Screenplay at the 68th Golden Globe Awards, which were declared in the afternoon.. This illustrates “joining evidence”

slide-12
SLIDE 12

12

Semi-structuring Text with Open IE

<Nolan, directed, Inception> <Inception, won, Best Sound> <2011 Oscars, announced, Best Sound> <Inception, nominated, Best Actor> <The movie Inception, missed out, Golden Globe Awards> <Chris Nolan, director of, The movie Inception> <Inception’s script, edited by, Chris Nolan> <Inception, lost to, The Social Network> <Best Actor, declared at, 83rd Academy Awards> <The Social Network, winner of, Best Screenplay> <Golden Globes, announced, Best Screenplay>

slide-13
SLIDE 13

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated

13

slide-14
SLIDE 14

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated

Form noisy quasi-KG that joins evidence from multiple documents

  • n-the-fly

14

slide-15
SLIDE 15

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated

15

slide-16
SLIDE 16

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

16

slide-17
SLIDE 17

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

17

Add types: ★ Hearst patterns

X and other <TYPE> <TYPE> such as X ...

★ From KG

slide-18
SLIDE 18

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

18

slide-19
SLIDE 19

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

19

slide-20
SLIDE 20

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

20

Insert alignment edges using lexicons and similarity

slide-21
SLIDE 21

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

21

slide-22
SLIDE 22

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

22

slide-23
SLIDE 23

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

Simplify by considering edges as homogeneous and undirected

23

slide-24
SLIDE 24

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

24

slide-25
SLIDE 25

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type 0.9 1.0 1.0 0.6 1.0 1.0 0.9 1.0 0.9 0.4 0.4 0.4 0.3 0.3 0.3 0.2 0.2 0.2 0.1 0.3 1.0 0.9 0.4 0.9 0.1 0.1

25

slide-26
SLIDE 26

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type 0.9 1.0 1.0 0.6 1.0 1.0 0.9 1.0 0.9 0.4 0.4 0.4 0.3 0.3 0.3 0.2 0.2 0.2 0.1 0.3 1.0 0.9 0.4 0.9 0.1 0.1

26

Compute node weights using similarity with question words

slide-27
SLIDE 27

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type 0.9 1.0 1.0 0.6 1.0 1.0 0.9 1.0 0.9 0.4 0.4 0.4 0.3 0.3 0.3 0.2 0.2 0.2 0.1 0.3 1.0 0.9 0.4 0.9 0.1 0.1

27

slide-28
SLIDE 28

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type 0.9 1.0 1.0 0.6 1.0 1.0 0.9 1.0 0.9 1.0 0.9 0.9

28

slide-29
SLIDE 29

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type 0.9 1.0 1.0 0.6 1.0 1.0 0.9 1.0 0.9 1.0 0.9 0.9

29

Identify cornerstones by thresholding node weights

slide-30
SLIDE 30

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

30

slide-31
SLIDE 31

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

31

slide-32
SLIDE 32

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

32

Answers lie on paths connecting cornerstones

slide-33
SLIDE 33

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

33

slide-34
SLIDE 34

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

34

slide-35
SLIDE 35

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

35

Shorter paths are better

slide-36
SLIDE 36

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

36

slide-37
SLIDE 37

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

37

0.9 0.8 1.0 1.0 0.8 1.0 0.9 0.8 0.7 0.9 0.9 0.9 1.0 1.0 0.8 0.9 0.5 0.6 0.7 0.8 0.7 0.4 0.3 0.8 0.8 0.8 0.6 0.8 0.9 0.9 0.8 0.4

slide-38
SLIDE 38

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

38

0.9 0.8 1.0 1.0 0.8 1.0 0.9 0.8 0.7 0.9 0.9 0.9 1.0 1.0 0.8 0.9 0.5 0.6 0.7 0.8 0.7 0.4 0.3 0.8 0.8 0.8 0.6 0.8 0.9 0.9 0.8 0.4

Insert edge weights: ★ Word proximities in documents ★ Lexicon similarities

slide-39
SLIDE 39

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

39

0.9 0.8 1.0 1.0 0.8 1.0 0.9 0.8 0.7 0.9 0.9 0.9 1.0 1.0 0.8 0.9 0.5 0.6 0.7 0.8 0.7 0.4 0.3 0.8 0.8 0.8 0.6 0.8 0.9 0.9 0.8 0.4

slide-40
SLIDE 40

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type 0.9 0.8 0.8 0.9 0.9 0.8 0.8 0.9

40

0.6 0.7 0.4 0.3 0.8 0.8 0.4

slide-41
SLIDE 41

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type 0.9 0.8 0.8 0.9 0.9 0.8 0.8 0.9

41

0.6 0.7 0.4 0.3 0.8 0.8 0.4

Higher weight paths are better

slide-42
SLIDE 42

42

Formalizing Intuitions: What we Have

★ Quasi-KG: ★ Answers on paths connecting cornerstones ★ Higher weight paths are preferred ★ Shorter paths are preferred Mapping nodes and edges to types Nodes and edges Mapping nodes and edges to weights

slide-43
SLIDE 43

43

Steiner Trees

★ Given: ○ Undirected and weighted graph G ○ Subset of vertices = Terminals ★ Find: ○ Minimum cost tree containing all terminals ★ Cornerstones = Terminals! ★ Costs = 1 - weights

slide-44
SLIDE 44

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type 0.9 1.0 1.0 0.6 1.0 1.0 0.9 1.0 0.9 1.0 0.9 0.9

44

slide-45
SLIDE 45

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type 0.9 1.0 1.0 0.6 1.0 1.0 0.9 1.0 0.9 1.0 0.9 0.9

45

slide-46
SLIDE 46

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type 0.9 1.0 1.0 0.6 1.0 1.0 0.9 1.0 0.9 1.0 0.9 0.9

46

Cornerstones appear in groups

slide-47
SLIDE 47

47

Group Steiner Trees (GST)

★ Contains at least one terminal from each group ★ Compute Group Steiner Trees on quasi-KG ★ Method from Ding et al. [ICDE 2007] ★ Dynamic programming exponential in #keywords ★ O(n log n) in graph size

slide-48
SLIDE 48

Inception lost to won The movie Inception Golden Globes 83rd Academy Awards 2011 Oscars Best Sound Best Screenplay Golden Globe Awards announced The Social Network announced type science thriller Chris Nolan director of Nolan directed Inception’s script Question: Which Nolan films won an Oscar but missed a Golden Globe? edited by Best Actor declared at missed out winner of nominated film type

48

slide-49
SLIDE 49

Inception lost to won Golden Globes 2011 Oscars Best Sound Best Screenplay announced The Social Network announced Nolan directed Question: Which Nolan films won an Oscar but missed a Golden Globe? winner of film type

49

slide-50
SLIDE 50

Inception lost to won Golden Globes 2011 Oscars Best Sound Best Screenplay announced The Social Network announced Nolan directed Question: Which Nolan films won an Oscar but missed a Golden Globe? winner of film type

50

★ Answers are nodes ★ Cornerstones are not answers ★ Answers are entities ★ Answers respect type constraints ★ Answers ranked by intuitive criteria

slide-51
SLIDE 51

Inception lost to won Best Sound Best Screenplay announced The Social Network announced directed Question: Which Nolan films won an Oscar but missed a Golden Globe? winner of type

★ Answers are nodes ★ Cornerstones are not answers ★ Answers are entities ★ Answers respect type constraints ★ Answers ranked by intuitive criteria

51

slide-52
SLIDE 52

Inception Best Sound Best Screenplay The Social Network Question: Which Nolan films won an Oscar but missed a Golden Globe?

★ Answers are nodes ★ Cornerstones are not answers ★ Answers are entities ★ Answers respect type constraints ★ Answers ranked by intuitive criteria

52

slide-53
SLIDE 53

Inception The Social Network Question: Which Nolan films won an Oscar but missed a Golden Globe?

★ Answers are nodes ★ Cornerstones are not answers ★ Answers are entities ★ Answers respect type constraints ★ Answers ranked by intuitive criteria

53

slide-54
SLIDE 54

Inception The Social Network Question: Which Nolan films won an Oscar but missed a Golden Globe?

Sum of node weights 9.75 Number of GSTs 5 Sum of tree costs 6.78 Distance to cornerstones 19 Sum of node weights 8.56 Number of GSTs 2 Sum of tree costs 7.98 Distance to cornerstones 27

54

★ Answers are nodes ★ Cornerstones are not answers ★ Answers are entities ★ Answers respect type constraints ★ Answers ranked by intuitive criteria

slide-55
SLIDE 55

Inception Question: Which Nolan films won an Oscar but missed a Golden Globe?

Sum of node weights 9.75 Number of GSTs 5 Sum of tree costs 6.78 Distance to cornerstones 19

55

★ Answers are nodes ★ Cornerstones are not answers ★ Answers are entities ★ Answers respect type constraints ★ Answers ranked by intuitive criteria

slide-56
SLIDE 56

56

Experimental Results: Setup

★ Benchmarks ○ 150 Complex questions from WikiAnswers [Abujabal et al. 2017] ○ 150 Complex questions from Google Trends ○ Corpus: Google Top-10 / Stratified sampling from Web / News / Wikipedia ★ Metric ○ Mean Reciprocal Rank ○ Precision@1 ○ Hit@5 ★ Baselines ○ Neural QA ○ Graph methods

slide-57
SLIDE 57

57

Experimental Results: Performance

QUEST outperforms

  • thers
slide-58
SLIDE 58

58

Experimental Results: Performance

Cannot join multi-doc evidence

slide-59
SLIDE 59

59

Experimental Results: Performance

GSTs help

slide-60
SLIDE 60

60

Experimental Results: Anecdotes

★ Which Japanese baseball player was contracted for Los Angeles Angels who also played for Hokkaido Nippon-ham fighters? ★ Which aspiring model split with Chloe Moretz and is dating Lexi Wood? ★ Where did Sylvie Vartan meet her future husband Johnny Hallyday?

slide-61
SLIDE 61

Demo, code and data: http://qa.mpi-inf.mpg.de/quest/

Thank you!

Contributions in QUEST

★ Entity answers to complex questions from multi-document quasi-KG ★ Design rationale: Allow noise and cope later ★ Unsupervised method using Group Steiner Trees ★ Robust to noise and ungrammatical constructs ★ No decomposition of question necessary ★ Provides explanatory evidence