KE4IR S E y K b d e r e I w P o p Knowledge Extraction - - PowerPoint PPT Presentation

▶
ke4ir
SMART_READER_LITE
LIVE PREVIEW

KE4IR S E y K b d e r e I w P o p Knowledge Extraction - - PowerPoint PPT Presentation

May 06, 2023 576 likes •1.24k views

KE4IR S E y K b d e r e I w P o p Knowledge Extraction for Information Retrieval Marco Rospocher rospocher@fbk.eu dkm.fbk.eu/rospocher @marcorospocher joint work with: Francesco Corcoglioniti, Mauro Dragoni, Alessio Palmero

slide-1
SLIDE 1

joint work with:

13th ESWC 2016 | 29 May - 2 June 2016 | Anissaras, Crete, Greece

Francesco Corcoglioniti, Mauro Dragoni, Alessio Palmero Aprosio

Marco Rospocher

rospocher@fbk.eu dkm.fbk.eu/rospocher @marcorospocher

Knowledge Extraction for Information Retrieval

KE4IR

P I K E S p
  • w
e r e d b y
slide-2
SLIDE 2 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Main Message

  • Exploiting the knowledge extracted from
  • queries
  • documents

improves Document Retrieval performances!

slide-3
SLIDE 3 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Outline

  • Document Retrieval and Motivation
  • Our approach: KE4IR
  • Evaluation: Results and Findings
slide-4
SLIDE 4 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Document Retrieval

slide-5
SLIDE 5 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Document Retrieval

astronomers influenced by Gauss

slide-6
SLIDE 6 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Document Retrieval

astronomers influenced by Gauss

slide-7
SLIDE 7 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Document Retrieval

astronomers influenced by Gauss

slide-8
SLIDE 8 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Motivation

Overcome limitations of traditional IR

  • Traditional IR systems match the terms or possible term-

based expansions (e.g., synonyms, related terms)

  • Issues:
  • relevant documents may not contain all the query terms
  • a document having all terms is not necessarily highly relevant
slide-9
SLIDE 9 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y
  • Complement textual terms with semantic terms extracted

from queries and documents In a nutshell

KE4IR

P I K E S powered by
slide-10
SLIDE 10 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y
  • Complement textual terms with semantic terms extracted

from queries and documents In a nutshell

KE4IR

P I K E S powered by

astronomers influenced by Gauss

slide-11
SLIDE 11 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y
  • Complement textual terms with semantic terms extracted

from queries and documents In a nutshell

KE4IR

P I K E S powered by

astronomers influenced by Gauss

Textual Content

astronom influenc gauss

slide-12
SLIDE 12 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y
  • Complement textual terms with semantic terms extracted

from queries and documents In a nutshell

KE4IR

P I K E S powered by

astronomers influenced by Gauss

Textual Content

astronom influenc gauss dbpedia:Carl_Friedrich_Gauss yago:Astronomer109818343 framebase:Subjective_influence century:1700 …

Semantic Content

slide-13
SLIDE 13 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y
  • Mentions are snippets of text denoting entities, events and

relations

  • One mention A set of semantic terms
  • Relevance of a semantic term: number of mentions a term

derives from Mentions & Semantic Terms astronomers influenced by Gauss

KE4IR

P I K E S powered by
slide-14
SLIDE 14 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y
  • Mentions are snippets of text denoting entities, events and

relations

  • One mention A set of semantic terms
  • Relevance of a semantic term: number of mentions a term

derives from Mentions & Semantic Terms astronomers influenced by Gauss

KE4IR

P I K E S powered by
slide-15
SLIDE 15 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Semantic Layers - 1. URI (aka “entities”) astronomers influenced by Gauss

KE4IR

P I K E S powered by
slide-16
SLIDE 16 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Semantic Layers - 1. URI (aka “entities”) astronomers influenced by Gauss dbpedia:Carl_Friedrich_Gauss

KE4IR

P I K E S powered by
slide-17
SLIDE 17 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Semantic Layers - 2. TYPES astronomers influenced by Gauss

KE4IR

P I K E S powered by
slide-18
SLIDE 18 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Semantic Layers - 2. TYPES astronomers influenced by Gauss yago:Astronomer109818343

KE4IR

P I K E S powered by
slide-19
SLIDE 19 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Semantic Layers - 2. TYPES astronomers influenced by Gauss dbpedia:Carl_Friedrich_Gauss yago:GermanMathematicians yago:Astronomer109818343

KE4IR

P I K E S powered by
slide-20
SLIDE 20 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Semantic Layers - 3. FRAME astronomers influenced by Gauss

KE4IR

P I K E S powered by
slide-21
SLIDE 21 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Semantic Layers - 3. FRAME astronomers influenced by Gauss

KE4IR

P I K E S powered by

influenced Gauss astronomers

slide-22
SLIDE 22 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Semantic Layers - 3. FRAME astronomers influenced by Gauss

KE4IR

P I K E S powered by

framebase:Subjective_influence framebase:Subjective_influence-Cognizer framebase:Subjective_influence-Agent

influenced Gauss astronomers

slide-23
SLIDE 23 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Semantic Layers - 3. FRAME astronomers influenced by Gauss

KE4IR

P I K E S powered by

framebase:Subjective_influence framebase:Subjective_influence-Cognizer framebase:Subjective_influence-Agent

influenced Gauss astronomers <framebase:Subjective_influence , dbpedia:Carl_Friedrich_Gauss>

slide-24
SLIDE 24 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Semantic Layers - 4. TIME astronomers influenced by Gauss

KE4IR

P I K E S powered by
slide-25
SLIDE 25 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Semantic Layers - 4. TIME astronomers influenced by Gauss dbpedia:Carl_Friedrich_Gauss dbo:dateOfBirth “1777”

KE4IR

P I K E S powered by
slide-26
SLIDE 26 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Semantic Layers - 4. TIME astronomers influenced by Gauss dbpedia:Carl_Friedrich_Gauss dbo:dateOfBirth “1777” XVIII century

KE4IR

P I K E S powered by
slide-27
SLIDE 27 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Semantic Layers - 4. TIME astronomers influenced by Gauss dbpedia:Carl_Friedrich_Gauss dbo:dateOfBirth “1777” XVIII century century:18

KE4IR

P I K E S powered by

year:1777 decade:177 century:17

slide-28
SLIDE 28 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Summing Up

Layer Term Mentions

TEXTUAL astronom astronomers TEXTUAL influenc influenced TEXTUAL gauss Gauss

astronomers influenced by Gauss

KE4IR

P I K E S powered by
slide-29
SLIDE 29 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Summing Up

Layer Term Mentions

TEXTUAL astronom astronomers TEXTUAL influenc influenced TEXTUAL gauss Gauss

astronomers influenced by Gauss

KE4IR

P I K E S powered by URI dbpedia:Carl Friedrich Gauss Gauss TYPE yago:GermanMathematicians Gauss TYPE yago:NumberTheorists Gauss TYPE yago:FellowsOfTheRoyalSociety Gauss TYPE ...other 18 terms ... Gauss TYPE yago:Astronomer109818343 astronomers, Gauss TYPE yago:Physicist110428004 astronomers, Gauss TYPE yago:Person100007846 astronomers, Gauss TYPE ...other 9 terms ... astronomers, Gauss FRAME ⟨Subjective influence-influence.v, Carl . . . Gauss⟩ influenced FRAME ⟨Subjective influence, Carl Friedrich Gauss⟩ influenced FRAME ⟨Frame, Carl Friedrich Gauss⟩ influenced TIME day:1777-04-30 Gauss TIME day:1855-02-23 Gauss TIME century:17 Gauss TIME ...other 7 terms Gauss
slide-30
SLIDE 30 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Retrieval Model

  • Inspired to the

Vector Space Model (VSM)

  • Queries and documents are represented as vector of terms
  • sim(d,q) > 0 document is relevant for the query

astronomers influenced by Gauss q = (qi) d = (di)

sim = d q

.

KE4IR

P I K E S powered by
slide-31
SLIDE 31 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y
  • Concatenation of layer-specific vectors
  • Three ingredients:
  • Term Frequency (tf)
  • Inverse Document Frequency (idf)
  • Layer weight (w)

KE4IR

P I K E S powered by

Building the vectors

slide-32
SLIDE 32 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Building the vectors: example

Layer Term Mentions

TEXTUAL astronom astronomers TEXTUAL influenc influenced TEXTUAL gauss Gauss URI dbpedia:Carl Friedrich Gauss Gauss TYPE yago:GermanMathematicians Gauss TYPE yago:NumberTheorists Gauss TYPE yago:FellowsOfTheRoyalSociety Gauss TYPE ...other 18 terms ... Gauss TYPE yago:Astronomer109818343 astronomers, Gauss TYPE yago:Physicist110428004 astronomers, Gauss TYPE yago:Person100007846 astronomers, Gauss TYPE ...other 9 terms ... astronomers, Gauss FRAME ⟨Subjective influence-influence.v, Carl . . . Gauss⟩ influenced FRAME ⟨Subjective influence, Carl Friedrich Gauss⟩ influenced FRAME ⟨Frame, Carl Friedrich Gauss⟩ influenced TIME day:1777-04-30 Gauss TIME day:1855-02-23 Gauss TIME century:17 Gauss TIME ...other 7 terms Gauss

astronomers influenced by Gauss

KE4IR

P I K E S powered by
slide-33
SLIDE 33 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Building the vectors: example

Layer Term Mentions

TEXTUAL astronom astronomers TEXTUAL influenc influenced TEXTUAL gauss Gauss URI dbpedia:Carl Friedrich Gauss Gauss TYPE yago:GermanMathematicians Gauss TYPE yago:NumberTheorists Gauss TYPE yago:FellowsOfTheRoyalSociety Gauss TYPE ...other 18 terms ... Gauss TYPE yago:Astronomer109818343 astronomers, Gauss TYPE yago:Physicist110428004 astronomers, Gauss TYPE yago:Person100007846 astronomers, Gauss TYPE ...other 9 terms ... astronomers, Gauss FRAME ⟨Subjective influence-influence.v, Carl . . . Gauss⟩ influenced FRAME ⟨Subjective influence, Carl Friedrich Gauss⟩ influenced FRAME ⟨Frame, Carl Friedrich Gauss⟩ influenced TIME day:1777-04-30 Gauss TIME day:1855-02-23 Gauss TIME century:17 Gauss TIME ...other 7 terms Gauss

astronomers influenced by Gauss

tfi

1.0 1.0 1.0 1.0 0.030 0.030 0.030 0.030 0.114 0.114 0.114 0.114 0.333 0.333 0.333 0.1 0.1 0.1 0.1

KE4IR

P I K E S powered by
slide-34
SLIDE 34 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Building the vectors: example

Layer Term Mentions

TEXTUAL astronom astronomers TEXTUAL influenc influenced TEXTUAL gauss Gauss URI dbpedia:Carl Friedrich Gauss Gauss TYPE yago:GermanMathematicians Gauss TYPE yago:NumberTheorists Gauss TYPE yago:FellowsOfTheRoyalSociety Gauss TYPE ...other 18 terms ... Gauss TYPE yago:Astronomer109818343 astronomers, Gauss TYPE yago:Physicist110428004 astronomers, Gauss TYPE yago:Person100007846 astronomers, Gauss TYPE ...other 9 terms ... astronomers, Gauss FRAME ⟨Subjective influence-influence.v, Carl . . . Gauss⟩ influenced FRAME ⟨Subjective influence, Carl Friedrich Gauss⟩ influenced FRAME ⟨Frame, Carl Friedrich Gauss⟩ influenced TIME day:1777-04-30 Gauss TIME day:1855-02-23 Gauss TIME century:17 Gauss TIME ...other 7 terms Gauss

astronomers influenced by Gauss

tfi

1.0 1.0 1.0 1.0 0.030 0.030 0.030 0.030 0.114 0.114 0.114 0.114 0.333 0.333 0.333 0.1 0.1 0.1 0.1

KE4IR

P I K E S powered by

idfi

2.018 3.404 1.568 3.404 2.624 2.583 1.057 ... 1.432 0.958 0.003 ... 5.802 5.802 3.499 3.404 3.404 0.196 ...

.

slide-35
SLIDE 35 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Building the vectors: example

Layer Term Mentions

TEXTUAL astronom astronomers TEXTUAL influenc influenced TEXTUAL gauss Gauss URI dbpedia:Carl Friedrich Gauss Gauss TYPE yago:GermanMathematicians Gauss TYPE yago:NumberTheorists Gauss TYPE yago:FellowsOfTheRoyalSociety Gauss TYPE ...other 18 terms ... Gauss TYPE yago:Astronomer109818343 astronomers, Gauss TYPE yago:Physicist110428004 astronomers, Gauss TYPE yago:Person100007846 astronomers, Gauss TYPE ...other 9 terms ... astronomers, Gauss FRAME ⟨Subjective influence-influence.v, Carl . . . Gauss⟩ influenced FRAME ⟨Subjective influence, Carl Friedrich Gauss⟩ influenced FRAME ⟨Frame, Carl Friedrich Gauss⟩ influenced TIME day:1777-04-30 Gauss TIME day:1855-02-23 Gauss TIME century:17 Gauss TIME ...other 7 terms Gauss

astronomers influenced by Gauss

tfi

1.0 1.0 1.0 1.0 0.030 0.030 0.030 0.030 0.114 0.114 0.114 0.114 0.333 0.333 0.333 0.1 0.1 0.1 0.1

KE4IR

P I K E S powered by

idfi

2.018 3.404 1.568 3.404 2.624 2.583 1.057 ... 1.432 0.958 0.003 ... 5.802 5.802 3.499 3.404 3.404 0.196 ...

.

wi

0.5 0.5 0.5 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125

.

slide-36
SLIDE 36 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Building the vectors: example

Layer Term Mentions

TEXTUAL astronom astronomers TEXTUAL influenc influenced TEXTUAL gauss Gauss URI dbpedia:Carl Friedrich Gauss Gauss TYPE yago:GermanMathematicians Gauss TYPE yago:NumberTheorists Gauss TYPE yago:FellowsOfTheRoyalSociety Gauss TYPE ...other 18 terms ... Gauss TYPE yago:Astronomer109818343 astronomers, Gauss TYPE yago:Physicist110428004 astronomers, Gauss TYPE yago:Person100007846 astronomers, Gauss TYPE ...other 9 terms ... astronomers, Gauss FRAME ⟨Subjective influence-influence.v, Carl . . . Gauss⟩ influenced FRAME ⟨Subjective influence, Carl Friedrich Gauss⟩ influenced FRAME ⟨Frame, Carl Friedrich Gauss⟩ influenced TIME day:1777-04-30 Gauss TIME day:1855-02-23 Gauss TIME century:17 Gauss TIME ...other 7 terms Gauss

astronomers influenced by Gauss

tfi

1.0 1.0 1.0 1.0 0.030 0.030 0.030 0.030 0.114 0.114 0.114 0.114 0.333 0.333 0.333 0.1 0.1 0.1 0.1

KE4IR

P I K E S powered by

idfi

2.018 3.404 1.568 3.404 2.624 2.583 1.057 ... 1.432 0.958 0.003 ... 5.802 5.802 3.499 3.404 3.404 0.196 ...

.

wi

0.5 0.5 0.5 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125 0.125

.

qi

1.009 1.702 0.784 0.426 0.010 0.010 0.004 ... 0.020 0.014 ∼0 ... 0.242 0.242 0.146 0.043 0.043 0.002 ...

=

slide-37
SLIDE 37 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Implementation

KE4IR

P I K E S powered by
slide-38
SLIDE 38 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Implementation

KE4IR

P I K E S powered by

PIKES

slide-39
SLIDE 39 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

In a nutshell

PIKES[ACM-SAC2016]

slide-40
SLIDE 40 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

In a nutshell

PIKES

Phase1: Linguistic Feature Extraction

[ACM-SAC2016]

slide-41
SLIDE 41 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

In a nutshell

PIKES

Phase1: Linguistic Feature Extraction Phase2: Knowledge Distillation

[ACM-SAC2016]

slide-42
SLIDE 42 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

PIKES

  • State-of-the-art tool for frame-based ontology population
  • FrameBase Ontology Populator
  • Modular nature
  • All output exposed as RDF

+ Named Graph for knowledge tracing

  • Efficiently process large corpora (700K tokens/hour)

Main Characteristics

[ACM-SAC2016]

slide-43
SLIDE 43 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

PIKES

Summary

http://pikes.fbk.eu/

[ACM-SAC2016]

slide-44
SLIDE 44 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

PIKES

Summary

http://pikes.fbk.eu/

[ACM-SAC2016]

slide-45
SLIDE 45 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

PIKES

Summary

http://pikes.fbk.eu/

[ACM-SAC2016]

slide-46
SLIDE 46 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

PIKES

Summary

http://pikes.fbk.eu/

[ACM-SAC2016]

slide-47
SLIDE 47 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

KE4IR

P I K E S powered by
  • 331 documents, 35 queries [Waitelonis et al, 2015]
  • Multi-value relevances (1=irrelevant, 5=relevant)
  • Diverse queries: from keyword-base search to queries

requiring semantic capabilities Evaluation Setup

slide-48
SLIDE 48 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y
  • 2 baselines:
  • Google custom search API
  • Textual layer only (~Lucene)
  • Measures: Prec1,5,10, MAP

, MAP10, NDCG, NDCG10

  • Same weights for textual and semantic layers:
  • TEXTUAL (50%)
  • URI (12,5%), TYPE (12,5%), FRAME (12,5%), TIME (12,5%)

KE4IR

P I K E S powered by

Evaluation Setup

slide-49
SLIDE 49 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

KE4IR

P I K E S powered by

Evaluation Results: Comparison with the baselines

Approach/System Prec1 Prec5 Prec10 NDCG NDCG10 MAP MAP10 Google 0.543 0.411 0.343 0.434 0.405 0.255 0.219 Textual 0.943 0.669 0.453 0.832 0.782 0.733 0.681 KE4IR 0.971 0.680 0.474 0.854 0.806 0.758 0.713

slide-50
SLIDE 50 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

KE4IR

P I K E S powered by

Evaluation Results: Comparison with the baselines

Approach/System Prec1 Prec5 Prec10 NDCG NDCG10 MAP MAP10 Google 0.543 0.411 0.343 0.434 0.405 0.255 0.219 Textual 0.943 0.669 0.453 0.832 0.782 0.733 0.681 KE4IR 0.971 0.680 0.474 0.854 0.806 0.758 0.713 KE4IR vs. Textual 3.03% 1.71% 4.55% 2.64% 2.99% 3.50% 4.74%

slide-51
SLIDE 51 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

KE4IR

P I K E S powered by

Evaluation Results: Comparison with the baselines

Approach/System Prec1 Prec5 Prec10 NDCG NDCG10 MAP MAP10 Google 0.543 0.411 0.343 0.434 0.405 0.255 0.219 Textual 0.943 0.669 0.453 0.832 0.782 0.733 0.681 KE4IR 0.971 0.680 0.474 0.854 0.806 0.758 0.713

statistically significant

KE4IR vs. Textual 3.03% 1.71% 4.55% 2.64% 2.99% 3.50% 4.74%

slide-52
SLIDE 52 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

KE4IR

P I K E S powered by

Evaluation Results: Comparison with the baselines

Approach/System Prec1 Prec5 Prec10 NDCG NDCG10 MAP MAP10 Google 0.543 0.411 0.343 0.434 0.405 0.255 0.219 Textual 0.943 0.669 0.453 0.832 0.782 0.733 0.681 KE4IR 0.971 0.680 0.474 0.854 0.806 0.758 0.713

statistically significant

Knowledge Extraction positively affects the Document Retrieval performances!

KE4IR vs. Textual 3.03% 1.71% 4.55% 2.64% 2.99% 3.50% 4.74%

slide-53
SLIDE 53 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

KE4IR

P I K E S powered by

Evaluation Results: Impact of various layer combinations

Layers (TEXTUAL+) Prec1 Prec5 Prec10 NDCG NDCG10 MAP MAP10

URI,TYPE,FRAME,TIME 0.971 0.680 0.474 0.854 0.806 0.758 0.713 URI,TYPE,FRAME 0.971 0.680 0.474 0.853 0.804 0.757 0.712 URI,TYPE,TIME 0.971 0.680 0.474 0.851 0.802 0.757 0.712 URI,TYPE 0.971 0.680 0.474 0.849 0.801 0.755 0.710 URI,FRAME,TIME 0.971 0.674 0.465 0.844 0.796 0.750 0.702 URI,FRAME 0.971 0.674 0.465 0.842 0.795 0.749 0.702 URI,TIME 0.971 0.674 0.465 0.840 0.791 0.747 0.700 URI 0.971 0.674 0.465 0.837 0.791 0.747 0.700 TYPE,FRAME,TIME 0.943 0.674 0.471 0.848 0.799 0.745 0.700 TYPE,TIME 0.943 0.674 0.471 0.843 0.794 0.743 0.697 TYPE,FRAME 0.943 0.674 0.468 0.847 0.797 0.743 0.695 FRAME,TIME 0.943 0.674 0.462 0.842 0.793 0.741 0.693 TYPE 0.943 0.674 0.468 0.842 0.792 0.740 0.693 TIME 0.943 0.669 0.462 0.836 0.786 0.737 0.689 FRAME 0.943 0.674 0.453 0.839 0.789 0.737 0.686 (only textual) 0.943 0.669 0.453 0.832 0.782 0.733 0.681

slide-54
SLIDE 54 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

KE4IR

P I K E S powered by

Evaluation Results: Impact of various layer combinations

Layers (TEXTUAL+) Prec1 Prec5 Prec10 NDCG NDCG10 MAP MAP10

URI,TYPE,FRAME,TIME 0.971 0.680 0.474 0.854 0.806 0.758 0.713 URI,TYPE,FRAME 0.971 0.680 0.474 0.853 0.804 0.757 0.712 URI,TYPE,TIME 0.971 0.680 0.474 0.851 0.802 0.757 0.712 URI,TYPE 0.971 0.680 0.474 0.849 0.801 0.755 0.710 URI,FRAME,TIME 0.971 0.674 0.465 0.844 0.796 0.750 0.702 URI,FRAME 0.971 0.674 0.465 0.842 0.795 0.749 0.702 URI,TIME 0.971 0.674 0.465 0.840 0.791 0.747 0.700 URI 0.971 0.674 0.465 0.837 0.791 0.747 0.700 TYPE,FRAME,TIME 0.943 0.674 0.471 0.848 0.799 0.745 0.700 TYPE,TIME 0.943 0.674 0.471 0.843 0.794 0.743 0.697 TYPE,FRAME 0.943 0.674 0.468 0.847 0.797 0.743 0.695 FRAME,TIME 0.943 0.674 0.462 0.842 0.793 0.741 0.693 TYPE 0.943 0.674 0.468 0.842 0.792 0.740 0.693 TIME 0.943 0.669 0.462 0.836 0.786 0.737 0.689 FRAME 0.943 0.674 0.453 0.839 0.789 0.737 0.686 (only textual) 0.943 0.669 0.453 0.832 0.782 0.733 0.681

slide-55
SLIDE 55 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

KE4IR

P I K E S powered by
  • General Remarks
  • TYPE & URI: more frequent, less “reliable”
  • FRAME & TIME: less frequent, positively impact
  • Analysis on selected examples

Evaluation Results: Query-by-query analysis

Query Text ∆ NDCG@10 ∆ MAP Nazis confiscate or destroy art and literature 0.154 0.099 Modern Age in English Literature

  • 0.117
  • 0.095

Napoleon’s Russian Campaign 0.151 0.147 First woman who won a Nobel Prize

slide-56
SLIDE 56 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

KE4IR

P I K E S powered by

Balancing Textual vs Semantic Content

0.0 0.2 0.4 0.6 0.8 1.0 0.55 0.60 0.65 0.70 0.75 Semantic Weight MAP textual+semantics textual

slide-57
SLIDE 57 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

KE4IR

P I K E S powered by

Balancing Textual vs Semantic Content

0.0 0.2 0.4 0.6 0.8 1.0 0.55 0.60 0.65 0.70 0.75 Semantic Weight MAP textual+semantics textual

w(semantics) = x w(textual) = 1- x

slide-58
SLIDE 58 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

KE4IR

P I K E S powered by

Balancing Textual vs Semantic Content

0.0 0.2 0.4 0.6 0.8 1.0 0.55 0.60 0.65 0.70 0.75 Semantic Weight MAP textual+semantics textual

Evaluation Setting 0.5

w(semantics) = x w(textual) = 1- x

slide-59
SLIDE 59 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

KE4IR

P I K E S powered by

Balancing Textual vs Semantic Content

0.0 0.2 0.4 0.6 0.8 1.0 0.55 0.60 0.65 0.70 0.75 Semantic Weight MAP textual+semantics textual

Highest MAP for 0.65 Evaluation Setting 0.5

w(semantics) = x w(textual) = 1- x

slide-60
SLIDE 60 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

KE4IR

P I K E S powered by

Balancing Textual vs Semantic Content

0.0 0.2 0.4 0.6 0.8 1.0 0.55 0.60 0.65 0.70 0.75 Semantic Weight MAP textual+semantics textual

0.92

Highest MAP for 0.65 Evaluation Setting 0.5

w(semantics) = x w(textual) = 1- x

slide-61
SLIDE 61 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

KE4IR

P I K E S powered by

Balancing Textual vs Semantic Content

0.0 0.2 0.4 0.6 0.8 1.0 0.55 0.60 0.65 0.70 0.75 Semantic Weight MAP textual+semantics textual

0.92

Too Much Semantics Will Kill You!

Highest MAP for 0.65 Evaluation Setting 0.5

w(semantics) = x w(textual) = 1- x

slide-62
SLIDE 62 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Evaluation Material

KE4IR

P I K E S powered by

http://pikes.fbk.eu/ke4ir/

slide-63
SLIDE 63 Knowledge Extraction for Information Retrieval - Corcoglioniti at al.

KE4IR

P I K E S p
  • w
e r e d b y

Conclusions

  • Exploiting the knowledge extracted from queries and

documents improves IR performances

  • Evaluation results legitimise testing KE4IR in real-world

situations

  • Looking ahead to the future….
  • larger document collections (e.g., TREC WT10g, ClueWeb)
  • favouring precision over recall in KE?
  • domain-adaptation
slide-64
SLIDE 64

Marco Rospocher

rospocher@fbk.eu dkm.fbk.eu/rospocher @marcorospocher pikes.fbk.eu

KE4IR

PIKES p
  • w
e r e d b y

pikes.fbk.eu/ke4ir premon.fbk.eu

RDFpro

rdfpro.fbk.eu

MoKi

  • moki.fbk.eu

knowledgestore.fbk.eu

PIKES