KnowledgeStore Scalable Framework for Interlinking Text and - - PowerPoint PPT Presentation

▶
knowledgestore
SMART_READER_LITE
LIVE PREVIEW

KnowledgeStore Scalable Framework for Interlinking Text and - - PowerPoint PPT Presentation

Mar 19, 2024 256 likes •710 views

KnowledgeStore Scalable Framework for Interlinking Text and Knowledge Marco Rospocher, Bernardo Magnini, Luciano Serafini Fondazione Bruno Kessler (FBK) INTRODUCTION Information is typically available both in unstructured and

slide-1
SLIDE 1

KnowledgeStore

— Scalable Framework for Interlinking Text and Knowledge — Marco Rospocher, Bernardo Magnini, Luciano Serafini Fondazione Bruno Kessler (FBK)

slide-2
SLIDE 2

INTRODUCTION

  • Information is typically available both in unstructured

and structured form

  • Deep and Large scale NLP now enables to bridge the

two “world”

  • Development of frameworks for integrating

unstructured and structured content only partially investigated

slide-3
SLIDE 3

KnowledgeStore

  • A scalable, fault-tolerant, and Semantic Web

grounded storage system to jointly store, manage, retrieve, and query, both structured and unstructured data

slide-4
SLIDE 4
  • Among a collection of news articles, a user is

interested in retrieving all 2014 articles reporting statements of a 20th century US president where he is positively mentioned as “commander-in-chief”.

KnowledgeStore

Motivating scenario

slide-5
SLIDE 5
  • Among a collection of news articles, a user is

interested in retrieving all 2014 articles reporting statements of a 20th century US president where he is positively mentioned as “commander-in-chief”.

KnowledgeStore

Motivating scenario

slide-6
SLIDE 6

KnowledgeStore

In a nutshell

slide-7
SLIDE 7

KnowledgeStore

Exploitation

  • Enhanced applications (e.g., decision support

systems)

  • Developing, debugging, training, and evaluating NLP

and knowledge processing tasks

  • Reasoning on Extracted Information (e.g., on Events)
  • Text Exploration
slide-8
SLIDE 8
  • A walk through the KnowledgeStore
  • The KnowledgeStore “live”
  • The KnowledgeStore in NewsReader
  • Next Challenges

OUTLINE

slide-9
SLIDE 9

KnowledgeStore

Functional View

slide-10
SLIDE 10

KnowledgeStore

Data Model

  • It’s Flexible
  • It’s an OWL 2 Ontology
slide-11
SLIDE 11

KnowledgeStore

Example: Data Model For Event Extraction from News

slide-12
SLIDE 12

KnowledgeStore

Architectural View

Java applications KnowledgeStore Frontend Server KnowledgeStore Java client Hadoop HDFS

(name & data nodes)

HBase (multiple server nodes) Virtuoso

(single node)

Mention Resource Entity Axiom Context Representation

Any application

(HTTP access to the KS, possibly exploiting SPARQL client libraries)

Zookeeper

(mult. nodes) distributed synchronization

Client-side Server-side

SPARQL endpoint CRUD endpoint

OMID

(single node) transaction manager

slide-13
SLIDE 13
  • The KnowledgeStore User Interface
  • lookup: given the URI of an object (i.e., resource,

mention, entity), retrieves all the content about it

  • SPARQL: run arbitrary queries against the

SPARQL endpoint

KnowledgeStore

Looking through the glass box

slide-14
SLIDE 14

KnowledgeStore

ICT 316404 FP7-ICT-2011-8 www.newsreader-project.eu

recording history by processing massive streams of daily news

Jan 2013 - Dec 2015

slide-15
SLIDE 15

KnowledgeStore in

slide-16
SLIDE 16

Dataset used in the Demo (NewsReader Project):

  • Domain: Global Automotive Industry
  • 1.3M news documents (2003-2013), provided by LexisNexis (www.lexisnexis.nl)
  • 1.3M NLP annotation files (NAF format) obtained processing the news (NewsReader

NLP Pipeline)

  • 205M mentions of events, persons, organisations, locations, time expressions...
  • 535M of RDF triples about events, persons, organisations, locations, time

expressions...:

  • 439M extracted from text
  • 96M coming from selected background knowledge (DBpedia)

KnowledgeStore

LIVE DEMO

slide-17
SLIDE 17

KnowledgeStore in

slide-18
SLIDE 18

KnowledgeStore in

Decision Making on top of the KnowledgeStore

slide-19
SLIDE 19

KnowledgeStore in

Exploited in Three Hack Day Events

slide-20
SLIDE 20
  • Capable of handling:
  • large number of requests (>110K)
  • multiple concurrent requests (40 requests/sec.)
  • low response time (30-214ms)

KnowledgeStore in

Exploited in Three Hack Day Events

slide-21
SLIDE 21
  • Inferring Knowledge not Explicitly Mentioned in Text (powered by Event

Situation Ontology - ESO)

  • Example: “Yesterday, Chrysler hired Jim Press to lead its sales and marketing”

KnowledgeStore

Reasoning on Events

2007 2006 2005 2008 2009 2010

slide-22
SLIDE 22
  • Inferring Knowledge not Explicitly Mentioned in Text (powered by Event

Situation Ontology - ESO)

  • Example: “Yesterday, Chrysler hired Jim Press to lead its sales and marketing”

KnowledgeStore

Reasoning on Events

Even ent: hire Even ent typ ype: e: ESO:JoiningAnOrganization Even ent roles: roles: ESO:employer dbpedia:Chrysler ESO:employee dbpedia:Jim_Press

At time: 2007/09/07 2007 2006 2005 2008 2009 2010

slide-23
SLIDE 23
  • Inferring Knowledge not Explicitly Mentioned in Text (powered by Event

Situation Ontology - ESO)

  • Example: “Yesterday, Chrysler hired Jim Press to lead its sales and marketing”

KnowledgeStore

Reasoning on Events

Even ent: hire Even ent typ ype: e: ESO:JoiningAnOrganization Even ent roles: roles: ESO:employer dbpedia:Chrysler ESO:employee dbpedia:Jim_Press

At time: 2007/09/07 2007 2006 2005 2008 2009 2010

dbpedia:Jim_Press ESO:notEmployedAt dbpedia:Chrysler

pre-Situation

slide-24
SLIDE 24
  • Inferring Knowledge not Explicitly Mentioned in Text (powered by Event

Situation Ontology - ESO)

  • Example: “Yesterday, Chrysler hired Jim Press to lead its sales and marketing”

KnowledgeStore

Reasoning on Events

Even ent: hire Even ent typ ype: e: ESO:JoiningAnOrganization Even ent roles: roles: ESO:employer dbpedia:Chrysler ESO:employee dbpedia:Jim_Press

At time: 2007/09/07 2007 2006 2005 2008 2009 2010

dbpedia:Jim_Press ESO:notEmployedAt dbpedia:Chrysler

pre-Situation

dbpedia:Jim_Press ESO:employedAt dbpedia:Chrysler

post-Situation

slide-25
SLIDE 25
  • Applied on the 1.3M global automotive industry news
  • Extremely fast: 1,333s (~22m) to process ~500M triples
  • Triggered 2M new triples (i.e., not in the text), organised in

397,885 situations

  • 255,470 events have at least a pre/post/during situation:
  • 71.2% of the events having at least two distinct roles

KnowledgeStore

Reasoning on Events

slide-26
SLIDE 26

KnowledgeStore

Looking ahead to the future…

slide-27
SLIDE 27
  • What knowledge can be additionally inferred from what mentioned in text?

KnowledgeStore

Beyond “Asserted” Knowledge…

2009 2008 2007 2010 2011 2012

slide-28
SLIDE 28
  • What knowledge can be additionally inferred from what mentioned in text?

KnowledgeStore

Beyond “Asserted” Knowledge…

2009 2008 2007 2010 2011 2012

Even ent: hire Even ent typ ype: e: ESO:JoiningAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A

At time: 2007/09/07

A ESO:employedAt B

post-Situation

A ESO:notEmployedAt B

pre-Situation

slide-29
SLIDE 29
  • What knowledge can be additionally inferred from what mentioned in text?

KnowledgeStore

Beyond “Asserted” Knowledge…

2009 2008 2007 2010 2011 2012

Even ent: hire Even ent typ ype: e: ESO:JoiningAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A

At time: 2007/09/07

A ESO:employedAt B

post-Situation

A ESO:notEmployedAt B

pre-Situation

Even ent: fire Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer C ESO:employee A

At time: 2011/08/05

A ESO:employedAt C

pre-Situation

A ESO:notEmployedAt C

post-Situation

slide-30
SLIDE 30
  • What knowledge can be additionally inferred from what mentioned in text?

KnowledgeStore

Beyond “Asserted” Knowledge…

2009 2008 2007 2010 2011 2012

Even ent: hire Even ent typ ype: e: ESO:JoiningAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A

At time: 2007/09/07

A ESO:employedAt B

post-Situation

A ESO:notEmployedAt B

pre-Situation

Even ent: fire Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer C ESO:employee A

At time: 2011/08/05

A ESO:employedAt C

pre-Situation

A ESO:notEmployedAt C

post-Situation

slide-31
SLIDE 31
  • What knowledge can be additionally inferred from what mentioned in text?

KnowledgeStore

Beyond “Asserted” Knowledge…

2009 2008 2007 2010 2011 2012

Even ent: hire Even ent typ ype: e: ESO:JoiningAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A

At time: 2007/09/07

A ESO:employedAt B

post-Situation

A ESO:notEmployedAt B

pre-Situation

Even ent: fire Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer C ESO:employee A

At time: 2011/08/05

A ESO:employedAt C

pre-Situation

A ESO:notEmployedAt C

post-Situation

Even ent: fire Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A
slide-32
SLIDE 32
  • What knowledge can be additionally inferred from what mentioned in text?

KnowledgeStore

Beyond “Asserted” Knowledge…

2009 2008 2007 2010 2011 2012

Even ent: hire Even ent typ ype: e: ESO:JoiningAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A

At time: 2007/09/07

A ESO:employedAt B

post-Situation

A ESO:notEmployedAt B

pre-Situation

Even ent: fire Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer C ESO:employee A

At time: 2011/08/05

A ESO:employedAt C

pre-Situation

A ESO:notEmployedAt C

post-Situation

Even ent: fire Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A Even ent: resign Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A
slide-33
SLIDE 33
  • What knowledge can be additionally inferred from what mentioned in text?

KnowledgeStore

Beyond “Asserted” Knowledge…

2009 2008 2007 2010 2011 2012

Even ent: hire Even ent typ ype: e: ESO:JoiningAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A

At time: 2007/09/07

A ESO:employedAt B

post-Situation

A ESO:notEmployedAt B

pre-Situation

Even ent: fire Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer C ESO:employee A

At time: 2011/08/05

A ESO:employedAt C

pre-Situation

A ESO:notEmployedAt C

post-Situation

Even ent: fire Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A Even ent: resign Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A

Even ent: hire Even ent typ ype: e: ESO:JoiningAnOrganization Even ent roles: roles: ESO:employer C ESO:employee A

slide-34
SLIDE 34
  • What knowledge can be additionally inferred from what mentioned in text?

KnowledgeStore

Beyond “Asserted” Knowledge…

2009 2008 2007 2010 2011 2012

Even ent: hire Even ent typ ype: e: ESO:JoiningAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A

At time: 2007/09/07

A ESO:employedAt B

post-Situation

A ESO:notEmployedAt B

pre-Situation

Even ent: fire Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer C ESO:employee A

At time: 2011/08/05

A ESO:employedAt C

pre-Situation

A ESO:notEmployedAt C

post-Situation

Even ent: fire Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A Even ent: resign Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A

Even ent: hire Even ent typ ype: e: ESO:JoiningAnOrganization Even ent roles: roles: ESO:employer C ESO:employee A

Even ent: change registered name Even ent typ ype: e: ChangeRegisteredName Even ent roles: roles: source B target C
slide-35
SLIDE 35
  • What knowledge can be additionally inferred from what mentioned in text?

KnowledgeStore

Beyond “Asserted” Knowledge…

2009 2008 2007 2010 2011 2012

Even ent: hire Even ent typ ype: e: ESO:JoiningAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A

At time: 2007/09/07

A ESO:employedAt B

post-Situation

A ESO:notEmployedAt B

pre-Situation

Even ent: fire Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer C ESO:employee A

At time: 2011/08/05

A ESO:employedAt C

pre-Situation

A ESO:notEmployedAt C

post-Situation

Even ent: fire Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A Even ent: resign Even ent typ ype: e: ESO:LeavingAnOrganization Even ent roles: roles: ESO:employer B ESO:employee A

Even ent: hire Even ent typ ype: e: ESO:JoiningAnOrganization Even ent roles: roles: ESO:employer C ESO:employee A

Even ent: change registered name Even ent typ ype: e: ChangeRegisteredName Even ent roles: roles: source B target C

Can we infer that some event took place? What is the most probable one (if any)?

slide-36
SLIDE 36
  • Knowledge Crystallisation
  • When can a fact automatically extracted from information extraction tools be

considered as “Background Knowledge”?

  • Aspects to be considered:
  • “cleaning” of the data
  • consistency of event
  • “John” has been fired today from company “Alpha AGf”, but the

KnowledgeStore contains the fact that “Alpha AGf” was closed 4 years ago

  • event matching and integration
  • augment background knowledge entities with facts extracted from the

pipeline

  • How To Tackle This? Combining Statistical and Crisp Reasoning?

KnowledgeStore

Crystallising Extracted Knowledge

slide-37
SLIDE 37
  • Integrating knowledge extracted from text (news, twits,…),

pictures, movies…

  • E.g., retrieve all documents and media where Bill Clinton makes

statements about the Army

KnowledgeStore

Beyond Text…

slide-38
SLIDE 38
  • Integrating knowledge extracted from text (news, twits,…),

pictures, movies…

  • E.g., retrieve all documents and media where Bill Clinton makes

statements about the Army

KnowledgeStore

Beyond Text…

"Stripes,"*Dole,*1996*

!

He! (Bill! Clinton)! quickly! became! the! first! civilian! commander;in;! chief!to!salute!his!marine!guards! while! entering! or! exi@ng! an! aircraA.!!

slide-39
SLIDE 39
  • Integrating knowledge extracted from text (news, twits,…),

pictures, movies…

  • E.g., retrieve all documents and media where Bill Clinton makes

statements about the Army

KnowledgeStore

Beyond Text…

"Stripes,"*Dole,*1996*

!

He! (Bill! Clinton)! quickly! became! the! first! civilian! commander;in;! chief!to!salute!his!marine!guards! while! entering! or! exi@ng! an! aircraA.!!

slide-40
SLIDE 40
  • Integrating knowledge extracted from text (news, twits,…),

pictures, movies…

  • E.g., retrieve all documents and media where Bill Clinton makes

statements about the Army

KnowledgeStore

Beyond Text…

"Stripes,"*Dole,*1996*

!

He! (Bill! Clinton)! quickly! became! the! first! civilian! commander;in;! chief!to!salute!his!marine!guards! while! entering! or! exi@ng! an! aircraA.!!

slide-41
SLIDE 41
  • Integrating knowledge extracted from text (news, twits,…),

pictures, movies…

  • E.g., retrieve all documents and media where Bill Clinton makes

statements about the Army

KnowledgeStore

Beyond Text…

"Stripes,"*Dole,*1996*

!

He! (Bill! Clinton)! quickly! became! the! first! civilian! commander;in;! chief!to!salute!his!marine!guards! while! entering! or! exi@ng! an! aircraA.!!

what is a mention in a picture / movie? how to represent it?

slide-42
SLIDE 42
  • Interpreting / extracting / aligning knowledge from different media

(e.g., video, commentary, text, …)

KnowledgeStore

Beyond Text… and even more…

Frame Commentary Knowledge

“Sanchez, Sanchez,. . .

  • goal. Sanchez

equalizes for Chile” dbpedia:Alexis_Sanchez scorestAt 32min “Yellow card for the Chilean defender” dbpedia:Mauricio_Pinilla yellowCardAt 102min “Now is Marcelo turn, to kick the fourth penalty” “Marcelo. . . Goal” dbpedia:Marcelo_Vieira kicks SuppPenalty4 SuppPenalty4 leadsTo goal

slide-43
SLIDE 43

— http://knowledgestore.fbk.eu —

Thank You! Questions?

The 知識ストア

Team

Roldano Cattoni, Francesco Corcoglioniti, Bernardo Magnini, Alessio Palmero Aprosio, Mohammed Qwaider, Marco Rospocher, Luciano Serafini