Joint Posterior Revision of NLP Annotations via Ontological - - PowerPoint PPT Presentation

▶
joint posterior revision of nlp annotations via
SMART_READER_LITE
LIVE PREVIEW

Joint Posterior Revision of NLP Annotations via Ontological - - PowerPoint PPT Presentation

Dec 03, 2022 98 likes •620 views

#4479 Joint Posterior Revision of NLP Annotations via Ontological Knowledge Marco Rospocher Francesco Corcoglioniti Context: Knowledge Extraction Kia has hired Peter Schreyer as chief design officer. Joint Posterior Revision of NLP

slide-1
SLIDE 1

Marco Rospocher Francesco Corcoglioniti

Joint Posterior Revision of NLP Annotations via Ontological Knowledge

#4479

slide-2
SLIDE 2

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Context: Knowledge Extraction

Kia has hired Peter Schreyer as chief design officer.

slide-3
SLIDE 3

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Context: Knowledge Extraction

Kia has hired Peter Schreyer as chief design officer.

slide-4
SLIDE 4

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Context: Knowledge Extraction

Kia has hired Peter Schreyer as chief design officer.

NLP Tasks:

  • Named Entity Recognition and Classification (NERC)

Organization

slide-5
SLIDE 5

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Context: Knowledge Extraction

Kia has hired Peter Schreyer as chief design officer.

NLP Tasks:

  • Named Entity Recognition and Classification (NERC)

Organization

  • Entity Linking (EL)

dbpedia:Kia_Motors

slide-6
SLIDE 6

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Context: Knowledge Extraction

Kia has hired Peter Schreyer as chief design officer.

NLP Tasks:

  • Named Entity Recognition and Classification (NERC)

Organization

  • Entity Linking (EL)

dbpedia:Kia_Motors

  • Semantic Role Labeling (SRL)

…

framenet:employer

slide-7
SLIDE 7

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Motivating Examples

  • Mr. Washington was runner-up at Wimbledon in 1996.
slide-8
SLIDE 8

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Motivating Examples

  • Mr. Washington was runner-up at Wimbledon in 1996.

http://nlp.stanford.edu:8080/corenlp

slide-9
SLIDE 9

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Motivating Examples

  • Mr. Washington was runner-up at Wimbledon in 1996.

http://nlp.stanford.edu:8080/corenlp http://demo.dbpedia-spotlight.org

http://dbpedia.org/resource/ Washington_(state)

slide-10
SLIDE 10

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Motivating Examples

  • Mr. Washington was runner-up at Wimbledon in 1996.

http://nlp.stanford.edu:8080/corenlp http://demo.dbpedia-spotlight.org

http://dbpedia.org/resource/ Washington_(state)

The GW Bridge is a double-decked suspension bridge over the Hudson.

slide-11
SLIDE 11

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Motivating Examples

  • Mr. Washington was runner-up at Wimbledon in 1996.

http://nlp.stanford.edu:8080/corenlp http://demo.dbpedia-spotlight.org

http://dbpedia.org/resource/ Washington_(state)

The GW Bridge is a double-decked suspension bridge over the Hudson.

http://demo.dbpedia-spotlight.org

http://dbpedia.org/resource/ George_Washington_Bridge

slide-12
SLIDE 12

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Motivating Examples

  • Mr. Washington was runner-up at Wimbledon in 1996.

http://nlp.stanford.edu:8080/corenlp http://demo.dbpedia-spotlight.org

http://dbpedia.org/resource/ Washington_(state)

The GW Bridge is a double-decked suspension bridge over the Hudson.

http://nlp.stanford.edu:8080/corenlp http://demo.dbpedia-spotlight.org

http://dbpedia.org/resource/ George_Washington_Bridge

slide-13
SLIDE 13

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Abstracting

… token1 token2 token3 token4 token5 token6 ….

slide-14
SLIDE 14

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Abstracting

… token1 token2 token3 token4 token5 token6 ….

slide-15
SLIDE 15

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Abstracting

… token1 token2 token3 token4 token5 token6 ….

Task1 Taskn Task2

slide-16
SLIDE 16

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Abstracting

… token1 token2 token3 token4 token5 token6 ….

Task1 Taskn Task2 a1,1 a1,2 a1,k … a2,1 a2,2 a2,i … an,1 an,2 an,j …

slide-17
SLIDE 17

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Abstracting

… token1 token2 token3 token4 token5 token6 ….

Task1 Taskn Task2 a1,1 a1,2 a1,k … a2,1 a2,2 a2,i … an,1 an,2 an,j …

slide-18
SLIDE 18

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Abstracting

… token1 token2 token3 token4 token5 token6 ….

Task1 Taskn Task2 a1,1 a1,2 a1,k … a2,1 a2,2 a2,i … an,1 an,2 an,j …

slide-19
SLIDE 19

How can we assess and improve the coherence of the various NLP annotations on an entity mention?

RESEARCH PROBLEM

slide-20
SLIDE 20

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

In a nutshell

… token1 token2 token3 token4 token5 token6 ….

Task1 Taskn Task2 a1,1 a1,2 a1,k … a2,1 a2,2 a2,i … an,1 an,2 an,j …

  • ntological background knowledge
slide-21
SLIDE 21

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

In a nutshell

… token1 token2 token3 token4 token5 token6 ….

Task1 Taskn Task2 a1,1 a1,2 a1,k … a2,1 a2,2 a2,i … an,1 an,2 an,j …

  • ntological background knowledge
slide-22
SLIDE 22

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

In a nutshell

… token1 token2 token3 token4 token5 token6 ….

Task1 Taskn Task2 a1,1 a1,2 a1,k … a2,1 a2,2 a2,i … an,1 an,2 an,j …

  • ntological background knowledge
slide-23
SLIDE 23

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

In a nutshell

… token1 token2 token3 token4 token5 token6 ….

Task1 Taskn Task2 a1,1 a1,2 a1,k … a2,1 a2,2 a2,i … an,1 an,2 an,j …

  • ntological background knowledge
slide-24
SLIDE 24

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Contributions

1. JPARK: a probabilistic model capable to estimate a posteriori the overall confidence of NLP annotations 2. A concrete instantiation of the model for NERC and EL (using YAGO as ontological knowledge) 3. Application of the NERC and EL model to revise the annotations of Stanford NER and DBpedia Spotlight

slide-25
SLIDE 25
slide-26
SLIDE 26

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

The Model

P (a|m, B, K )

slide-27
SLIDE 27

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

The Model

P (a|m, B, K )

(ai , … , an) NLP Annotations

entity mention NLP Background Knowledge “The” Ontological Knowledge

slide-28
SLIDE 28

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

The Model

P (a|m, B, K )

(ai , … , an) NLP Annotations

entity mention NLP Background Knowledge “The” Ontological Knowledge

P (a ,C|m, B, K )

set of classes from K

slide-29
SLIDE 29

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

The Model

P (a|m, B, K ) P (ai|m, B ) P (C|ai , K )

(ai , … , an) NLP Annotations

entity mention NLP Background Knowledge “The” Ontological Knowledge

P (a ,C|m, B, K )

set of classes from K

slide-30
SLIDE 30

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

The Model

confidence score P (a|m, B, K ) P (ai|m, B ) P (C|ai , K )

(ai , … , an) NLP Annotations

entity mention NLP Background Knowledge “The” Ontological Knowledge

P (a ,C|m, B, K )

set of classes from K

slide-31
SLIDE 31

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

The Model

confidence score learned from data P (a|m, B, K ) P (ai|m, B ) P (C|ai , K )

(ai , … , an) NLP Annotations

entity mention NLP Background Knowledge “The” Ontological Knowledge

P (a ,C|m, B, K )

set of classes from K

slide-32
SLIDE 32

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

The Model

confidence score learned from data P (a|m, B, K ) = arg maxa P (ai|m, B ) P (C|ai , K )

(ai , … , an) NLP Annotations

entity mention NLP Background Knowledge “The” Ontological Knowledge

P (a ,C|m, B, K )

set of classes from K

slide-33
SLIDE 33

NERC and EL Model

slide-34
SLIDE 34

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Ingredients

  • Ontological Knowledge
  • Estimating
  • Estimating P (C|aEL , K )

P (C|aNERC , K )

slide-35
SLIDE 35

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Ingredients

  • Ontological Knowledge
  • Estimating
  • Estimating P (C|aEL , K )

P (C|aNERC , K )

slide-36
SLIDE 36

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Ingredients

  • Ontological Knowledge
  • Estimating
  • Estimating

Leverage a gold standard corpus G annotated with NERC types and

  • ntological classes (or EL annotations)

P (C|aEL , K ) P (C|aNERC , K )

slide-37
SLIDE 37

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Ingredients

  • Ontological Knowledge
  • Estimating
  • Estimating

Leverage a gold standard corpus G annotated with NERC types and

  • ntological classes (or EL annotations)

# co-occurences

P (C|aEL , K ) P (C|aNERC , K ) _ ∑C nG(C , aNERC) nG(C , aNERC) ~

slide-38
SLIDE 38

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Ingredients

  • Ontological Knowledge
  • Estimating
  • Estimating

Leverage a gold standard corpus G annotated with NERC types and

  • ntological classes (or EL annotations)

# co-occurences

P (C|aEL , K ) P (C|aNERC , K ) _ ∑C nG(C , aNERC) nG(C , aNERC) ~

Leverage alignments between EL Knowledge Base and

slide-39
SLIDE 39

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Ingredients

  • Ontological Knowledge
  • Estimating
  • Estimating

Leverage a gold standard corpus G annotated with NERC types and

  • ntological classes (or EL annotations)

# co-occurences

P (C|aEL , K ) P (C|aNERC , K ) _ ∑C nG(C , aNERC) nG(C , aNERC) ~

Leverage alignments between EL Knowledge Base and

{

1 entity aEL is instance of C 0 otherwise

slide-40
SLIDE 40

Application and Evaluation

slide-41
SLIDE 41

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Tools

  • NERC: [Finkel et al., 2005]
  • EL: [Daiber et al., 2013]
slide-42
SLIDE 42

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

NERC+EL Datasets

  • AIDA CoNLL-YAGO [Hoffart et al., 2011]
  • MEANTIME [Minard et al., 2016]
  • TAC-KBP [ Ji et al., 2011]
slide-43
SLIDE 43

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Does the posteriori joint revision of the annotations from Stanford NER and DBpedia Spotlight, via YAGO, improve their NERC and EL performances?

Research Question

slide-44
SLIDE 44

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Does the posteriori joint revision of the annotations from Stanford NER and DBpedia Spotlight, via YAGO, improve their NERC and EL performances?

Research Question

slide-45
SLIDE 45

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Results

Bold = statistical significant (approx. rand. test)

slide-46
SLIDE 46

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Results

Bold = statistical significant (approx. rand. test)

slide-47
SLIDE 47

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Results

Bold = statistical significant (approx. rand. test)

slide-48
SLIDE 48

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Does the posteriori joint revision of the annotations from Stanford NER and DBpedia Spotlight, via YAGO, improve their NERC and EL performances?

Research Question

slide-49
SLIDE 49

Joint Posterior Revision of NLP Annotations via Ontological Knowledge - M. Rospocher & F. Corcoglioniti

Conclusions

  • Novel probabilistic model, leveraging ontological knowledge,

for improving NLP entity annotations

  • Instantiation of the model for the NERC and EL tasks
  • Empirical confirmation (3 datasets) of the capability of the

model to improve the quality of the annotations

  • Future Work: extension to other tasks (e.g., SRL)
slide-50
SLIDE 50

Marco Rospocher

rospocher@fbk.eu dkm.fbk.eu/rospocher @marcorospocher

KE4IR

PIKES powered by

pikes.fbk.eu/ke4ir premon.fbk.eu

RDFpro

rdfpro.fbk.eu

MoKi

  • moki.fbk.eu

knowledgestore.fbk.eu pikes.fbk.eu

PIKES

pikes.fbk.eu/jpark pikes.fbk.eu/psl4ea

BPMN Ontology

dkm.fbk.eu/bpmn-ontology bit.ly/pescado-onto github.com/dkmfbk/TexOwl

Event & Situation Ontology

github.com/newsreader/eso