NIFify: Towards Better Quality Entity Linking Datasets Henry - - PowerPoint PPT Presentation

▶

Sep 23, 2022 135 likes •405 views

NIFify: Towards Better Quality Entity Linking Datasets Henry Rosales-M endez, Aidan Hogan and Barbara Poblete University of Chile { hrosales,ahogan,bpoblete } @dcc.uchile.cl May 14, 2019 LA-WEB 2019 - 10th Latin American Web Congress

SLIDE 1

NIFify: Towards Better Quality Entity Linking Datasets †

Henry Rosales-M´ endez, Aidan Hogan and Barbara Poblete

University of Chile {hrosales,ahogan,bpoblete}@dcc.uchile.cl

May 14, 2019

† LA-WEB 2019 - 10th Latin American Web Congress

SLIDE 2

Example

SLIDE 3

Example - Entity Recognition

SLIDE 4

Example - Entity Disambiguation

SLIDE 5

Name Variations in Entity Linking

Michael J. Jackson King of Pop Michael Joseph Jackson

SLIDE 6

Name Variations in Entity Linking

Michael Jackson

SLIDE 7

Are there benchmark datasets to measure Entity Linking results?

SLIDE 8

Overview of popular EL datasets

Dataset Mn Typ Format MSNBC ✗ ✗ MSNBC IITB ✓ ✗ IITB AIDA/CoNLL ✓ ✗ AIDA ACE2004 ✗ ✗ MSNBC AQUAINT ✗ ✗ MSNBC DBpedia Spotlight ✓ ✗ Lexvo KORE50 ✓ ✗ AIDA N3-RSS 500 ✓ ✗ NIF Reuters 128 ✓ ✗ NIF News-100 ✓ ✗ NIF Wes2015 ✓ ✗ NIF SemEval 2015 Task 13 ✓ ✗ SemEval Thibaudet ✗ ✓ RENDEN Bergson ✗ ✓ RENDEN DBpedia Abstracts ✗ ✗ NIF MEANTIME ✓ ✓ CAT VoxEL ✓ ✗ NIF

SLIDE 9

Overview of popular EL datasets

Dataset Mn Typ Format MSNBC ✗ ✗ MSNBC IITB ✓ ✗ IITB AIDA/CoNLL ✓ ✗ AIDA ACE2004 ✗ ✗ MSNBC AQUAINT ✗ ✗ MSNBC DBpedia Spotlight ✓ ✗ NIF KORE50 ✓ ✗ NIF N3-RSS 500 ✓ ✗ NIF Reuters 128 ✓ ✗ NIF News-100 ✓ ✗ NIF Wes2015 ✓ ✗ NIF SemEval 2015 Task 13 ✓ ✗ SemEval Thibaudet ✗ ✓ RENDEN Bergson ✗ ✓ RENDEN DBpedia Abstracts ✗ ✗ NIF MEANTIME ✓ ✓ CAT VoxEL ✓ ✗ NIF

SLIDE 10

Proposal

NIFify: a tool that simultaneously supports the creation, visualization, and validation of NIF datasets, as well as the comparison of EL systems.

SLIDE 11

Related Work

NIF-Dataset creation

BENGAL Automatic NIF Creation

Demo Source Code QRTool

SLIDE 12

Related Work

NIF-Dataset creation

BENGAL Automatic NIF Creation

Demo Source Code QRTool

NIF-Dataset validation

Eaglet NIF-Dataset Validation Demo Source Code

SLIDE 13

Related Work

NIF-Dataset creation

BENGAL Automatic NIF Creation

Demo Source Code QRTool

NIF-Dataset validation

Eaglet NIF-Dataset Validation Demo Source Code

Benchmarking

NIF-Dataset Creation Orbis Benchmark Demo Source Code Visualization GERBIL Benchmark Demo Source Code Visualization

SLIDE 14

Related Work

NIF-Dataset creation

BENGAL Automatic NIF Creation

Demo Source Code QRTool

NIF-Dataset validation

Eaglet NIF-Dataset Validation Demo Source Code

Benchmarking

NIF-Dataset Creation Orbis Benchmark Demo Source Code Visualization GERBIL Benchmark Demo Source Code Visualization

NIF-Dat

SLIDE 15

NIFify - Creation

SLIDE 16

NIFify - Creation

SLIDE 17

NIFify - Creation

SLIDE 18

NIFify - Creation

SLIDE 19

NIFify - Validation

SLIDE 20

NIFify - Validation

SLIDE 21

Errors found in current NIF datasets

Dataset Spelling Error Link Error Format Error DBpedia Spotlight 8 23 4 N3-RSS 500 1 34 – Reuters 128 4 71 – News-100 9 1515 – Wes2015 – 609 – VoxEL – 8 –

https://users.dcc.uchile.cl/~hrosales/dataset_errors.html

SLIDE 22

Errors found in current NIF datasets

SLIDE 23

NIFify - Benchmarking

SLIDE 24

NIFify - Benchmarking

SLIDE 25

Conclusion

NIFify: Creation/Validation/Visualization/Benchmark
Demo:

https://users.dcc.uchile.cl/~hrosales/NIFify_v2.html

Source Code:

https://github.com/henryrosalesmendez/NIFify_v2

SLIDE 26

NIFify: Towards Better Quality Entity Linking Datasets †

Henry Rosales-M´ endez, Aidan Hogan and Barbara Poblete

University of Chile {hrosales,ahogan,bpoblete}@dcc.uchile.cl

May 14, 2019

† LA-WEB 2019 - 10th Latin American Web Congress