Tulip: Lightweight Entity Recognition and Disambiguation Using - - PowerPoint PPT Presentation
Tulip: Lightweight Entity Recognition and Disambiguation Using Wikipedia-Based Topic Centroids Marek Lipczak Arash Koushkestani Evangelos Milios Problem definition The goal of Entity Recognition and Disambiguation (ERD) Identify
2
The goal of Entity Recognition and Disambiguation (ERD)
□ Identify mentions of entities □ Link the mentions to a relevant entry in an external knowledge base □ The knowledge base is typically a large subset of Wikipedia articles
Example:
3
Recognition
□ Is this a valid mention of an entity present in the knowledge base?
Disambiguation
□ Which of the potential entities (senses) is correct?
4
Recognition
□ Is this a valid mention of an entity present in the knowledge base?
Disambiguation
□ Which of the potential entities (senses) is correct?
Default sense – the entity with a largest number of wiki-links with the
□ Tulip focuses on default sense entities □ Main goal is to recognize whether the default sense is consistent with
5
Visual Text Analytics Lab
□ Some experience with using ERD systems □ No experience implementing ERD systems
Key issue with state-of-the-art systems: obvious false positive mistakes
□ Visualize Prof. Smith's research interests:
Data Mining Machine Learning 50 cent
Our goal: minimize the number of false positives
6
Spotter
□ Find all mentions of entities in the text (Solr Text Tagger) □ Special handling for personal names
Recognizer
□ Retrieve profjles of spotted entities (from Sunfmower) □ Generate a topic centroid representing the document □ Select entities consistent with the document
7
Spotter
□ Find all mentions of entities in the text (Solr Text Tagger) □ Special handling for personal names
Recognizer
□ Retrieve profjles of spotted entities (from Sunfmower) □ Generate a topic centroid representing the document □ Select entities consistent with the document
8
Solr (Lucene) is a text search engine
□ Indexes textual documents □ Retrieve documents for keyword-based queries
Solr Text Tagger
□ Indexes entity surface forms stored in a lexicon
E.g., Baltimore Ravens, Ravens, Baltimore (…)
□ Uses full text documents as queries □ Finds all entity mentions in the document □ Retrieves the mentioned entities (candidate selection) □ Implemented based on Solr's Finite State Transducers
By David Smiley and Rupert Westenthaler (thanks!)
9
Three sources of entity surface forms (external datasets)
□ Entity names (from Freebase) □ Wiki-links anchor text (from Wikipedia) □ Web anchor text (from Google's Wikilinks corpus)
10
Three sources of entity surface forms (external datasets)
□ Entity names (from Freebase) □ Wiki-links anchor text (from Wikipedia) □ Web anchor text (from Google's Wikilinks corpus)
Special handling of personal names
□ “Jack” and “London” are not allowed as surface forms for Jack London □ Instead they are indexed as “generic” personal names and will be
11
Three sources of entity surface forms (external datasets)
□ Entity names (from Freebase) □ Wiki-links anchor text (from Wikipedia) □ Web anchor text (from Google's Wikilinks corpus)
Special handling of personal names
□ “Jack” and “London” are not allowed as surface forms for Jack London □ Instead they are indexed as “generic” personal names and will be
Flagging suspicious surface forms (e.g., “It” - Stephen King's novel)
□ stop-word fjlter marks all stop-words or phrases composed of stop-
□ Wiktionary fjlter marks all common nouns, verbs, adjectives, etc.
□ lower-case fjlter marks all lower-case words or phrases
12
Default sense for all mentions (Freebase only)
13
Default sense for all mentions (Freebase only) Default sense for all mentions (Freebase + Wikpedia)
14
Default sense for all mentions (Freebase only) Default sense for all mentions (Freebase + Wikpedia) Suspicious mentions removed
15
Default sense for all mentions (Freebase only) Default sense for all mentions (Freebase + Wikpedia) Suspicious mentions removed How can we remove Michael Kors and bring back Home Depot?
□ Relatedness of entities to the document
16
Spotter
□ Find all mentions of entities in the text (Solr Text Tagger) □ Special handling for personal names
Recognizer
□ Retrieve profjles of spotted entities (from Sunfmower) □ Generate a topic centroid representing the document □ Select entities consistent with the document
17
Our solution
□ Retrieve a profjle of every entity mentioned in the text □ Agglomerate the profjles in a centroid representing the document □ Check which entities are coherent with the topics (relatedness score)
18
Our solution
□ Retrieve a profjle of every entity mentioned in the text □ Agglomerate the profjles in a centroid representing the document □ Check which entities are coherent with the topics (relatedness score) □ How do we create the entity profjles?
19
A concept graph based on unifjed category graph from 120 Wikipedia
□ Each language version acts like a witness for the importance of stored
Compact and accurate category profjles for all Wikipedia articles
□ Removal of unimportant categories □ Inference of more general categories
20
Sunfmower graph is:
□ Directed □ Weighted (importance score) □ Sparse (only k most important
Category-based profjle is
□ All categories at distance < d □ Term weights based on edge weights □ E.g., k = 3, d = 2 □ Path weight is the product of edge weights
w(Intel → Comp. of US → Ec. of US) = 0.42*0.27 = 0.11
□ Category weight is the sum of path weights
w(Ec. of US) = 0.11 + 0.19 = 0.3
21
Retrieve category-based profjles for all default senses (example next slide)
22
23
Retrieve category-based profjles for all default senses (example next slide) Topic Centroid Generation
□ Centroid is a linear combination of entity profjles □ Default senses of non-suspicious mentions only
24
Retrieve category-based profjles for all default senses (example next slide) Topic Centroid Generation
□ Centroid is a linear combination of entity profjles □ Default senses of non-suspicious mentions only
Topic Centroid Refjnement
□ Entities far from the centroid are removed from the core □ Cosine similarity with predefjned threshold tcoh=0.2
25
Retrieve category-based profjles for all default senses (example next slide) Topic Centroid Generation
□ Centroid is a linear combination of entity profjles □ Default senses of non-suspicious mentions only
Topic Centroid Refjnement
□ Entities far from the centroid are removed from the core □ Cosine similarity with predefjned threshold tcoh=0.2
Entity Scoring
□ Relatedness score assigned to each default sense entity
26
Retrieve category-based profjles for all default senses (example next slide) Topic Centroid Generation
□ Centroid is a linear combination of entity profjles □ Default senses of non-suspicious mentions only
Topic Centroid Refjnement
□ Entities far from the centroid are removed from the core □ Cosine similarity with predefjned threshold tcoh=0.2
Entity Scoring
□ Relatedness score assigned to each default sense entity
System output
□ Entities with score > tcoh □ Entity with best relatedness score for each mention
27
Tulip got second place in the long track
□ Category-based topic centroids – promising solution for relatedness □ Top recall among all submitted systems (?!) □ Lowest latency among all submitted systems
28
Entity Recognition and Disambiguation is typically just a single step in a
To be practical the ERD system has to be lightweight:
□ Fast – lowest latency among all competing systems, over 200
□ Adaptable – both Solr Text Tagger and Sunfmower can be easily
□ Compact – the full system requires less than 4 GB of operational
29
Entity Recognition and Disambiguation is typically just a single step in a
To be practical the ERD system has to be lightweight:
□ Fast – lowest latency among all competing systems, throughput of
□ Adaptable – both Solr Text Tagger and Sunfmower can be easily
□ Compact – the full system requires less than 4 GB of operational
30
Analysis on 50 documents with ground-truth data (1166 entities) 85% of mentions that can be disambiguated, should be disambiguated
□ Another 5% is explicitly disambiguated with another mention in the
Focusing on default sense Tulip missed < 5% of entities
31
Analysis on 50 documents with ground-truth data (1166 entities) 85% of mentions that can be disambiguated, should be disambiguated
□ Another 5% is explicitly disambiguated with another mention in the
Focusing on default sense Tulip missed < 5% of entities
32
Analysis on 50 documents with ground-truth data (1166 entities) 85% of mentions that can be disambiguated, should be disambiguated
□ Another 5% is explicitly disambiguated with another mention in the
Focusing on default sense Tulip missed < 5% of entities
33
Wikipedia-based category profjles can be used to determine the
Small size of category profjles allows the system to represent the
The pruning of suspicious mentions and focus on the default sense
The accuracy of extracted entities relies more on the successful
35
Two level Finite State Transducers approach
□ Word to index (each edge is a letter) □ Surface form to list of entities (each edge is a word)