A Time Machine for Text Search Klaus Berberich, Srikanta Bedathur, - PowerPoint PPT Presentation

A Time Machine for Text Search Klaus Berberich, Srikanta Bedathur, Thomas Neumann, Gerhard Weikum Max-Planck Institute for Informatics, Saarbrücken, Germany

Motivation � Historical information needs, e.g., Contemporary (~2001) articles about the movie “Harry Potter and the Sorcerer’s Stone” � Relevant pages have disappeared but are preserved by Web archives (e.g., archive.org) � Search over Web archives is limited and ignores the time-axis Klaus Berberich – A Time Machine for Text Search

Motivation � Time-Travel Text Search extends keyword querying by a time-point of interest t “harry potter” @ 2001/11/14 � Other temporally versioned text collections � Wikis � Repositories (e.g., controlled by CVS, Subversion) � Your Desktop Klaus Berberich – A Time Machine for Text Search

Outline � Motivation � Collection, Query, and Relevance Model � Time-Travel Inverted File Index � Reducing Index Size � Tuning Index Performance � Experimental Evaluation � Conclusions Klaus Berberich – A Time Machine for Text Search

Collection Model � Document d is a sequence of time-stamped versions � Version is a vector of searchable terms � Document deletion results in tombstone version ⊥ � Discrete time, timestamps are non-negative � State of document collection as of time t Klaus Berberich – A Time Machine for Text Search

Query Model � Time-travel query q t consists of � keyword part q (i.e., a set of query terms) � time-point of interest t � Time-travel query q t is evaluated over D t so that only versions that existed at time t are considered Klaus Berberich – A Time Machine for Text Search

Relevance Model � We adapt Okapi BM25 as a relevance model � Term-frequency score (TF) � Inverse document-frequency score (IDF) Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index � Idea: Transparently extend “IR’s workhorse” so that the existing wealth of extensions remains applicable � We extend postings by a validity time-interval IDF TF B + -Tree B + -Tree “harry” “potter” ( d1, 11.2, [t1, t2) ) “harry” ( d8, 10.9, [t7, t9) ) ( d1, 10.6, [t2, t5) ) Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index � Time-travel query q t can be processed by scanning index lists while ignoring non-relevant postings � Example : “harry”@t8 IDF TF B + -Tree B + -Tree “harry” “potter” ( d1, 11.2, [t1, t2) ) “harry” ( d8, 10.9, [t7, t9) ) ( d1, 10.6, [t2, t5) ) Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index � Time-travel query q t can be processed by scanning index lists while ignoring non-relevant postings � Example : “harry”@t8 IDF TF B + -Tree B + -Tree “harry” “potter” ( d1, 11.2, [t1, t2) ) “harry” ( d8, 10.9, [t7, t9) ) ( d1, 10.6, [t2, t5) ) w idf (“harry”, t8) = 3.08 Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index � Time-travel query q t can be processed by scanning index lists while ignoring non-relevant postings � Example : “harry”@t8 IDF TF B + -Tree B + -Tree “harry” “potter” ( d1, 11.2, [t1, t2) ) “harry” scan ( d8, 10.9, [t7, t9) ) ( d1, 10.6, [t2, t5) ) w idf (“harry”, t8) = 3.08 Klaus Berberich – A Time Machine for Text Search

Reducing Index Size � Shortcoming: Since we create one posting per version per term, the resulting index is very large TF B + -Tree “harry” ( d1, 11.2, [t1, t2) ) HUGE!!! ( d8, 10.9, [t7, t9) ) ( d1, 10.6, [t2, t5) ) (Wikipedia Revision History ~8.6B postings) Klaus Berberich – A Time Machine for Text Search

A Time Machine for Text Search Klaus Berberich, Srikanta Bedathur, - PowerPoint PPT Presentation

A Time Machine for Text Search Klaus Berberich, Srikanta Bedathur, Thomas Neumann, Gerhard Weikum Max-Planck Institute for Informatics, Saarbrcken, Germany Motivation Historical information needs, e.g., Contemporary (~2001) articles

11 Text search Robert Elssser Robert Elssser Text search Different scenarios: Dynamic

Algorithms theory 15 Text search (1) Prof. Dr. S. Albers Winter term 07/08 Text search

Elastic Search - Aditi Choksi (EW18455) Elastic Search Search engine Distributed

Theory I Algorithm Design and Analysis (10 - Text search, part 1) Prof. Dr. Th. Ottmann 1 Text

Unsupervised Machine Translation Sachin Kumar Conditional Text Generation Generate text

What is the best full text search engine for Python? Andrii Soldatenko @a_soldatenko Agenda:

To be or not to be. Neo4j Full Text Search Tips and Tricks Christophe Willemsen CTO -

Text Search and Similarity Search PG 12.112.2, F.31 Dr. Chris Mayfield Department of Computer

Text Mining in Search Engines By: DJ Ambler With special thanks to the Internet Overview

1 Aspects of Search Quality System Aspects of Evaluation Relevancy Response time:

Discovery Search Searches most of the our subscriptions and open access platforms from a single

Search-Based Unsupervised Text Generation Lili Mou Dept. Computing Science, University of Alberta

Web Data Representation Web Graph, Text, Images, Metadata, Search spaces Web Search 1 The Web

Full-Text Search Explained Philipp Krenn @xeraa Infrastructure | Developer Advocate

Semantic Full-Text Search Semantic Full Text Search Talk @ SIGIR JIWES Talk @ SIGIR

Searching Binary search (log n ) Better Can we achieve (1) search time? Chapter 9

WNY RPC Board of Directors Meeting September 12, 2018 Jamestown Community College Carnahan

Roaring 20s Timeline 1920-1929 Prohibition; 1920 Speakeasies were created to sell and serve

Garden City Public Schools K-12 School Librarians: School Librarians: Impacting Student

Administrative Leadership Meeting Randy Woodson Chancellor Tuesday, March 12, 2019 Upcoming

DASNY: Building New York State Dormitory Authority - State of New York Stephen D. Curro, P.E.

Westchester County Existing Conditions and Outlook Mass Transit Task Force February 22, 2013

Commissioner Update to SBOE JUNE 12, 2019 1 86 th Legislative Session Recap 2 Legislative

GOOGL E WHAT S NE XT ? Go o g le E duc a tio n T o o ls g .c o / e duc hro me b o o