dimensionality reduction for information retrieval using
play

Dimensionality Reduction for Information Retrieval using Vector - PowerPoint PPT Presentation

Nov 26, 2023 •154 likes •519 views

Introduction Rare Term Vector Replacement Evaluation Summary & Conclusions Dimensionality Reduction for Information Retrieval using Vector Replacement of Rare Terms Tobias Berka, Marian Vajter sic April 30, 2011 Tobias Berka, Marian

  1. Introduction Rare Term Vector Replacement Evaluation Summary & Conclusions Dimensionality Reduction for Information Retrieval using Vector Replacement of Rare Terms Tobias Berka, Marian Vajterˇ sic April 30, 2011 Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  2. Introduction Rare Term Vector Replacement Evaluation Summary & Conclusions Outline 1 Introduction Dimensionality Reduction 2 Rare Term Vector Replacement Zipf’s Law Replacement Vectors Rare Term Replacement 3 Evaluation Retrieval Performance Computational Performance Stability 4 Summary & Conclusions Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  3. Introduction Rare Term Vector Replacement Dimensionality Reduction Evaluation Summary & Conclusions Introduction Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  4. Introduction Rare Term Vector Replacement Dimensionality Reduction Evaluation Summary & Conclusions Goals Reduce dimensionality, Preserve or improve... Pair-wise distances, Cross-class scatter, Retrieval / clustering / classification performance. Detect... Contributing factors, Individual components, Signals or noise. Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  5. Introduction Rare Term Vector Replacement Dimensionality Reduction Evaluation Summary & Conclusions Methods Great Classics: Linear Methods, Singular Value Decomposition, Principal Component Analysis (PCA), Non-negative Matrix Factorization(s), Independent Component Analysis. Canonical Extension: Kernel Methods, Maps: Mesh Fitting, Self-Organization, Manifold Learning: Local Linearization, Local Non-linear Reduction. Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  6. Introduction Rare Term Vector Replacement Dimensionality Reduction Evaluation Summary & Conclusions My Interest Better retrieval, More complete retrieval. Dynamic searching, Less reliance on static indices, Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  7. Introduction Rare Term Vector Replacement Dimensionality Reduction Evaluation Summary & Conclusions My Interest Interactive semi-supervised clustering, Exploratory data analysis, Search. Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  8. Introduction Rare Term Vector Replacement Dimensionality Reduction Evaluation Summary & Conclusions My Interest Sparse ❀ dense. Good for super-scalar CPUs, More efficient parallelism. Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  9. Introduction Zipf’s Law Rare Term Vector Replacement Replacement Vectors Evaluation Rare Term Replacement Summary & Conclusions Rare Term Vector Replacement Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  10. Introduction Zipf’s Law Rare Term Vector Replacement Replacement Vectors Evaluation Rare Term Replacement Summary & Conclusions Zipf’s Law “The [document] frequency of a word is reciprocally proportional to its frequency rank.” 1 f i ∝ rank ( f i ) . Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  11. Introduction Zipf’s Law Rare Term Vector Replacement Replacement Vectors Evaluation Rare Term Replacement Summary & Conclusions Zipf’s Law in Practice “Most words occur only in a very small number of documents.” Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  12. Introduction Zipf’s Law Rare Term Vector Replacement Replacement Vectors Evaluation Rare Term Replacement Summary & Conclusions Zipf’s Law in Pictures 10000 occurrences Q1=1 Q2=2 Q3=7 mean=75.93 1000 cut-off=694 occurrences 100 10 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 feature (relative) Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  13. Introduction Zipf’s Law Rare Term Vector Replacement Replacement Vectors Evaluation Rare Term Replacement Summary & Conclusions Zipf’s Law in Pictures occurrences Q1=1 Q2=1 Q3=4 100 mean=6.52 cut-off=10 occurrences 10 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 feature (relative) Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  14. Introduction Zipf’s Law Rare Term Vector Replacement Replacement Vectors Evaluation Rare Term Replacement Summary & Conclusions Zipf’s Law vs. Dimensionality Reduction Eliminate rare terms? High importance for information retrieval! Can we compress them? Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  15. Introduction Zipf’s Law Rare Term Vector Replacement Replacement Vectors Evaluation Rare Term Replacement Summary & Conclusions Replacement Vectors Let’s compute replacement vectors! Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  16. Introduction Zipf’s Law Rare Term Vector Replacement Replacement Vectors Evaluation Rare Term Replacement Summary & Conclusions Centroid Summarization We operate on a corpus in vector form. Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  17. Introduction Zipf’s Law Rare Term Vector Replacement Replacement Vectors Evaluation Rare Term Replacement Summary & Conclusions Centroid Summarization Select the vectors containing a rare term. Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  18. Introduction Zipf’s Law Rare Term Vector Replacement Replacement Vectors Evaluation Rare Term Replacement Summary & Conclusions Centroid Summarization Compute the centroid. Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  19. Introduction Zipf’s Law Rare Term Vector Replacement Replacement Vectors Evaluation Rare Term Replacement Summary & Conclusions Vector Truncation Discard the rare features. occurrences Q1=1 Q2=1 Q3=4 100 mean=6.52 cut-off=10 occurrences 10 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 feature (relative) Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  20. Introduction Zipf’s Law Rare Term Vector Replacement Replacement Vectors Evaluation Rare Term Replacement Summary & Conclusions Computing Replacement Vectors For all rare terms, we compute the following: Select all documents containing the rare term, Compute the (weighted) average vector, Truncate all rare terms from this vector. Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  21. Introduction Zipf’s Law Rare Term Vector Replacement Replacement Vectors Evaluation Rare Term Replacement Summary & Conclusions A More Efficient Algorithm For all documents, For all rare terms, Add the common terms to the average vector. Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  22. Introduction Zipf’s Law Rare Term Vector Replacement Replacement Vectors Evaluation Rare Term Replacement Summary & Conclusions New Document Representation For all documents, we compute the following: Truncate all rare terms from the document vectors (i.e. retain only common terms), Add the linear combination of all replacement vectors, For all rare terms in the document, Scaled by the weighted term frequency, Normalize the result to unit length. Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  23. Introduction Zipf’s Law Rare Term Vector Replacement Replacement Vectors Evaluation Rare Term Replacement Summary & Conclusions Subsequent Rank Reduction Once we have computed the replacement vectors, we compute a rank-reduced PCA, Reduces number of features by 50%, improves the retrieval performance, Low number of features, dense data matrix – use a symmetric eigensolver, In LAPACK terms: xSPEVX , xSYEV , etc. Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  24. Introduction Retrieval Performance Rare Term Vector Replacement Computational Performance Evaluation Stability Summary & Conclusions Evaluation Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

  25. Introduction Retrieval Performance Rare Term Vector Replacement Computational Performance Evaluation Stability Summary & Conclusions Reuters Corpus Reuters corpus, training set, all categories. 0.8 sparse TD-IDF (47,236) vector replacement (535) rank-reduced vector replacement (392) 0.75 0.7 mean precision 0.65 0.6 0.55 0.5 0 10 20 30 40 50 60 70 80 90 100 hit list rank Tobias Berka, Marian Vajterˇ sic Dimensionality Reduction for Information Retrieval using Vecto

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

STAT 209 Dimensionality Reduction November 26, 2019 Colin Reimer Dawson 1 / 24 Dimensionality

STAT 209 Dimensionality Reduction November 26, 2019 Colin Reimer Dawson 1 / 24 Dimensionality

Dimensionality Reduction STAT 209 Dimensionality Reduction November 26, 2019 Colin Reimer Dawson 1 / 24 Dimensionality Reduction Outline Dimensionality Reduction 2 / 24 Dimensionality Reduction High Dimensional Data Modern datasets

470 views • 24 slides
Search in High-Dimensional spaces and Dimensionality Reduction i i li d i D. Gunopulos 1

Search in High-Dimensional spaces and Dimensionality Reduction i i li d i D. Gunopulos 1

Search in High-Dimensional spaces and Dimensionality Reduction i i li d i D. Gunopulos 1 Retrieval techniques for high- dimensional datasets The retrieval problem: The retrieval problem: Given a set of objects S , and a query

543 views • 25 slides
Introduction to Information Retrieval http://informationretrieval.org IIR 18: Latent Semantic

Introduction to Information Retrieval http://informationretrieval.org IIR 18: Latent Semantic

Latent semantic indexing Dimensionality reduction LSI in information retrieval Introduction to Information Retrieval http://informationretrieval.org IIR 18: Latent Semantic Indexing Hinrich Sch utze Institute for Natural Language

1.42k views • 75 slides
Dimensionality Reduction Algorithms (and how to interpret their output) Dalya Baron (Tel Aviv

Dimensionality Reduction Algorithms (and how to interpret their output) Dalya Baron (Tel Aviv

Dimensionality Reduction Algorithms (and how to interpret their output) Dalya Baron (Tel Aviv University) XXX Winter School, November 2018 What is Dimensionality Reduction? Dimensionality Reduction algorithm 28 x 28 features per object 2

437 views • 33 slides
Dimensionality Reduction Alexandros Tantos Assistant Professor Aristotle University of

Dimensionality Reduction Alexandros Tantos Assistant Professor Aristotle University of

DataCamp Dimensionality Reduction in R DIMENSIONALITY REDUCTION IN R Dimensionality Reduction Alexandros Tantos Assistant Professor Aristotle University of Thessaloniki DataCamp Dimensionality Reduction in R Curse of Dimensionality

550 views • 35 slides
6.1 Dimensionality reduction Previously in the course, we have discussed algorithms suited for a

6.1 Dimensionality reduction Previously in the course, we have discussed algorithms suited for a

CS/CNS/EE 253: Advanced Topics in Machine Learning Topic: Dimensionality Reduction Lecturer: Andreas Krause Scribe: Matt Faulkner Date: 25 January 2010 6.1 Dimensionality reduction Previously in the course, we have discussed algorithms suited for

442 views • 6 slides
DIMENSIONALITY REDUCTION DIMENSIONALITY REDUCTION MATTHIEU BLOCH April 21, 2020 1 / 26

DIMENSIONALITY REDUCTION DIMENSIONALITY REDUCTION MATTHIEU BLOCH April 21, 2020 1 / 26

DIMENSIONALITY REDUCTION DIMENSIONALITY REDUCTION MATTHIEU BLOCH April 21, 2020 1 / 26 MULTIDIMENSIONAL SCALING MULTIDIMENSIONAL SCALING There are situations for which Euclidean distance is not appropriate Suppose we have access to a

500 views • 26 slides
NPFL103: Information Retrieval (11) Latent semantic indexing Pavel Pecina Institute of Formal

NPFL103: Information Retrieval (11) Latent semantic indexing Pavel Pecina Institute of Formal

Latent semantic indexing Dimensionality reduction LSI in information retrieval LSI as sofu clustering NPFL103: Information Retrieval (11) Latent semantic indexing Pavel Pecina Institute of Formal and Applied Linguistics Faculty of

935 views • 30 slides
Applied Machine Learning Dimensionality reduction using PCA Siamak Ravanbakhsh COMP 551 (Fall

Applied Machine Learning Dimensionality reduction using PCA Siamak Ravanbakhsh COMP 551 (Fall

Applied Machine Learning Dimensionality reduction using PCA Siamak Ravanbakhsh COMP 551 (Fall 2020) Learning objectives What is dimensionality reduction? What is it good for? Linear dimensionality reduction: Principal Component Analysis

661 views • 26 slides
T-61.3050 Machine Learning: Basic Principles Dimensionality Reduction Kai Puolam aki

T-61.3050 Machine Learning: Basic Principles Dimensionality Reduction Kai Puolam aki

Multivariate Methods Dimensionality Reduction T-61.3050 Machine Learning: Basic Principles Dimensionality Reduction Kai Puolam aki Laboratory of Computer and Information Science (CIS) Department of Computer Science and Engineering Helsinki

459 views • 30 slides
Nonlinear Dimensionality Reduction Donovan Parks Overview Direct visualization vs.

Nonlinear Dimensionality Reduction Donovan Parks Overview Direct visualization vs.

Nonlinear Dimensionality Reduction Donovan Parks Overview Direct visualization vs. dimensionality reduction Nonlinear dimensionality reduction techniques: ISOMAP, LLE, Charting A fun example that uses non- metric, replicated

882 views • 38 slides
Introduction to Information Retrieval http://informationretrieval.org IIR 18: Latent Semantic

Introduction to Information Retrieval http://informationretrieval.org IIR 18: Latent Semantic

Singular Value Decomposition Dimensionality reduction Latent Semantic Indexing Introduction to Information Retrieval http://informationretrieval.org IIR 18: Latent Semantic Indexing Hinrich Sch utze Institute for Natural Language

1.27k views • 106 slides
Preprocessing and Dimensionality Reduction J er emy Fix CentraleSup elec

Preprocessing and Dimensionality Reduction J er emy Fix CentraleSup elec

Datasets Preprocessing Dimensionality reduction Preprocessing and Dimensionality Reduction J er emy Fix CentraleSup elec jeremy.fix@centralesupelec.fr 2017 1 / 73 Datasets Preprocessing Dimensionality reduction Where to get data

1.22k views • 73 slides
Lecture 7 Jan-Willem van de Meent DIMENSIONALITY REDUCTION Borrowing from : Percy Liang

Lecture 7 Jan-Willem van de Meent DIMENSIONALITY REDUCTION Borrowing from : Percy Liang

Unsupervised Machine Learning and Data Mining DS 5230 / DS 4420 - Fall 2018 Lecture 7 Jan-Willem van de Meent DIMENSIONALITY REDUCTION Borrowing from : Percy Liang (Stanford) Dimensionality Reduction Goal: Map high dimensional

1.17k views • 70 slides
Clustering and Dimensionality Reduction Preview Clustering K -means clustering

Clustering and Dimensionality Reduction Preview Clustering K -means clustering

Clustering and Dimensionality Reduction Preview Clustering K -means clustering Mixture models Hierarchical clustering Dimensionality reduction Principal component analysis Multidimensional scaling Isomap

1.61k views • 17 slides
Spectral Dimensionality Reduction via Learning Eigenfunctions Yoshua Bengio Thanks to Pascal

Spectral Dimensionality Reduction via Learning Eigenfunctions Yoshua Bengio Thanks to Pascal

Spectral Dimensionality Reduction via Learning Eigenfunctions Yoshua Bengio Thanks to Pascal Vincent, Jean-Franois Paiement, Olivier Delalleau, Marie Ouimet, and Nicolas Le Roux. Dimensionality Reduction For many distributions, it is

408 views • 22 slides
East Division Elementary School GC/CM March 7 th , 2018 Board Update Agenda for this evening

East Division Elementary School GC/CM March 7 th , 2018 Board Update Agenda for this evening

East Division Elementary School GC/CM March 7 th , 2018 Board Update Agenda for this evening Madison Elementary School Design Update Gerry Pless Design Progress Cost Estimating and Value Engineering Permitting Land

429 views • 17 slides
Brunso n Re pla c e me nt Sc ho o l Site Optio ns Aug ust 26-27, 2020 1 We lc ome & Ope

Brunso n Re pla c e me nt Sc ho o l Site Optio ns Aug ust 26-27, 2020 1 We lc ome & Ope

Brunso n Re pla c e me nt Sc ho o l Site Optio ns Aug ust 26-27, 2020 1 We lc ome & Ope ning Dr . Ange la P. Hair ston, Supe rinte nde nt Winsto n-Sa le m / F o rsyth Co unty Sc ho o ls 2 Intr oduc tions WS/ F CS Pla nning a

309 views • 27 slides
CENCOSUD CORPORATE PRESENTATION Second Quarter 2020 About Cencosud Well Established Leading

CENCOSUD CORPORATE PRESENTATION Second Quarter 2020 About Cencosud Well Established Leading

CENCOSUD CORPORATE PRESENTATION Second Quarter 2020 About Cencosud Well Established Leading +6.2 MM Square Meters of Retailer in South America landbank fuel future grouth across the region +50 years of history +120K employees 1,107 Retail

312 views • 14 slides
ntegrated eospatial nnovations IGI mbH - 40 years celebration Founded 1978 More than 500

ntegrated eospatial nnovations IGI mbH - 40 years celebration Founded 1978 More than 500

ntegrated eospatial nnovations IGI mbH - 40 years celebration Founded 1978 More than 500 customers in 84 different countries World-wide technical support IGI office in Kreuztal, Germany 3 Open Skies Area of Operation IGI Sensor

798 views • 36 slides
CDC PUBLIC HEALTH GRAND ROUNDS Autism Spectrum Disorder: From Numbers to Know-How April pril

CDC PUBLIC HEALTH GRAND ROUNDS Autism Spectrum Disorder: From Numbers to Know-How April pril

CDC PUBLIC HEALTH GRAND ROUNDS Autism Spectrum Disorder: From Numbers to Know-How April pril 22, 2014 22, 2014 Autism Etiology: What We Know and How to Learn More Julie Daniels, MPH, PhD Associate Professor, Epidemiology & Maternal and

1k views • 63 slides
L OS A NGELES B ASIN S TORMWATER C ONSERVATION S TUDY Los Angeles County Flood Control District

L OS A NGELES B ASIN S TORMWATER C ONSERVATION S TUDY Los Angeles County Flood Control District

L OS A NGELES B ASIN S TORMWATER C ONSERVATION S TUDY Los Angeles County Flood Control District U.S. Department of the Interior Bureau of Reclamation TASK 5 Infrastructure & Operations Concepts Public Stormwater Capture Concept

690 views • 35 slides
promises and pitfalls Lina Gega, Paul Strickland, Owen Barry, Peter Langdon & Leen

promises and pitfalls Lina Gega, Paul Strickland, Owen Barry, Peter Langdon & Leen

Technology-mediated interventions to improve psychological and social skills: promises and pitfalls Lina Gega, Paul Strickland, Owen Barry, Peter Langdon & Leen Vereenooghe Background Two types of technology: Virtual Environments with

744 views • 27 slides
Reducing Replication Bandwidth for Distributed Document Databases Lianghong Xu 1 , Andy Pavlo 1 ,

Reducing Replication Bandwidth for Distributed Document Databases Lianghong Xu 1 , Andy Pavlo 1 ,

Reducing Replication Bandwidth for Distributed Document Databases Lianghong Xu 1 , Andy Pavlo 1 , Sudipta Sengupta 2 Jin Li 2 , Greg Ganger 1 Carnegie Mellon University 1 , Microsoft Research 2 Document-oriented Databases { { " _id " :

488 views • 24 slides

More recommend