Merging Results from Multiple Sources in Video Retrieval Wei-Hao - - PowerPoint PPT Presentation

Merging Results from Multiple Sources in Video Retrieval

Wei-Hao Lin

Language Technologies Institute School of Computer Science Carnegie Mellon University

whlin@cs.cmu.edu

2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Multiple-Source Video Retrieval

Query: Find video segments containing “Madeleine Albright” CNN ABC C-SPAN

2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Motivation

• We want to exploit source-dependent characteristics to

improve video retrieval performance.

• However, Source Merging isn’t a trivial problem.

0.007 0.014 0.006 Random ?? Merging ABC + CNN 0.625 ABC + CNN 0.780 CNN 0.053 ABC Find video segments that are Weather News MAP Data Task

2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Problem: Merging Results

Merging Method 1 Merging Method 2 Merging Method 3

2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Issues

• Degree of Document Overlapping
• Degree of Cooperation
• Local Relevance Function
• Local-to-Global Transformation Function

2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Degree of Document Overlapping

• When overlapping is

high, we can exploit the cross-resource information.

S1 S2 S3

2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Degree of Cooperation

• When resources are

cooperative, we can exploit information in addition to ranks.

Uncooperative Source Cooperative Source Pr(R|D) Pr(R|D) Pr(R|D) DID DID DID DID DID DID Text Text Text Corpus Statistics

2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Local Relevance Function

• Express the extent to which the

document from a source is relevant to the query

• Rank
• Score, Normalized Score
• Probability that the document is relevant

(conditioned on the rank)

2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Local-to-Global Transformation Function

• Map the relevance

score of a document in one source to a score that reflects its relevance across sources more accurately

L1(di) L2(di) G(L1(di)) G(L2(di)) S1 S2 Merged List

2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Related Works

Degree of Document Overlapping High Low

Meta-Search Distributed IR

Score Rank Without Learning With Learning

Borda Fuse Weighted Borda Fuse, Bayes Fuse CombANZ, CombSUM, CombMNZ Linear Combination Exp + Nor

Full Text Summary Rank Score

Local Search Local Search Cross- Source Similarity Round Robin

Summary

Local Search Sampled Data + Regression CORI

2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Multi-Source Video Retrieval

• Low or not-so-obvious document overlapping
• Approaches developed for meta-search may not be

suitable here.

• The data are fully accessible.
• Score, rank, full “text”, source statistics are available.
• No need to sample and build source description
• Video isn’t just Text
• Judgments from multimodalities must be combined.

2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Planned Evaluation

• TREC 2003 Video track
• ~120 hours of ABC World News Tonight, CNN

Headlines News, ~13 hours of C-SPAN

• Half of them are already labeled and will be

the test bed.

• Tasks
• Feature Extraction (17 features)
• Evaluation Metric
• Mean Average Precision (MAP)

2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Planned Experiments

• Establish random baseline and upper bound
• Full “text” vs. multiple-source retrieval +

merging

• Source merging in individual modality,

including text (closed caption, speech transcript) and image, face

• Local relevance function
• Local-to-global transformation function

2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

References

• Distributed Information Retrieval
• sampled data + regression: Luo Si and Jamie Callan. Using sampled data and regression to merge search engine
• results. In Proceedings of the 26th annual international ACM SIGIR conference on research and development in

information retrieval, Toronto, Canada, July 28 - August 01 2003.

• Use summary (headline): Xiao Mang Shou and Mark Sanderson. Experiments on data fusion using headline information.

In Proceedings of the 25th annual international ACM SIGIR conference on research and development in information retrieval, University of Tampere, Finland, August 11 - 15 2002.

• Cross-rank similarity: Jamie Callan, Fabio Crestani, Henrik Nottelmann, Pietro Pala, and Xiao Mang Shou. Resource

selection and data fusion in multimedia distributed digital libraries. In Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval, Toronto, Canada, July 28 - August 01 2003.

• CORI: J. Callan. Advances in Information Retrieval, chapter Distributed Information Retrieval, pages 127--150. Kluwer

Academic Publishers, 2000.

• Meta-Search
• CombMNZ: Joon Ho Lee. Analyses of multiple evidence combination. In Proceedings of the 20th annual international

ACM SIGIR conference on Research and development in information retrieval, Philadelphia, Pennsylvania, United States, July 27 - 31 1997.

• Use full text for local search: Steve Lawrence and C. Lee Giles. Context and page analysis for improved web search.

Internet Computing, 2(4):38--46, 1998.

• use summary: Theodora Tsikrika and Mounia Lalmas. Merging techniques for performing data fusion on the
• web. In Proceedings of the tenth international conference on Information and knowledge management,

Atlanta, Georgia, USA, October 05 - 10 2001.

• Model two distributions: R. Manmatha, T. Rath, and F. Feng. Modeling score distributions for combining the outputs of

search engines. In Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, New Orleans, Louisiana, United States, 2001.

• Rank and score: Javed A. Aslam and Mark Montague. Models for metasearch. In Proceedings of the 24th

annual international ACM SIGIR conference on Research and development in information retrieval, New Orleans, Louisiana, United States, 2001.

• Meta-search: Cynthia Dwork, Ravi Kumar, Moni Naor, and D. Sivakumar. Rank aggregation methods for the
• web. In Proceedings of the 10th International World Wide Web Conference, Hong Kong, May 01 - 05 2001.