α -divergences Contextual dissimilarity measures Experiments Reranking with Contextual dissimilarity measures from representational Bregman k -means VISAPP 2010 Olivier Schwander Frank Nielsen { schwander,nielsen } @lix.polytechnique.fr ENS Cachan – ´ ´ Ecole Polytechnique – Sony CSL Angers, May 21st 2010 Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Contextual dissimilarity measures Experiments α -divergences Definition and properties Representational Bregman divergences Clustering Contextual dissimilarity measures Definition Retrieval process Reranking with contexts Experiments Experimental setup Results Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Contextual dissimilarity measures Experiments Introduction Content-Based Image Retrieval ◮ Query by example paradigm ◮ Input: a query image ◮ Output: a rank list, with images sorted by similarity Challenges ◮ Size of databases (millions or billions of entries) ◮ Speed, memory problems ◮ Similarity is subjective (Personalization ? User interaction ? Contextual research ?) Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Contextual dissimilarity measures Experiments Motivation: hierarchical contexts Are we searching for ◮ an animal ? ◮ a mammifer ? ◮ a cat ? ◮ a Siamese cat ? ◮ Felix ? Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Contextual dissimilarity measures Experiments Motivation: region contexts Are we searching for ◮ sea ? ◮ forest ? ◮ moutains ? ◮ all the three ? Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Definition and properties Contextual dissimilarity measures Representational Bregman divergences Experiments Clustering α -divergences One parameter ◮ α ∈ R On positive arrays (unnormalized discrete probabilities) � � 1 − α 1+ α 4 1 − α 2 p i + 1+ α � 2 q i − p 2 q 2 1 − α 2 i i if α � = ± 1 � p i log p i q i + q i − p i = KL ( p � q ) D α ( p � q ) = if α = − 1 � p i log q i p i + p i − q i = KL ( q � p ) if α = 1 Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Definition and properties Contextual dissimilarity measures Representational Bregman divergences Experiments Clustering Properties Invariance to reparametrization (Cencov) ◮ Levi-Civita connexion (Riemannian geometry) ◮ α -connexion Information monotonicity ◮ Merging bins gives lower distance between histograms Canonical divergences for constant curvature spaces Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Definition and properties Contextual dissimilarity measures Representational Bregman divergences Experiments Clustering Bregman divergences One parameter ◮ F : R d → R ◮ Convex and differentiable B F ( p � q ) = F ( p ) − F ( q ) − � p − q , ∇ F ( q ) � Generalization of many usual distances ◮ Machine learning ◮ Computer vision ◮ Information geometry Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Definition and properties Contextual dissimilarity measures Representational Bregman divergences Experiments Clustering Some examples of Bregman-divergences Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Definition and properties Contextual dissimilarity measures Representational Bregman divergences Experiments Clustering Representational Bregman divergence Separable Bregman divergence ◮ B F ( p � q ) = � d i =0 B F ( p i � q i ) Representation function k ◮ Continuous, monotonously increasing ◮ Possibly non-linear ◮ Change of the coordinate system x i = k ( s i ) Induced generator ◮ U ( x ) = � d i =1 U ( x i ) = � d i =1 U ( k ( s i )) = F ( s ) Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Definition and properties Contextual dissimilarity measures Representational Bregman divergences Experiments Clustering Representational α -divergences 2 2 � 1 − α � 1 − α U α ( x ) = x 1 + α 2 2 1 − α k α ( x ) = 2 − α x 2 Bregman divergence ◮ B U , k = B U ◦ k = B F Warnings ◮ Not strictly convex (only U is) ◮ Not symmetrical Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Definition and properties Contextual dissimilarity measures Representational Bregman divergences Experiments Clustering α -centroids Definition n � c R = arg min B U , k ( p i � c ) c ∈X i =1 n � c L = arg min B U , k ( c � p i ) c ∈X i =1 Closed-form formulas 2 �� � 1 − α 1 − α 2 c R n − = 2 p 1 − α i 2 �� � 1+ α 1+ α 2 n − c L = p 2 1+ α i Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Definition and properties Contextual dissimilarity measures Representational Bregman divergences Experiments Clustering Clustering with α -divergences k -means, classical Lloyd algorithm ◮ rely on Bregman k -means, Banerjee 2005 ◮ Bregman k -means on k representation (not the same k ) Assignment step ◮ with α -divergences Relocation step ◮ with α -centroids Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Definition Contextual dissimilarity measures Retrieval process Experiments Reranking with contexts Contextual dissimilarity measures (Perronnin 2009) Φ f ( ω ; q , p , u ) = f ( q , ω p + (1 − ω ) u ) cs f ( q , p | u ) = arg min 0 ≤ ω ≤ 1 Φ g ( ω ; q , p , u ) p q ⊥ Approximate q with a q ⋆ mixture of q and u u q Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Definition Contextual dissimilarity measures Retrieval process Experiments Reranking with contexts Building contexts Hierarchical contexts (Perronnin 2009) Rank list 1st context 2nd context 3rd context 4th context Partition contexts Rank list 1st context 2nd context 3rd context 4th context Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Definition Contextual dissimilarity measures Retrieval process Experiments Reranking with contexts Simple Content-Based Image Retriebal system Query by example ◮ Query image ◮ Rank list With global descriptors ◮ GIST descriptors ◮ No bag-of-word Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Definition Contextual dissimilarity measures Retrieval process Experiments Reranking with contexts Reranking with a single context Given a short list ◮ Take a subset of the short list (from clustering or by truncating) ◮ Estimate the context: centroid of the points ◮ For all entry cs f ( q , p i | u k ) ◮ Rerank according the new scores Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Definition Contextual dissimilarity measures Retrieval process Experiments Reranking with contexts Reranking with multiple contexts Given a short list ◮ Build a set of contexts ◮ For each context, for all entry cs f ( q , p i | u k ) ◮ Average the different scores ◮ Rerank according the new scores Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Experimental setup Contextual dissimilarity measures Results Experiments Comparisons Contextual Measure of Baseline Dissimilarity ◮ GIST descriptors ◮ GIST descriptors ◮ α -divergences ◮ α -divergences ◮ Kullback-Leibler ◮ Kullback-Leibler divergence divergence No bag-of-words, and global descriptors ◮ focus on dissimilarity measure Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Experimental setup Contextual dissimilarity measures Results Experiments Dataset Holidays dataset ◮ from INRIA ◮ J´ egou et al. 2008 Details ◮ 1500 images ◮ 500 classes ◮ 3 images by classes Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Experimental setup Contextual dissimilarity measures Results Experiments Evaluation Mean average precision (mAP) ◮ Average of the precisions at the point of each relevant document in the rank list Warning ◮ Scores not directly comparable with Perronin one’s ◮ No use of a bag-of-word ◮ GIST descriptors Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Experimental setup Contextual dissimilarity measures Results Experiments Results: influence of α Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
α -divergences Experimental setup Contextual dissimilarity measures Results Experiments Results: various divergences Olivier Schwander, Frank Nielsen Reranking with Contextual Dissimilarity Measures
Recommend
More recommend
