Sublinear Time Nearest Neighbor Search over Generalized Weighted - PowerPoint PPT Presentation

Sublinear Time Nearest Neighbor Search over Generalized Weighted Space Yifan Lei 𝐑𝐣𝐛𝐨𝐡 𝐈𝐯𝐛𝐨𝐡 Mohan S. Kankanhalli Anthony K. H. Tung School of Computing, National University of Singapore Source code: https://github.com/1flei/aws_alsh 2019/6/11 1

Applications  Nearest Neighbor Search (NNS) is widely used  Example: booking hotel for ICML 2019  Considering the conditions to the convention centre, i.e., price, distance, and rating  Query 𝑟 : a hotel that the user booked before and felt excellent  Weight vector 𝑥 : different users have different preference to the hotel conditions, which lead to different choices of hotels Price Distance Rating Price Distance Rating Hotel 𝑟 300 7 10 Hotel 1 400 8 10 Hotel 2 350 6 8 𝑥 = 0.001, 1, 1 → Hotel 2 Hotel 3 250 9 8 → Hotel 1 𝑥 = 0, 1, 3 𝑥 = 0.001, −1, 1 → Hotel 3 Hotel 4 200 6 6 𝑥 = −0.001, −1, −1 → Hotel 4 2019/6/11 2

Problem Defi finition  Given  A dataset 𝒠 of 𝑜 data objects in ℝ 𝑒  A query 𝑟 ∈ ℝ 𝑒 with a weight vector 𝑥 ∈ ℝ 𝑒  Measure: the Generalized Weighted Square Euclidean Distance (GWSED) 𝑒 𝑥 𝑒 𝑥 𝑗 𝑝 𝑗 − 𝑟 𝑗 2 𝑒 𝑥 𝑝, 𝑟 = ෍ 𝑗=1  Nearest Neighbor Search (NNS) over 𝑒 𝑥  To find 𝑝 ∗ ∈ 𝒠 s.t. 𝑝 ∗ = arg min 𝑝∈𝒠 𝑒 𝑥 (𝑝, 𝑟)  This problem is very fundamental  Furthest Neighbor Search (FNS) and MIPS can be reduced to NNS over 𝑒 𝑥 ,  i.e., 𝑥 𝑗 = −1, ∀𝑗 ⟹ arg min 𝑝∈𝒠 𝑒 𝑥 𝑝, 𝑟 = arg max 𝑝 − 𝑟 𝑝∈𝒠 2019/6/11 3

Background and Motivations  Locality-Sensitive Hashing (LSH)  Sublinear time for Near Neighbor Search  Insight: construct a hash function ℎ s.t. 𝑄𝑠[ℎ 𝑝 = ℎ(𝑟)] is monotonic in 𝐸𝑗𝑡𝑢(𝑝, 𝑟)  Hidden condition: 𝐸𝑗𝑡𝑢(𝑝, 𝑟) must be a metric  LSH schemes cannot solve NNS over 𝑒 𝑥 directly ( 𝑒 𝑥 is no longer a metric if 𝑥 𝑗 < 0 )  There is NO sublinear method for this problem  Motivations  Similar to 𝑒 𝑥 , inner product (i.e., 𝑝 𝑈 𝑟 ) is also not a metric  However, Shrivastava & Li (2014) introduced a sublinear time method based Asymmetric LSH which constructs 𝑄(𝑝) and 𝑅(𝑟) for data objects 𝑝 ∈ 𝒠 and each query 𝑟 , respectively. 2019/6/11 4

Spherical Asymmetric Transformation  Negative result:  There is no Asymmetric LSH family over ℝ 𝑒 for NNS over 𝑒 𝑥 ( Lemma 1 and Theorem 2 )  Spherical Asymmetric Transformation (SphAT): ℝ 𝑒 → ℝ 2𝑒 𝑄 𝑝 = 𝐷𝑃𝑇 𝑝 ; 𝑇𝐽𝑂 𝑝 𝑅 𝑟, 𝑥 = 𝑥⨂𝐷𝑃𝑇 𝑟 ; 𝑥⨂𝑇𝐽𝑂 𝑟  where 𝑥⨂𝐷𝑃𝑇 𝑟 = (𝑥 1 cos 𝑟 1 , 𝑥 2 cos 𝑟 2 , … , 𝑥 𝑒 cos 𝑟 𝑒 )  Properties of SphAT:  𝑒 𝑥 𝑝, 𝑟 ~ Euclidean distance (or Angular distance) between 𝑄 𝑝 and 𝑅(𝑟, 𝑥)  SphAT is weight-oblivious (because 𝑄(⋅) is independent of 𝑥 ) ⟹ build index before 𝑟 and 𝑥 2019/6/11 5

Two Proposed Methods  SL-ALSH = SphAT + E2LSH  SphAT: arg min 𝑝∈𝒠 𝑒 𝑥 𝑝, 𝑟 ⇒ arg min 𝑄 𝑝 − 𝑅 𝑟, 𝑥 𝑝∈𝒠  Apply E2LSH on 𝑄 𝑝 and 𝑅 𝑟, 𝑥 for NNS over Euclidean distance  S2-ALSH = SphAT + SimHash 𝑄 𝑝 𝑈 𝑅(𝑟,𝑥)  SphAT: arg min 𝑝∈𝒠 𝑒 𝑥 𝑝, 𝑟 ⇒ arg max 𝑄(𝑝) 𝑅(𝑟,𝑥) 𝑝∈𝒠  Apply SimHash on 𝑄 𝑝 and 𝑅 𝑟, 𝑥 for NNS over Angular distance  Main Results  𝑄𝑠[ℎ 𝑄 𝑝 = ℎ(𝑅(𝑟, 𝑥))] is monotonic in 𝑒 𝑥 (𝑝, 𝑟) ( Lemmas 3 and 4 )  SL-ALSH and S2-ALSH solve the problem of NNS over 𝑒 𝑥 with sublinear time ( Theorems 3 and 4 ) 2019/6/11 6

Datasets and Settings  Datasets  Mnist ( 𝑜 = 60,000 and 𝑒 = 784 )  Sift ( 𝑜 = 1,000,000 and 𝑒 = 128 )  Movielens ( 𝑜 = 52,889 and 𝑒 = 150 )  Five types of weight vector 𝑥 Types Illustrations Identical All “1” Uniformly distributed in 0,1 𝑒 Binary Normal 𝑒 -dimensional normal distribution 𝒪(0, 𝐽) Uniformly distributed in 0,1 𝑒 Uniform Negative All “ - 1” 2019/6/11 7

Bucketing Experiments Figure: The best fraction of dataset to scan to achieve certain level of recalls ( lower is better ). 2019/6/11 8

Conclusions  Demonstrate that there is no Asymmetric LSH family over ℝ 𝑒 for the problem of NNS over 𝑒 𝑥  Introduce a novel SphAT from ℝ 𝑒 to ℝ 2𝑒  SphAT is weight-oblivious  𝑄𝑠[ℎ 𝑄 𝑝 = ℎ(𝑅(𝑟, 𝑥))] is monotonic in 𝑒 𝑥 (𝑝, 𝑟)  Propose the first two sublinear time methods SL-ALSH and S2-ALSH for NNS over 𝑒 𝑥  Extensive experiments verify that SL-ALSH and S2-ALSH answer the NNS queries in sublinear time and support various types of weight vectors. 2019/6/11 9

Poster Session [Poster #82: Tue Jun 11th 06:30 — 09:00 PM @Pacific Ballroom] Thank you for your attention! 2019/6/11 10