Sublinear Time Nearest Neighbor Search over Generalized Weighted - - PowerPoint PPT Presentation

Sublinear Time Nearest Neighbor Search over Generalized Weighted Space

Source code: https://github.com/1flei/aws_alsh

Yifan Lei 𝐑𝐣𝐛𝐨𝐡 𝐈𝐯𝐛𝐨𝐡 Mohan S. Kankanhalli Anthony K. H. Tung

School of Computing, National University of Singapore

2019/6/11 1

Applications

 Nearest Neighbor Search (NNS) is widely used  Example: booking hotel for ICML 2019

 Considering the conditions to the convention centre, i.e., price, distance, and rating  Query 𝑟: a hotel that the user booked before and felt excellent  Weight vector 𝑥: different users have different preference to the hotel conditions, which lead to

different choices of hotels

2019/6/11 2

Price Distance Rating Hotel 1 400 8 10 Hotel 2 350 6 8 Hotel 3 250 9 8 Hotel 4 200 6 6 Price Distance Rating Hotel 𝑟 300 7 10 𝑥 = 0.001, 1, 1 𝑥 = 0, 1, 3 𝑥 = 0.001, −1, 1 𝑥 = −0.001, −1, −1 → Hotel 2 → Hotel 1 → Hotel 3 → Hotel 4

Problem Defi finition

 Given

 A dataset 𝒠 of 𝑜 data objects in ℝ𝑒  A query 𝑟 ∈ ℝ𝑒 with a weight vector 𝑥 ∈ ℝ𝑒  Measure: the Generalized Weighted Square Euclidean Distance (GWSED) 𝑒𝑥

𝑒𝑥 𝑝, 𝑟 = ෍

𝑗=1 𝑒

𝑥𝑗 𝑝𝑗 − 𝑟𝑗 2

 Nearest Neighbor Search (NNS) over 𝑒𝑥

 To find 𝑝∗ ∈ 𝒠 s.t. 𝑝∗ = arg min

𝑝∈𝒠 𝑒𝑥(𝑝, 𝑟)

 This problem is very fundamental

 Furthest Neighbor Search (FNS) and MIPS can be reduced to NNS over 𝑒𝑥,  i.e., 𝑥𝑗 = −1, ∀𝑗 ⟹ arg min

𝑝∈𝒠 𝑒𝑥 𝑝, 𝑟 = arg max 𝑝∈𝒠

𝑝 − 𝑟

2019/6/11 3

Background and Motivations

 Locality-Sensitive Hashing (LSH)

 Sublinear time for Near Neighbor Search  Insight: construct a hash function ℎ s.t. 𝑄𝑠[ℎ 𝑝 = ℎ(𝑟)] is monotonic in 𝐸𝑗𝑡𝑢(𝑝, 𝑟)  Hidden condition: 𝐸𝑗𝑡𝑢(𝑝, 𝑟) must be a metric

 LSH schemes cannot solve NNS over 𝑒𝑥 directly (𝑒𝑥 is no longer a metric if 𝑥𝑗 < 0)  There is NO sublinear method for this problem  Motivations

 Similar to 𝑒𝑥, inner product (i.e., 𝑝𝑈𝑟) is also not a metric  However, Shrivastava & Li (2014) introduced a sublinear time method based Asymmetric LSH which

constructs 𝑄(𝑝) and 𝑅(𝑟) for data objects 𝑝 ∈ 𝒠 and each query 𝑟, respectively.

2019/6/11 4

Spherical Asymmetric Transformation

 Negative result:

 There is no Asymmetric LSH family over ℝ𝑒 for NNS over 𝑒𝑥 (Lemma 1 and Theorem 2)

 Spherical Asymmetric Transformation (SphAT): ℝ𝑒 → ℝ2𝑒

𝑄 𝑝 = 𝐷𝑃𝑇 𝑝 ; 𝑇𝐽𝑂 𝑝 𝑅 𝑟, 𝑥 = 𝑥⨂𝐷𝑃𝑇 𝑟 ; 𝑥⨂𝑇𝐽𝑂 𝑟

 where 𝑥⨂𝐷𝑃𝑇 𝑟 = (𝑥1 cos 𝑟1 , 𝑥2 cos 𝑟2 , … , 𝑥𝑒 cos 𝑟𝑒)

 Properties of SphAT:

 𝑒𝑥 𝑝, 𝑟 ~ Euclidean distance (or Angular distance) between 𝑄 𝑝 and 𝑅(𝑟, 𝑥)  SphAT is weight-oblivious (because 𝑄(⋅) is independent of 𝑥) ⟹ build index before 𝑟 and 𝑥 2019/6/11 5

Two Proposed Methods

 SL-ALSH = SphAT + E2LSH

 SphAT: arg min

𝑝∈𝒠 𝑒𝑥 𝑝, 𝑟 ⇒ arg min 𝑝∈𝒠

𝑄 𝑝 − 𝑅 𝑟, 𝑥

 Apply E2LSH on 𝑄 𝑝 and 𝑅 𝑟, 𝑥 for NNS over Euclidean distance

 S2-ALSH = SphAT + SimHash

 SphAT: arg min

𝑝∈𝒠 𝑒𝑥 𝑝, 𝑟 ⇒ arg max 𝑝∈𝒠 𝑄 𝑝 𝑈𝑅(𝑟,𝑥) 𝑄(𝑝) 𝑅(𝑟,𝑥)

 Apply SimHash on 𝑄 𝑝 and 𝑅 𝑟, 𝑥 for NNS over Angular distance

 Main Results

 𝑄𝑠[ℎ 𝑄 𝑝

= ℎ(𝑅(𝑟, 𝑥))] is monotonic in 𝑒𝑥(𝑝, 𝑟) (Lemmas 3 and 4)

 SL-ALSH and S2-ALSH solve the problem of NNS over 𝑒𝑥 with sublinear time (Theorems 3 and 4) 2019/6/11 6

Datasets and Settings

 Datasets

 Mnist (𝑜 = 60,000 and 𝑒 = 784)  Sift (𝑜 = 1,000,000 and 𝑒 = 128)  Movielens (𝑜 = 52,889 and 𝑒 = 150)

 Five types of weight vector 𝑥

Types Illustrations Identical All “1” Binary Uniformly distributed in 0,1 𝑒 Normal 𝑒-dimensional normal distribution 𝒪(0, 𝐽) Uniform Uniformly distributed in 0,1 𝑒 Negative All “-1”

2019/6/11 7

Bucketing Experiments

Figure: The best fraction of dataset to scan to achieve certain level of recalls (lower is better).

2019/6/11 8

Conclusions

 Demonstrate that there is no Asymmetric LSH family over ℝ𝑒 for the problem of NNS

• ver 𝑒𝑥

 Introduce a novel SphAT from ℝ𝑒 to ℝ2𝑒

 SphAT is weight-oblivious  𝑄𝑠[ℎ 𝑄 𝑝

= ℎ(𝑅(𝑟, 𝑥))] is monotonic in 𝑒𝑥(𝑝, 𝑟)

 Propose the first two sublinear time methods SL-ALSH and S2-ALSH for NNS over 𝑒𝑥  Extensive experiments verify that SL-ALSH and S2-ALSH answer the NNS queries in

sublinear time and support various types of weight vectors.

2019/6/11 9

2019/6/11 10

Thank you for your attention! Poster Session

[Poster #82: Tue Jun 11th 06:30—09:00 PM @Pacific Ballroom]