MIMA Group Fast Scalable Supervised Hashing Xin Luo 1 , Liqiang Nie 2 , Xiangnan He 3 Ye Wu 1 , Zhen-Duo Chen 1 , Xin-Shun Xu 1 1 Lab of M achine I ntelligence & M edia A nalysis, School of Software, Shandong University, China 2 School of Computer Science and Technology,Shandong University, China 3 National University of Singapore, Singapore School of Software, Shandong University MIMA @ SDU
Outline MIMA n Introduction n Proposed Method n Overall Objective Function n Optimization Algorithm n Out-of-Sample Extension n Experiments n FSSH_deep n Conclusion & Future Work School of Software, Shandong University MIMA @ SDU 2
Introduction MIMA n Nearest Neighbor Search � NNS � n Given a query point q, NNS returns the points closest (most similar) to q in the database. query n Underlying many machine learning, information retrieval, and computer vision problems. School of Software, Shandong University MIMA @ SDU 3
Introduction MIMA n Challenges in large-scale data applications: n Expensive storage cost n Slow query speed n data on the Internet increases explosively n curse of dimensionality problem n One popular solution is the hashing based approximate nearest neighbor (ANN) technique. School of Software, Shandong University MIMA @ SDU 4
Introduction MIMA Illustration comes from http://cs.nju.edu.cn/lwj/L2H.html Similarity Preserving School of Software, Shandong University MIMA @ SDU 5
Introduction MIMA Illustration comes from http://cs.nju.edu.cn/lwj/L2H.html Ø advantages of hashing: ü fast query speed ü low storage cost 1 0 0 0 1 0 1 0 0 1 1 0 0 0 0 1 0 1 1 0 0 1 0 1 XOR Hamming distance School of Software, Shandong University MIMA @ SDU 6
Introduction MIMA n According whether to use semantic information: n unsupervised hashing n supervised hashing (better retrieval accuracy) n We propose a novel supervised hashing method, named, F ast S calable S upervised H ashing (FSSH). School of Software, Shandong University MIMA @ SDU 7
Introduction MIMA n Two commonly used objective functions: instance pairwise semantic labels for n instances similarity instance i and instance j instance i is in class k are semantically similar i and j are semantically instance i is not in class k dissimilar r-bit binary hash codes for a projection from labels to n instances hash codes Frobenius norm the number of classes c School of Software, Shandong University MIMA @ SDU 8
Introduction MIMA n Two commonly used objective functions: limitations instance pairwise semantic labels for n instances similarity instance i and instance j instance i is in class k are semantically similar i and j are semantically instance i is not in class k dissimilar r-bit binary hash codes for a projection from hash n instances codes to labels Frobenius norm the number of classes c School of Software, Shandong University MIMA @ SDU 9
Proposed Method MIMA n Motivations: n How to generate hash codes fast? n How to make the model scalable to large-scale data? n How to guarantee precise hash codes? School of Software, Shandong University MIMA @ SDU 10
Proposed Method MIMA n Motivations: n How to generate hash codes fast? n How to make the model scalable to large-scale data? n How to guarantee precise hash codes? Ø simultaneously update all bits of hash codes Ø avoid the direct use of large matrix S Ø consider both semantic and visual information School of Software, Shandong University MIMA @ SDU 11
Proposed Method MIMA n Overall Objective Function School of Software, Shandong University MIMA @ SDU 12
Proposed Method - optimization MIMA n We present an iterative optimization algorithm, in which each iteration contains three steps, i.e., W Step, G Step, and B Step. n More specifically, n W step: fix G and B , update W ; n G step: fix W and B , update G ; n B step: fix W and G , update B . School of Software, Shandong University MIMA @ SDU 13
Proposed Method - optimization MIMA n W Step School of Software, Shandong University MIMA @ SDU 14
Proposed Method - optimization MIMA n W Step setting the derivative regarding W to zero School of Software, Shandong University MIMA @ SDU 15
Proposed Method - optimization MIMA n W Step setting the derivative regarding W to zero School of Software, Shandong University MIMA @ SDU 16
Proposed Method - optimization MIMA n W Step setting the derivative regarding W to zero where , , . m << n, c << n School of Software, Shandong University MIMA @ SDU 17
Proposed Method - optimization MIMA n G Step setting the derivative regarding W to zero where , , . School of Software, Shandong University MIMA @ SDU 18
Proposed Method - optimization MIMA n B Step Then, we transform the above equation into, where Tr( ) is the trace norm. School of Software, Shandong University MIMA @ SDU 19
Proposed Method - optimization MIMA n B Step Then, we transform the above equation into, constants where Tr( ) is the trace norm. School of Software, Shandong University MIMA @ SDU 20
Proposed Method - optimization MIMA n B Step Thus, B can also be solved with a closed-form solution stated as follows, School of Software, Shandong University MIMA @ SDU 21
Proposed Method - optimization MIMA School of Software, Shandong University MIMA @ SDU 22
Proposed Method - Out-of-Sample Extension MIMA FSSH_os simultaneously learns its hash functions and hash codes FSSH_ts uses linear regression as the hash function FSSH_deep adopts deep network as the hash function Suppose X query and B query are the original features and corresponding hash codes of the queries. FSSH_os School of Software, Shandong University MIMA @ SDU 23
Proposed Method - Out-of-Sample Extension MIMA FSSH_ts where λ e is a balance parameter, is the regularization term, and is the RBF kernel features. Then, the optimal P can be computed as, School of Software, Shandong University MIMA @ SDU 24
Experiments MIMA n Datasets n MNIST n CIFAR-10 n NUS-WIDE CIFAR-10 MNIST School of Software, Shandong University MIMA @ SDU 25
Experiments MIMA n Compared supervised methods n one-step hashing: KSH, SDH, FSDH. n two-step hashing: TSH, LFH, COSDISH. n Evaluation Metrics (accuracy) n Mean Average Precision (MAP), n Top-N Precision curves, n Precision-Recall curves. n Time cost (efficiency) School of Software, Shandong University MIMA @ SDU 26
Experiments - MAP results MIMA • The best MAP values of each case are shown in boldface. • One-step hashing: KSH, SDH, FSDH, and FSSH_os. • Two-step hashing: TSH, LFH, COSDISH, and FSSH_ts. School of Software, Shandong University MIMA @ SDU 27
Experiments - curves MIMA School of Software, Shandong University MIMA @ SDU 28
Experiments - time MIMA • The numbers of training images on CIFAR-10 and MNIST are 59,000 and 69,000, respectively. • Only 2,000 samples are used for training KSH and TSH due to their large complexity. School of Software, Shandong University MIMA @ SDU 29
FSSH _deep MIMA n FSSH_deep is one two-step variant of FSSH: n 1 st STEP: We use features which are extracted by an off- the-shelf deep network. n 2 nd STEP: We adopt CNN-F network as the hash function. (We train the network by solving a multi-label classi cation problem.) n Compared deep hashing methods include DSRH, DSCH, DRSCH, DPSH, VDSH, DTSH, and DSDH. School of Software, Shandong University MIMA @ SDU 30
FSSH _deep - MAP results MIMA • All baselines are end-to-end methods. • FSSH_deep is not end-to-end. • For a fair comparison, the results of baselines are directly reported from previous works. School of Software, Shandong University MIMA @ SDU 31
Conclusion & Future Work MIMA n We propose a novel supervised hashing method named Fast Scalable Supervised Hashing. n FSSH can be trained extremely fast. n FSSH is scalable to large-scale data. n FSSH generates precise hash codes. n Three variants of FSSH are further proposed: n two shallow variants, i.e., FSSH_os and FSSH_ts n one deep variant, i.e., FSSH_deep School of Software, Shandong University MIMA @ SDU 32
Conclusion & Future Work MIMA n Extensive experiments are conducted on three benchmark datasets. Experimental results shows the superiority of FSSH. n In future, we plan to realize our proposed FSSH method in an end-to-end deep version to boost its performance. School of Software, Shandong University MIMA @ SDU 33
MIMA Group Any Question? Codes are available at: https://lcbwlx.wixsite.com/fssh School of Software, Shandong University MIMA @ SDU
Recommend
More recommend
