Metric Learning Applied for Automatic Large Image Classification - - PowerPoint PPT Presentation
September, 2014 UPC Metric Learning Applied for Automatic Large Image Classification Supervisors TOON CALDERS (PhD) / ULB SAHILU WENDESON / IT4BI SALIM JOUILI (PhD) /EuraNova Image Database Classification How? Using K-Nearest Neighbor (kNN)
Metric Learning Applied for Automatic Large
September, 2014 UPC
Image Classification
SAHILU WENDESON / IT4BI
TOON CALDERS (PhD) /ULB SALIM JOUILI (PhD)/EuraNova Supervisors
Image Database
2
Classification Using K-Nearest Neighbor (kNN) Quality of Distance Measure Depends on How?
(k=1) (k=5) k neighbors choose based on Euclidean distance measure Depends on the distance measure
Q
(k=1) (k=5)
3
Outline
1) METRIC LEARNING 2) INDEXING 3) OBJECTIVES and CONTRIBUTIONS 4) EXPERIMENT RESULTS 5) DISCUSSION,CONCLUSION and FUTURE WORKS 3) OBJECTIVES and CONTRIBUTIONS
distance measure
– Consider correlation between features – Consider correlation between features – To provide curved as well as linear decision boundaries
using Mahalanobis metric space
5
http://www.cs.princeton.edu/courses/archive/fall08/cos436/Duda/PR_Mahal/PR_Mahal.htm, 1997.
Where is the cone of symmetric PSD
6
1http://www.ias.ac.in/resonance/Volumes/04/06/0020-0026.pdf
Using Cholesky decomposition : And rewrite dM As follow:
Euclidean Vs Mahanalobis Space
Age
Euclidean Space Mahalanalobis Space
Age
7
Weight M M=GTG Weight
– Driven by Nearest Neighbors – Information -theoretical – Online – Etc – Etc
– MMC, Xing et al. (2002) – NCA, leave-one out cross validation (LOO), Goldberger et al.(2004) – MCML, Globerson and Roweis (2005) – LMNN, Weinberger et al. (2005)
8
Euclidean Metric Mahanalobis Metric
Local neighborhood
Margin
9
M=GTG
Target Neighbor Impostors
– Scales to large datasets – Has fast test-time performance – Convex optimization – Can solve efficiently Can solve efficiently – No assumption about the data – Number of target, prior assignment
– Sensitive to outliers – Dimension Reduction
Principle Component Analysis (PCA)2
10 1http://www.cse.wustl.edu/~kilian/papers/jmlr08_lmnn.pdf
2http://computation.llnl.gov/casc/sapphire/pubs/148494.pdf
– In the mean-square error sense – Linear dimension reduction – Based on covariance matrix of the variables – Used to reduce computation time and avoid overfitting
11
PCA
Dataset (labeled ) Dataset (labeled ) Training set Training set Testing set Testing set Normalize and dimension reduction Normalize and dimension reduction Metric Learning LMNN LMNN Best PSD, M=GTG Best PSD, M=GTG
Test, 30% Build, 70% Plug
12
Evaluation
Build Model
Test model Model Build model using LMNN Model
0.15 0.2 0.25 0.3
intra/inter ratio
Mahalanobis
13
0.05 0.1 0.15
1 2 3 4 5 6 7 8 9 10
ratio
Class
Mahalanobis Euclidean
Mnist, has 10 classes, intra/inter ratio, number of target = 3
Dataset (labeled ) Dataset (labeled ) Training set Training set Testing set Testing set Normalize and dimension reduction Normalize and dimension reduction Metric Learning LMNN LMNN Best PSD, M=GTG Best PSD, M=GTG
Test, 30% Build, 70% Plug
14
Evaluation
Build Model
Test model Model Build model using LMNN Model
7.4 9.6 2.5 4.3 4.7 1.5
Bal ISOLET Mnist
atasets
15
4.3 5.9 7.4 3.2 2.6
2 4 6 8 10 12
Iris Faces
Error rate Data
Mahanalobis Euclidean Error rate LMNN Vs Euclidean Metrics, (k =5), number of target = 3
Statistics Mnist Letters Isolet Bal Wines Iris
#inputs 70000 20000 7797 535 152 128 #features 784 16 617 4 13 4 #reduced dimensions 164 16 172 4 13 4 #training examples 60000 14000 6238 375 106 90 #testing examples 10000 6000 1559 161 46 38 #classes 10 26 26 3 3 3
kNN
16
kNN
Euclidean 2,12 4.68 8.98 18.33 25.00 4.87 PCA 2.43 4.68 8.60 18.33 25.00 4.87 RCA 5.93 4.34 5.71 12.31 2.28 3.71 MMC N/A N/A N/A 15.66 30.96 3.55 NCA N/A N/A N/A 5.33 28.67 4.32 LMNN
PCA
1.72 3.60 4.36 11.16 8.72 4.37
Multiple Passes
1.69 2.80 4.30 5.86 7.59 4.26
Weinberger, K. Q. and L. K. Saul (2009). "Distance metric learning for large margin nearest neighbor classification." The Journal of Machine Learning Research 10: 207-244.
Classification using
kNN
Image Database
17
kNN
Time Complexity
Intractable Approximate Nearest Neighbor (ANN) Solution
http://www.cs.utexas.edu/~grauman/courses/spring2008/datasets.htm
Out-line
1) METRIC LEARNING 2) INDEXING 3) OBJECTIVES and CONTRIBUTIONS 4) EXPERIMENT RESULTS 5) DISCUSSION,CONCLUSION and FUTURE WORKS 3) OBJECTIVES and CONTRIBUTIONS
to have the same hash1, Sub-linear time search
Neighbor (ANN) Search,
19
1[Indyk-Motwani’98] http://people.csail.mit.edu/indyk/mmds.pdf
Q
P=4,
:-approximation ratio, r, radius
Cosine Similarity LP Distance Measure Hamming distance Jaccard index for set similarity ……
1 1 0 0
1100
1 1
Feature vector
1100
www.cs.utexas.edu/~grauman/.../jain_et_al_cvpr2008.ppt 20
Basic Hashing Function1 Learned Hashing Function
hr1…rb series of b randomized 10010
hr1…r4 G
Image database
r is d-dimensional random hyperplane, Gaussian distribution
21
1Jain, B. Kulis, and K. Grauman. Fast Image Search for Learned Metrics. In CVPR, 2008
10010 10110 10100 10011
Q
b randomized LSH functions
hr1…r4
10110 10101 10100
Q
Both cases
Colliding instances are searched <<n
22
from a p-stable distribution ,
random line and partition into equi-width segments w, and
near neighbors in each Buckets, under K hash functions.
Image Database
L = 3, number of hash Table
K, number of hash function
No indexing involved
L2 Hash Table L1 Hash Table L3 Hash Table
Key Values X Y W R S Key Values X’ Y’ R’ S’ key Values Y’’ W’’ R’’
Q
23
Out-line
1) METRIC LEARNING 2) INDEXING 3) OBJECTIVES and CONTRIBUTIONS 4) EXPERIMENT RESULTS 5) DISCUSSION,CONCLUSION and FUTURE WORKS 3) OBJECTIVES and CONTRIBUTIONS
– To study and implement
metric learning algorithm dimension reduction technique LSH in different metric space LSH in different metric space – To establish and implement machine learning evaluation techniques
– Formulate a fresh learned approach for both Cosine similarity and Euclidean metric space hashing
25
Out-line
1) METRIC LEARNING 2) INDEXING 3) OBJECTIVES and CONTRIBUTIONS 4) EXPERIMENTAL RESULTS 5) DISCUSSION,CONCLUSION and FUTURE WORKS 3) OBJECTIVES and CONTRIBUTIONS
Dataset (labeled ) Dataset (labeled ) Training set Training set Query set Query set Normalize and dimension reduction Normalize and dimension reduction Transform, M=GTG Transform, M=GTG Metric Learning LMNN LMNN Best PSD, M Best PSD, M
Test, 10% Build, 90% Plug Decompose, M
Hashing (LSH)
27
M=GTG M=GTG
Cosine Similarity Hashing Euclidean Space Hashing
Learned Basic Learned Original
Evaluation
Test model
Hashing (LSH)
Model
Exhaustive Vs Euclidean Space Hashing, 3NN
189 49
80 100 120 140 160 180 200
time (Msecond) Exhaustive Euclidean Hashing
28
Dataset
LetterRecognitioon Isolet Mnist
Instances
20,000 7796 70,000
Dimension
16 617 784 24 22 8 4 49
20 40 60 80
LetterRecognition Isolet Mnist
Datasets
Steps CosineLSH CosineLSH +LMNN E2LSH LMNN + E2LSH Euclidean LMNN
Metric learning projection (offline)
O(d) O(d2)
O(d)
O(d2) O(d) O(d2)
We use to guarantee searching of NN
29
(offline) Hash functions
O(b) O(b) O(Lk) O(Lk) O(0) O(0)
Signature (to represent the data point)
O(1) O(1) O(L) O(L) O(1) O(1)
Hashing: compute
O(bd) O(bd) O(dLk) O(dLk) O(0) O(0)
Search: identity the query's ANNs
O(Md) O(Md) O(LMd) O(LMd) O(dN) O(dN)
computing, 2002, pp. 380-388. http://www.cs.utexas.edu/~ai-lab/pubs/jain_kulis_grauman_cvpr2008.pdf
Dataset (labeled ) Dataset (labeled ) Training set Training set Query set Query set Normalize and dimension reduction Normalize and dimension reduction Transform, M=GTG Transform, M=GTG Metric Learning LMNN LMNN Best PSD, M Best PSD, M
Test, 10% Build, 90% Plug Decompose, M
Hashing (LSH)
30
M=GTG M=GTG
Cosine Similarity Hashing Euclidean Space Hashing
Learned Randomized Learned Original
Evaluation
Test model
Hashing (LSH)
Model K = 10
Accuracy Rate(Cosine Similarity Hashing)
0.6 0.8 1 ISOLET
31
0.2 0.4 0.6
4 8 16 32 64 128
Accuracy Rate
Bit
Randomized Learned
Accuracy Rate(Cosine Similarity Hashing)
0.6 0.8 1 MNIST
32
0.2 0.4 0.6 4 8 16 32 64 128
Accuracy Rate Bit
Randomized Learned
Accuracy Rate(Cosine Similarity Hashing)
0.6 0.8
Accuracy
Cifer100
33
0.2 0.4 4 8 16 32 64 128
Accuracy Rate Bit
Randomized Learned
Accuracy Rate(Euclidean Space Hashing)
0.901 0.682 0.882 0.937 0.812
Letter cifar-100
Accuracy Rate: E2LSH Vs LMNN+E2LSH Datasets
34
0.660 0.891 0.844 0.816 0.916 0.882 0.000 0.200 0.400 0.600 0.800 1.000
OliverFaces ISOLET Mnist
LMNN+E2LSH E2LSH
Accuracy rate
Time LMNN Euclidean Exhaustive techniques
35
Accuracy LMNN + E2LSH E2LSH kNN classification accuracy Euclidean space hashing Learned Cosine Basic Cosine Cosine space hashing
Out-line
1) METRIC LEARNING 2) INDEXING 3) OBJECTIVES and CONTRIBUTIONS 4) EXPERIMENTAL RESULTS 5) DISCUSSION,CONCLUSION and FUTURE WORKS 3) OBJECTIVES and CONTRIBUTIONS
Metric Learning, is used to learn metric distance using Mahanalobis metric space, LMNN E2LSH outperforms both unlearned and learned Cosine similarity hashing. similarity hashing. Incorporating metric learning algorithm (LMNN) into both metric space hashing (Cosine and Euclidean, E2LSH) has a competence to improve the performance significantly. LMNN into E2LSH (LMNN +E2LSH), improves E2LSH
37
The main goal of this research is to devise classifier by breeding metric learning algorithm and hashing technique in the context of large-scale image classification. Java, Eclipse IDE used to implement
38
Future work(s)
Extend this research work by adding feature extraction technique on top, to set up input data
Future Future
Use
LMNN extension algorithms and compare results of this thesis. Propose to advance this study using unsupervised (clustering) metric learning algorithms.
39
40