Introduction Sparse Representation ℓ 1 -Minimization Low-Rank Representation Future Topics Discussion High-Dimensional Pattern Recognition via Sparse Representation Allen Y. Yang Department of EECS, UCB yang@eecs.berkeley.edu HP Labs, March 2012 High-Dimensional Pattern Recognition via Sparse Representation http://www.eecs.berkeley.edu/~yang
Introduction Sparse Representation ℓ 1 -Minimization Low-Rank Representation Future Topics Discussion Shifting Paradigms in High-Dimensional Pattern Recognition Face Recognition Yale B CMU Multi-PIE Facebook Photo Tagging Object Recognition ETHZ Cows vs Cars Caltech 256 Caltech 101 3D Reconstruction Oxford Corridor Berkeley Downtown Google Earth High-Dimensional Pattern Recognition via Sparse Representation http://www.eecs.berkeley.edu/~yang
Introduction Sparse Representation ℓ 1 -Minimization Low-Rank Representation Future Topics Discussion Shifting Paradigms in Distributed Computing The Internet ! "#" $ %& & ' ( $ ' )# %( * + ' ( #, - # - ' + ! + . ' ( / , 0 ! ' . $ 12 2 3453 . 367" 89: 34; 35437<4= 1> 351912 ? 5 <@ <? 4; ( 1A347 %47934 ; <? : 7B<7 B3 A3C ? 3D3: 7B3 : 1C C <4 <75844397 C 3D3C ; E <; > <? 4C = 64? 53: <F<? 9; 72 1; 7' 84163<9 5844395? 3; " G 9 < E ? : 3 4<9F? 9F <: : 43; ; ; 619; 143: A= 7B3 ' H6147IG 2 6147 J <9KL %47934L 7B3 A<9K); ; 39? 14 351912 ? ; 7 <C ; 1 ; <? : B3 A3C ? 3D3: 7B<7 7B3 =39 E <; 89: 34D <C 83: <9: 518C : F1 86 A= M N 14 M O 657" PG: 1 917 43F<4: 7B3 : 1C C <4 <; 89: 34D <C 83: <7 7B? ; 61? 97 <F<? 9; 7 7B3 =39L P B3 ; <? : " %9 7B3 17B34 B<9: L %47934 ; <? : 7B<7 B3 7B18FB7 7B<7 P7B3 =39 ? ; ; 7? C C < C ? 77C 3 A? 7 89: 34D<C 83: LP <9: P518C : F1 86 <917B34 M N 14 M O 657"P G 9 <: : ? 7? 19L %47934L E B1 ; <? : B3 E <; ; 63<K? 9F 634; 19<C C =L ; <? : B3 7B18FB7 7B<7 7B3 : 1C C <4 <F<? 9; 7 2 1; 7 ' 84163<9 5844395? 3; E <; P> <? 4C = 64? 53: "P %47934 ; <? : B? ; <9<C =; ? ; 1> 7B3 D <4? 18; 3H5B<9F3 4<73 D<C 83; E <; A<; 3: 19 ; 85B 351912 ? 5 6<47? 58C <4; <; E <F3 4<73 : ? @ 34397? <7? 19; " %47934 ; <? : 7B343 B<: A339 C ? 77C 3 ? 2 6<57 19 ! "#" 74<: 3 : 3Q5? 7 A= 7B3 : 35C ? 93 1>7B3 : 1C C <4 A35<8; 3 <7 7B3 7? 2 3 1>7B3 R C <S< , 5514: L 7B3 : 1C C <4 E <; 3H7432 3C = 1D34D <C 83: <9: 7B<7 7B3 Q4; 7 M O 657 : 35C ? 93 B<: C ? 77C 3 ? 2 6<57" T 3 ; <? : 7B343 E 343 ? 9: ? 5<7? 19; 91E 7B<7 7B3 74<: 3 : 3Q5? 7 E <; A3F? 99? 9F 71 C 3D3C1@ " * 849? 9F 71 J 4<S? C <9: & 3H? 51L %47934 2 <: 3 ? 7 5C 3<4 7B<7 ? 7 E 18C : A3 <C 2 1; 7 ? 2 61; ; ? AC 3 > 14 7B1; 3 518974? 3; 71 3<49 3918FB > 143? F9 3H5B<9F3 71 6<= 7B3 ; 34D? 53 19 7B3? 4 : 3A7; " T 3 ; <? : 7B3 A3; 7 E <= 71 : 3<CE ? 7B 7B? ; E <; 71 8; 3 7B3 61C ? 5? 3; 187C ? 93: ? 9 * 43<; 84= #35437<4= U<2 3; J <K34); : 3A7 ? 9? 7? <7? D3" average 1M pixels average 1B voxels 50B webpages From desktop computing to mobile computing Vision-based robot control and navigation High-Dimensional Pattern Recognition via Sparse Representation http://www.eecs.berkeley.edu/~yang
Introduction Sparse Representation ℓ 1 -Minimization Low-Rank Representation Future Topics Discussion Accurate recognition of HD pattens presents unique challenges Programming real-time systems on low-power mobile devices is difficult. 1 Distributed, real-time applications demand extremely high accuracy . 2 Scenarios require the ability to obtain detailed 3-D representation of the models. 3 (a) Frueh & Zakhor ’03 (b) Su et al. ’09 High-Dimensional Pattern Recognition via Sparse Representation http://www.eecs.berkeley.edu/~yang
Introduction Sparse Representation ℓ 1 -Minimization Low-Rank Representation Future Topics Discussion Outline Main Message The rich phenomena of sparse representation in HD data can provide novel pattern recognition solutions and successfully mitigate the curse of dimensionality and other challenges. Robust face recognition via ℓ 1 -minimization and group sparsity 1 Accelerated sparse optimization algorithms and parallelization 2 Sparsity in matrix rank to extract robust image representation for 3-D reconstruction 3 Ongoing projects: Informative feature selection via Sparse PCA 1 Robust 3-D motion registration via sparse online low-rank projection 2 Compressive phase retrieval via semidefinite-programming 3 High-Dimensional Pattern Recognition via Sparse Representation http://www.eecs.berkeley.edu/~yang
Introduction Sparse Representation ℓ 1 -Minimization Low-Rank Representation Future Topics Discussion Face Recognition via Sparse Representation Face-subspace model [Belhumeur et al. ’97, Basri & Jacobs ’03] 1 Assume b belongs to Class i from K classes. b = α i , 1 a i , 1 + α i , 2 a i , 2 + · · · + α i , n 1 a i , n i , = A i α i . High-Dimensional Pattern Recognition via Sparse Representation http://www.eecs.berkeley.edu/~yang
Introduction Sparse Representation ℓ 1 -Minimization Low-Rank Representation Future Topics Discussion Face Recognition via Sparse Representation Face-subspace model [Belhumeur et al. ’97, Basri & Jacobs ’03] 1 Assume b belongs to Class i from K classes. b = α i , 1 a i , 1 + α i , 2 a i , 2 + · · · + α i , n 1 a i , n i , = A i α i . Nevertheless, Class i is the unknown label we need to solve: 2 α 1 2 3 α 2 5 = A x . . Sparse representation b = [ A 1 , A 2 , · · · , A K ] . 4 . α K High-Dimensional Pattern Recognition via Sparse Representation http://www.eecs.berkeley.edu/~yang
Introduction Sparse Representation ℓ 1 -Minimization Low-Rank Representation Future Topics Discussion Face Recognition via Sparse Representation Face-subspace model [Belhumeur et al. ’97, Basri & Jacobs ’03] 1 Assume b belongs to Class i from K classes. b = α i , 1 a i , 1 + α i , 2 a i , 2 + · · · + α i , n 1 a i , n i , = A i α i . Nevertheless, Class i is the unknown label we need to solve: 2 α 1 2 3 α 2 5 = A x . . Sparse representation b = [ A 1 , A 2 , · · · , A K ] . 4 . α K 0 ··· 0 ] T ∈ R n . x ∗ = [ 0 ··· 0 α T 3 i Sparse representation x ∗ encodes membership through its nonzero coefficients! Reference: Wright, AY, Sastry, Ma, Robust face recognition via sparse representation . IEEE PAMI , 2009. High-Dimensional Pattern Recognition via Sparse Representation http://www.eecs.berkeley.edu/~yang
Introduction Sparse Representation ℓ 1 -Minimization Low-Rank Representation Future Topics Discussion Sparse Optimization via ℓ 1 -Minimization An inverse problem with an underdetermined system of linear equations, A in general is full rank: where A ∈ R d × n , ( d < n ) b = A x Two interpretations: 1 Compressive sensing: A is a sensing matrix. 2 Sparse representation: A is a prior dictionary. High-Dimensional Pattern Recognition via Sparse Representation http://www.eecs.berkeley.edu/~yang
Introduction Sparse Representation ℓ 1 -Minimization Low-Rank Representation Future Topics Discussion Sparse Optimization via ℓ 1 -Minimization An inverse problem with an underdetermined system of linear equations, A in general is full rank: where A ∈ R d × n , ( d < n ) b = A x Two interpretations: 1 Compressive sensing: A is a sensing matrix. 2 Sparse representation: A is a prior dictionary. Infinitely many solutions for x , without extra regularization ( P 0 ) : x ∗ 0 = arg min � x � 0 subj. to A x = b ( P 1 ) : x ∗ 1 = arg min � x � 1 subj. to A x = b High-Dimensional Pattern Recognition via Sparse Representation http://www.eecs.berkeley.edu/~yang
Introduction Sparse Representation ℓ 1 -Minimization Low-Rank Representation Future Topics Discussion Image Occlusion, Corruption, and Disguise High-Dimensional Pattern Recognition via Sparse Representation http://www.eecs.berkeley.edu/~yang
Introduction Sparse Representation ℓ 1 -Minimization Low-Rank Representation Future Topics Discussion Image Occlusion, Corruption, and Disguise Sparse representation + sparse error 1 b = A x + e Cross-and-bouquet model [Wright et al. ’09, ’10] 2 I ´ „ x « ` A b = | = B w e When size of A grows proportionally with the sparsity in x , asymptotically CAB can correct 100% noise in e . Reference: Wright and Ma, Dense Error Correction via ℓ 1 Minimization , IEEE Trans. IT, 2011. High-Dimensional Pattern Recognition via Sparse Representation http://www.eecs.berkeley.edu/~yang
Introduction Sparse Representation ℓ 1 -Minimization Low-Rank Representation Future Topics Discussion Performance on the YaleB database Top 100 IEEExplore Download in June, 2010. 800+ citations on Google. High-Dimensional Pattern Recognition via Sparse Representation http://www.eecs.berkeley.edu/~yang
Introduction Sparse Representation ℓ 1 -Minimization Low-Rank Representation Future Topics Discussion Face Alignment Problem: Misalignment violates linear subspace model High-Dimensional Pattern Recognition via Sparse Representation http://www.eecs.berkeley.edu/~yang
Recommend
More recommend
