High-Dimensional Pattern Recognition via Sparse Representation Allen - - PowerPoint PPT Presentation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

High-Dimensional Pattern Recognition via Sparse Representation

Allen Y. Yang University of California, Berkeley and Atheer, Inc. UC Berkeley, June 2013

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Shifting Paradigms in High-Dimensional Pattern Recognition

Face Recognition

Yale B CMU Multi-PIE Facebook Photo Tagging http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Shifting Paradigms in High-Dimensional Pattern Recognition

Face Recognition

Yale B CMU Multi-PIE Facebook Photo Tagging

Object Recognition

ETHZ Cows vs Cars Caltech 101 Amazon Flow

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Shifting Paradigms in High-Dimensional Pattern Recognition

Face Recognition

Yale B CMU Multi-PIE Facebook Photo Tagging

Object Recognition

ETHZ Cows vs Cars Caltech 101 Amazon Flow

3D Reconstruction

Oxford Corridor Berkeley Downtown Google Earth

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Accurate recognition of HD models presents unique challenges

Big data vs small training sets: New theory is needed.

! "#" $ %& & ' ( $ ' )# %( * + ' ( #, - # - ' + ! + . ' ( / , 0 ! ' . $ 12 2 3453 . 367" 89: 34; 35437<4= 1> 351912 ? 5 <@ <? 4; ( 1A347 %47934 ; <? : 7B<7 B3 A3C ? 3D3: 7B3 : 1C C <4 <75844397 C 3D3C ; E <; > <? 4C = 64? 53: <F<? 9; 72 1; 7' 84163<9 5844395? 3; " G 9 < E ? : 3 4<9F? 9F <: : 43; ; ; 619; 143: A= 7B3 ' H6147IG 2 6147 J <9KL %47934L 7B3 A<9K); ; 39? 14 351912 ? ; 7 <C ; 1 ; <? : B3 A3C ? 3D3: 7B<7 7B3 =39 E <; 89: 34D <C 83: <9: 518C : F1 86 A= M N 14 M O 657" PG: 1 917 43F<4: 7B3 : 1C C <4 <; 89: 34D <C 83: <7 7B? ; 61? 97 <F<? 9; 7 7B3 =39L P B3 ; <? : " %9 7B3 17B34 B<9: L %47934 ; <? : 7B<7 B3 7B18FB7 7B<7 P7B3 =39 ? ; ; 7? C C < C ? 77C 3 A? 7 89: 34D<C 83: LP <9: P518C : F1 86 <917B34 M N 14 M O 657"P G 9 <: : ? 7? 19L %47934L E B1 ; <? : B3 E <; ; 63<K? 9F 634; 19<C C =L ; <? : B3 7B18FB7 7B<7 7B3 : 1C C <4 <F<? 9; 7 2 1; 7 ' 84163<9 5844395? 3; E <; P> <? 4C = 64? 53: "P %47934 ; <? : B? ; <9<C =; ? ; 1> 7B3 D <4? 18; 3H5B<9F3 4<73 D<C 83; E <; A<; 3: 19 ; 85B 351912 ? 5 6<47? 58C <4; <; E <F3 4<73 : ? @ 34397? <7? 19; " %47934 ; <? : 7B343 B<: A339 C ? 77C 3 ? 2 6<57 19 ! "#" 74<: 3 : 3Q5? 7 A= 7B3 : 35C ? 93 1>7B3 : 1C C <4 A35<8; 3 <7 7B3 7? 2 3 1>7B3 R C <S< , 5514: L 7B3 : 1C C <4 E <; 3H7432 3C = 1D34D <C 83: <9: 7B<7 7B3 Q4; 7 M O 657 : 35C ? 93 B<: C ? 77C 3 ? 2 6<57" T 3 ; <? : 7B343 E 343 ? 9: ? 5<7? 19; 91E 7B<7 7B3 74<: 3 : 3Q5? 7 E <; A3F? 99? 9F 71 C 3D3C1@ " * 849? 9F 71 J 4<S? C <9: & 3H? 51L %47934 2 <: 3 ? 7 5C 3<4 7B<7 ? 7 E 18C : A3 <C 2 1; 7 ? 2 61; ; ? AC 3 > 14 7B1; 3 518974? 3; 71 3<49 3918FB > 143? F9 3H5B<9F3 71 6<= 7B3 ; 34D? 53 19 7B3? 4 : 3A7; " T 3 ; <? : 7B3 A3; 7 E <= 71 : 3<CE ? 7B 7B? ; E <; 71 8; 3 7B3 61C ? 5? 3; 187C ? 93: ? 9 * 43<; 84= #35437<4= U<2 3; J <K34); : 3A7 ? 9? 7? <7? D3"

50B webpages average 1B voxels average 1M pixels http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Accurate recognition of HD models presents unique challenges

Big data vs small training sets: New theory is needed.

! "#" $ %& & ' ( $ ' )# %( * + ' ( #, - # - ' + ! + . ' ( / , 0 ! ' . $ 12 2 3453 . 367" 89: 34; 35437<4= 1> 351912 ? 5 <@ <? 4; ( 1A347 %47934 ; <? : 7B<7 B3 A3C ? 3D3: 7B3 : 1C C <4 <75844397 C 3D3C ; E <; > <? 4C = 64? 53: <F<? 9; 72 1; 7' 84163<9 5844395? 3; " G 9 < E ? : 3 4<9F? 9F <: : 43; ; ; 619; 143: A= 7B3 ' H6147IG 2 6147 J <9KL %47934L 7B3 A<9K); ; 39? 14 351912 ? ; 7 <C ; 1 ; <? : B3 A3C ? 3D3: 7B<7 7B3 =39 E <; 89: 34D <C 83: <9: 518C : F1 86 A= M N 14 M O 657" PG: 1 917 43F<4: 7B3 : 1C C <4 <; 89: 34D <C 83: <7 7B? ; 61? 97 <F<? 9; 7 7B3 =39L P B3 ; <? : " %9 7B3 17B34 B<9: L %47934 ; <? : 7B<7 B3 7B18FB7 7B<7 P7B3 =39 ? ; ; 7? C C < C ? 77C 3 A? 7 89: 34D<C 83: LP <9: P518C : F1 86 <917B34 M N 14 M O 657"P G 9 <: : ? 7? 19L %47934L E B1 ; <? : B3 E <; ; 63<K? 9F 634; 19<C C =L ; <? : B3 7B18FB7 7B<7 7B3 : 1C C <4 <F<? 9; 7 2 1; 7 ' 84163<9 5844395? 3; E <; P> <? 4C = 64? 53: "P %47934 ; <? : B? ; <9<C =; ? ; 1> 7B3 D <4? 18; 3H5B<9F3 4<73 D<C 83; E <; A<; 3: 19 ; 85B 351912 ? 5 6<47? 58C <4; <; E <F3 4<73 : ? @ 34397? <7? 19; " %47934 ; <? : 7B343 B<: A339 C ? 77C 3 ? 2 6<57 19 ! "#" 74<: 3 : 3Q5? 7 A= 7B3 : 35C ? 93 1>7B3 : 1C C <4 A35<8; 3 <7 7B3 7? 2 3 1>7B3 R C <S< , 5514: L 7B3 : 1C C <4 E <; 3H7432 3C = 1D34D <C 83: <9: 7B<7 7B3 Q4; 7 M O 657 : 35C ? 93 B<: C ? 77C 3 ? 2 6<57" T 3 ; <? : 7B343 E 343 ? 9: ? 5<7? 19; 91E 7B<7 7B3 74<: 3 : 3Q5? 7 E <; A3F? 99? 9F 71 C 3D3C1@ " * 849? 9F 71 J 4<S? C <9: & 3H? 51L %47934 2 <: 3 ? 7 5C 3<4 7B<7 ? 7 E 18C : A3 <C 2 1; 7 ? 2 61; ; ? AC 3 > 14 7B1; 3 518974? 3; 71 3<49 3918FB > 143? F9 3H5B<9F3 71 6<= 7B3 ; 34D? 53 19 7B3? 4 : 3A7; " T 3 ; <? : 7B3 A3; 7 E <= 71 : 3<CE ? 7B 7B? ; E <; 71 8; 3 7B3 61C ? 5? 3; 187C ? 93: ? 9 * 43<; 84= #35437<4= U<2 3; J <K34); : 3A7 ? 9? 7? <7? D3"

50B webpages average 1B voxels average 1M pixels

From desktop to mobile computing: There’s an app for everything!

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Accurate recognition of HD models presents unique challenges

Big data vs small training sets: New theory is needed.

! "#" $ %& & ' ( $ ' )# %( * + ' ( #, - # - ' + ! + . ' ( / , 0 ! ' . $ 12 2 3453 . 367" 89: 34; 35437<4= 1> 351912 ? 5 <@ <? 4; ( 1A347 %47934 ; <? : 7B<7 B3 A3C ? 3D3: 7B3 : 1C C <4 <75844397 C 3D3C ; E <; > <? 4C = 64? 53: <F<? 9; 72 1; 7' 84163<9 5844395? 3; " G 9 < E ? : 3 4<9F? 9F <: : 43; ; ; 619; 143: A= 7B3 ' H6147IG 2 6147 J <9KL %47934L 7B3 A<9K); ; 39? 14 351912 ? ; 7 <C ; 1 ; <? : B3 A3C ? 3D3: 7B<7 7B3 =39 E <; 89: 34D <C 83: <9: 518C : F1 86 A= M N 14 M O 657" PG: 1 917 43F<4: 7B3 : 1C C <4 <; 89: 34D <C 83: <7 7B? ; 61? 97 <F<? 9; 7 7B3 =39L P B3 ; <? : " %9 7B3 17B34 B<9: L %47934 ; <? : 7B<7 B3 7B18FB7 7B<7 P7B3 =39 ? ; ; 7? C C < C ? 77C 3 A? 7 89: 34D<C 83: LP <9: P518C : F1 86 <917B34 M N 14 M O 657"P G 9 <: : ? 7? 19L %47934L E B1 ; <? : B3 E <; ; 63<K? 9F 634; 19<C C =L ; <? : B3 7B18FB7 7B<7 7B3 : 1C C <4 <F<? 9; 7 2 1; 7 ' 84163<9 5844395? 3; E <; P> <? 4C = 64? 53: "P %47934 ; <? : B? ; <9<C =; ? ; 1> 7B3 D <4? 18; 3H5B<9F3 4<73 D<C 83; E <; A<; 3: 19 ; 85B 351912 ? 5 6<47? 58C <4; <; E <F3 4<73 : ? @ 34397? <7? 19; " %47934 ; <? : 7B343 B<: A339 C ? 77C 3 ? 2 6<57 19 ! "#" 74<: 3 : 3Q5? 7 A= 7B3 : 35C ? 93 1>7B3 : 1C C <4 A35<8; 3 <7 7B3 7? 2 3 1>7B3 R C <S< , 5514: L 7B3 : 1C C <4 E <; 3H7432 3C = 1D34D <C 83: <9: 7B<7 7B3 Q4; 7 M O 657 : 35C ? 93 B<: C ? 77C 3 ? 2 6<57" T 3 ; <? : 7B343 E 343 ? 9: ? 5<7? 19; 91E 7B<7 7B3 74<: 3 : 3Q5? 7 E <; A3F? 99? 9F 71 C 3D3C1@ " * 849? 9F 71 J 4<S? C <9: & 3H? 51L %47934 2 <: 3 ? 7 5C 3<4 7B<7 ? 7 E 18C : A3 <C 2 1; 7 ? 2 61; ; ? AC 3 > 14 7B1; 3 518974? 3; 71 3<49 3918FB > 143? F9 3H5B<9F3 71 6<= 7B3 ; 34D? 53 19 7B3? 4 : 3A7; " T 3 ; <? : 7B3 A3; 7 E <= 71 : 3<CE ? 7B 7B? ; E <; 71 8; 3 7B3 61C ? 5? 3; 187C ? 93: ? 9 * 43<; 84= #35437<4= U<2 3; J <K34); : 3A7 ? 9? 7? <7? D3"

50B webpages average 1B voxels average 1M pixels

From desktop to mobile computing: There’s an app for everything! Real-time performance calls for fast, distributed computing solutions.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Outline

Main Message The rich phenomena of sparse representation in HD data can provide novel pattern recognition solutions and successfully mitigate the curse of dimensionality and small sample set problems.

1

Accuracy: Sparse Representation-based Pattern Recognition.

2

Speed: Accelerated Sparse Optimization in Real Time.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Outline

Main Message The rich phenomena of sparse representation in HD data can provide novel pattern recognition solutions and successfully mitigate the curse of dimensionality and small sample set problems.

1

Accuracy: Sparse Representation-based Pattern Recognition.

2

Speed: Accelerated Sparse Optimization in Real Time.

3

Mobility: Wearable Augmented Reality – The Atheer Solution.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Face Recognition via Sparse Representation

1

Face-subspace model [Belhumeur et al. ’97, Basri & Jacobs ’03] Assume b belongs to Class i from K classes. b = αi,1ai,1 + αi,2ai,2 + · · · + αi,n1ai,ni , = Aiαi. Dimension of b can be thousands, but the subspace model is a few tens.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Face Recognition via Sparse Representation

1

Face-subspace model [Belhumeur et al. ’97, Basri & Jacobs ’03] Assume b belongs to Class i from K classes. b = αi,1ai,1 + αi,2ai,2 + · · · + αi,n1ai,ni , = Aiαi. Dimension of b can be thousands, but the subspace model is a few tens.

2

Nevertheless, Class i is the unknown label we need to solve: Sparse representation b = [A1, A2, · · · , AK ]  

α1 α2

. . .

αK

  = Ax.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Face Recognition via Sparse Representation

1

Face-subspace model [Belhumeur et al. ’97, Basri & Jacobs ’03] Assume b belongs to Class i from K classes. b = αi,1ai,1 + αi,2ai,2 + · · · + αi,n1ai,ni , = Aiαi. Dimension of b can be thousands, but the subspace model is a few tens.

2

Nevertheless, Class i is the unknown label we need to solve: Sparse representation b = [A1, A2, · · · , AK ]  

α1 α2

. . .

αK

  = Ax.

3

x∗ = [ 0 ··· 0 αT

i

0 ··· 0 ]T ∈ Rn.

Sparse representation x∗ encodes membership through its nonzero coefficients!

Reference: Wright, AY, Sastry, Ma, Robust face recognition via sparse representation. IEEE PAMI, 2009.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Image Occlusion, Corruption, and Disguise

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Image Occlusion, Corruption, and Disguise

1

Sparse representation + sparse error b = Ax + e

2

Cross-and-bouquet model [Wright et al. ’09, ’11] min

x,e x1 + e1

• subj. to

b = Ax + e When size of A grows proportionally with the sparsity in x, asymptotically CAB can correct 100% noise in e.

Reference: Wright and Ma, Dense Error Correction via ℓ1 Minimization, IEEE Trans. IT, 2011.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Performance on the YaleB database

Reference: Wright, AY, Sastry, Ma, Robust face recognition via sparse representation. IEEE PAMI, 2009. Top 25 IEEExplore Downloads since 2010. 1800+ citations on Google.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Face Alignment Problem: Misalignment violates linear subspace model

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Face Alignment Problem: Misalignment violates linear subspace model

1

Find an image transformation τ (2-D function that transforms image coordinates) min e1

• subj. to

b ◦ τi = Aix + e per each class Ai, while minimize the alignment error e.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Face Alignment Problem: Misalignment violates linear subspace model

1

Find an image transformation τ (2-D function that transforms image coordinates) min e1

• subj. to

b ◦ τi = Aix + e per each class Ai, while minimize the alignment error e.

2

Iterative linear approximation [Lucas & Kanade ’81, Hager & Belhumeur ’98]: b ◦ τi + ∇τ(b ◦ τi) · ∆τi ≈ Aix + e. Convert to a linear sparse optimization constraint.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Face Alignment Problem: Misalignment violates linear subspace model

1

Find an image transformation τ (2-D function that transforms image coordinates) min e1

• subj. to

b ◦ τi = Aix + e per each class Ai, while minimize the alignment error e.

2

Iterative linear approximation [Lucas & Kanade ’81, Hager & Belhumeur ’98]: b ◦ τi + ∇τ(b ◦ τi) · ∆τi ≈ Aix + e. Convert to a linear sparse optimization constraint.

3

Compensated training images are fed back to the sparse representation model: b = [τ −1

1

(A1), · · · , τ −1

K (AK )]x + e.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Region of Convergence for 2-D Alignment

Using affine transformations, ℓ1-min approach can compensate 3-D rotation up to 30 degree and 2-D translation up to 10 pixels.

References: Ganesh, Ma, Wagner, Wright, AY, Zhou, Face recognition by sparse representation, Cambridge University Press, 2011. Ma, AY, Wright, Wagner, Recognition via High-Dimensional Data Classification, US Patent, 2013.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

A Photo Booth for Face Recognition

Most existing face recognition solutions require large numbers of training images: A = [A1, . . . , AC ] Question: What if subjects of interest only have limited training images, e.g., one image per class?

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Single-Sample Face Recognition via Sparse Illumination Transfer

Alignment Stage: Illumination Compensation + Misalignment + Pixel Corruption

Single Training Image Test image align with pixel corruption

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Single-Sample Face Recognition via Sparse Illumination Transfer

Alignment Stage: Illumination Compensation + Misalignment + Pixel Corruption

Single Training Image Test image align with pixel corruption

Sparse Illumination Transfer (SIT) via Transfer Learning Approach Given new face illumination examples from additional irrelevant subjects c1, c2, . . . , cn: C = [· · · , ci − cj, · · · ]i=j,(i,j) belong to the same subject C is called a SIT dictionary independent of the training dictionary A, constructed offline.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion 1

Alignment Stage: ˆ τi = arg minτi ,xi ,yi ,e yi1 + e1,

• subj. to

b ◦ τi − aixi = Cyi + e

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion 1

Alignment Stage: ˆ τi = arg minτi ,xi ,yi ,e yi1 + e1,

• subj. to

b ◦ τi − aixi = Cyi + e

2

Recognition Stage: Illumination and Pose Transfer ˜ ai . = (aixi + Cyi) ◦ τ −1

i

.

Figure : Left: Testing b. Mid Left: Training ai. Mid Right: Cy i. Right: Warped ˜ ai.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Results and Comparison: Single-Sample Face Recognition

Figure : SIT on YaleB dataset, train on Multi-PIE Session I, test on Session I & II (166 subjects).

1

SIT improves existing face recognition solutions (alignment provided manually)

Method Session 1 (%) Session 2 (%) SRC ’09 88.0 53.6 ESRC ’12 89.6 56.6 SRC + SIT 91.6 59.0 ESRC + SIT 93.6 59.3

2

Full pipeline (alignment + recognition) with added pixel corruption

Corruption 10% 20% 30% 40% DSRC ’09 32.9% 31.7% 28.9% 24.1% MRR ’12 24.9% 14.5% 11.7% 9.2% SIT 74.3% 70.3% 67.1% 55.8%

References: Zhuang, AY, Zhang, Sastry, Ma, Single-Sample Face Recognition via Sparse Illumination Transfer, CVPR, 2013. US patent application filed by UC Berkeley, 2013.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Extension: Transform Invariant Low-rank Texture (TILT)

Objective function [Zhang et al. ’10] min

A,E,τ A∗ + λE1

• subj. to

I ◦ τ = A + E, where A is low-rank and E is sparse, τ parametrizes an image transformation. More Examples

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Extension: Sparse PCA

Reference: Naikal, AY, Sastry, Informative feature selection for object recognition via SPCA, ICCV (most remembered poster), 2011.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

More Extensions:

Target Tracking [Mei & Ling 2009, Liu et al. 2010, Li et al. 2011] Superresolution [Yang et al. 2009] Sparse dictionary learning [Aharon et al. 2006, Mairal et al. 2008, Duarte-Carvajalino & Sapiro 2009]

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Solving sparse optimization in HD is still very expensive

General linear-programming toolboxes do exist: cvx, SparseLab. However, standard interior-point methods are very expensive in HD space. Standard Form minx 1T x

• subj. to

Ax = b x ≥ 0

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Solving sparse optimization in HD is still very expensive

General linear-programming toolboxes do exist: cvx, SparseLab. However, standard interior-point methods are very expensive in HD space. Standard Form minx 1T x

• subj. to

Ax = b x ≥ 0 Questions:

1

Can standard methods be accelerated?

2

Can the algorithms be parallelized on modern multicore platforms?

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Accelerating ℓ1-min

1

Primal-Dual Interior-Point

Log-Barrier [Frisch ’55, Karmarkar ’84, Megiddo ’89, Monteiro-Adler ’89, Kojima-Megiddo-Mizuno ’93]

2

Homotopy

Homotopy [Osborne-Presnell-Turlach ’00, Malioutov-Cetin-Willsky ’05, Donoho-Tsaig ’06] Polytope Faces Pursuit (PFP) [Plumbley ’06] Least Angle Regression (LARS) [Efron-Hastie-Johnstone-Tibshirani ’04]

3

Gradient Projection

Gradient Projection Sparse Representation (GPSR) [Figueiredo-Nowak-Wright ’07] Truncated Newton Interior-Point Method (TNIPM) [Kim-Koh-Lustig-Boyd-Gorinevsky ’07]

4

Iterative Thresholding

Soft Thresholding [Donoho ’95] Sparse Reconstruction by Separable Approximation (SpaRSA) [Wright-Nowak-Figueiredo ’08]

5

Proximal Gradient [Nesterov ’83, Nesterov ’07]

FISTA [Beck-Teboulle ’09] Nesterov’s Method (NESTA) [Becker-Bobin-Cand´ es ’09]

6

Augmented Lagrangian Methods [Yang-Zhang ’09, AY et al ’10]

Bergman [Yin et al. ’08] YALL1 [Yang-Zhang ’09] SALSA [Figueiredo et al. ’09] Primal ALM, Dual ALM [AY et al ’10]

Reference: AY, Ganesh, Ma, Sastry, A review of fast ℓ1-minimization algorithms for robust face recognition. ICIP, 2010.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Augmented Lagrangian Method (ALM)

ℓ1-Min: x∗ = arg min x1

• subj. to

b = Ax (adding a quadratic penalty term for the equality constraint) Lµ(x) = x1 + µ 2 b − Ax2

2

• subj. to

b = Ax. Augmented Lagrange Function: Lµ(x, y) = x1 + y, b − Ax + µ 2 b − Ax2

2,

where y is the Lagrange multipliers for the constraint b = Ax. Theorem: Convergence Guarantee of ALM [Bertsekas ’03] When optimize Lµ(x, y) w.r.t. a sequence µk → ∞, and {yk} is bounded, then the limit point of {xk} is the global minimum of the original problem, namely, ℓ1-min. An alternating direction method for optimization.

1

Fix y, update xk+1: soft-thresholding.

2

Fix x, update y k+1: method of multipliers.

3

µ → ∞, repeat (1) and (2).

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Simulation: Speed of ℓ1-Min Solvers

Table : x0 ∈ R1000, x00 = 200, b = Ax0 ∈ R600.

Algorithm Estimate Runtime Speedup Interior Point 63 s Homotopy 1.7 s 40 X ALM 0.16 s 400 X

Reference (MATLAB implementation available on our website): AY, Zhou, Ganesh, Ma, Sastry, Fast ℓ1-minimization algorithms for robust face recognition. IEEE TIP, 2013.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Problem Parallelism of Face Alignment

Figure : Problem Parallelism (10 X acceleration) 50 100 150 200 250 10 20 30 40 Number of training users Elapsed time (s) GPU CPU, library threading CPU, manual threading

Reference (Parallel C/CUDA-C implementation available upon request): Shia, AY, Sastry, Ma, Fast ℓ1-minimization and parallelization for face recognition, Asilomar Conf, 2011.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

How to position great technologies to make a fantastic product?

Past IT revolutions have not engaged the vision community much

PC OS Internet Search Smartphone Personalized Media

False predictions diminish the credibility of AI (lessons from 1960s)

“The problem of computer vision can be solved as a summer project.” “Simple AI algorithms will provide natural language translators in a couple of years.”

• - False. A basic image edge detector was not invented until 1986 by Canny.
• - False. Foundation of machine learning theory was not established until late 1980s.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

More matured algorithms and successful applications give us optimism

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

The Next Wave: Big Data on Mobile Platforms?

Some Statistics

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Figure : North America Mobile Traffic Forecast

Good: By 2016, mobile Internet traffic will be 12 X larger than that in 2011. Video traffic will account for 2/3 of the total traffic on mobile platforms. Bad: There is a (free) app for everything!

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Let’s just think about shopping

The “Minority Report” shopping experience Why Cruise’s character doesn’t like the virtual shopping guide?

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Let’s just think about shopping

The “Minority Report” shopping experience Why Cruise’s character doesn’t like the virtual shopping guide?

1

Mistaken identity

2

Wrong ads at the wrong time

3

Targeted ads displayed in public embarrass customers

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

The Atheer Vision

1

Face recognition in the wild: Robust to facial disguise, glasses (and iris replacement)

2

Wearable AR: Virtual ads and shopping on the go

3

Natural human-computer interaction: Right information at your fingertip

4

Built-in privacy and security: Personalized information just for you

Figure : Atheer Wearable AR Concept

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Atheer Launch Event at All Things Digital 11

Please visit us at atheerlabs.com and on Facebook and Twitter!

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Take-Home Messages

High-Dimensional Pattern Recognition

1

High dimensionality of data could be both curse and blessing for pattern recognition.

2

Rich phenomena of sparsity enable new solutions to compensate the culprits of corruption,

• cclusion, and distortion in images/video.

3

In this process, the proper use of accurate data structure of individual problems can help in finding the optimal solution.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Take-Home Messages

High-Dimensional Pattern Recognition

1

High dimensionality of data could be both curse and blessing for pattern recognition.

2

Rich phenomena of sparsity enable new solutions to compensate the culprits of corruption,

• cclusion, and distortion in images/video.

3

In this process, the proper use of accurate data structure of individual problems can help in finding the optimal solution. Industrial Technology Transfer

1

After 50 years, computer vision finally stands on solid theoretical foundation.

2

With the growth of multimedia big data, computer vision will play an important role in defining revolutionary products.

3

Let’s work on something interesting together!

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Acknowledgments

UC Berkeley

Dr S. Sastry, N. Naikal, V. Shia

Microsoft Research Asia

Dr Y. Ma

Atheer Inc.

• Univ. Illinois
• A. Ganesh, H. Mobahi, A. Wagner, Z. Zhou

Columbia

Dr J. Wright

Publications

Wright, AY, Ganesh, Sastry, Ma. “Robust face recognition via sparse representation.” IEEE PAMI, 2009. Ganesh, Ma, Wagner, Wright, AY, Zhou. “Face recognition by sparse representation.” Cambridge University Press, 2011. AY, Ganesh, Zhou, Sastry, Ma.“A review of fast ℓ1-minimization algorithms in robust face recognition.” arXiv, 2010. Wagner, Shia, AY, Sastry, Ma. “Fast ℓ1-minimization and parallelization for face recognition.” Asilomar, 2011. Mobahi, Zhou, AY, Ma. “Holistic 3D reconstruction of urban structures from low-rank textures.” ICCV Workshop, 2011. Naikal, AY, Sastry. “Informative feature selection for objection recognition via Sparse PCA.” ICCV, 2011. Slaughter, AY, Bagwell, Checkles, Sentis, Vishwanath. “Sparse online low-rank projection and outlier rejection (SOLO) for 3-D rigid-body motion registration.” ICRA, 2012. Zhuang, AY, Zhang, Sastry, Ma, Single-Sample Face Recognition via Sparse Illumination Transfer, CVPR, 2013.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion

Sparse Representation vs Dense Representation in Classification

Dense Representation performance comparable if the samples are well conditioned xℓ2 = arg min x2

• subj. to

b = Ax Sparse representation is more discriminative when samples are highly coherent

Reference: Zhang et al, Sparse representation or collaborative representation, ICCV, 2011.

http://www.eecs.berkeley.edu/~yang High-Dimensional Pattern Recognition via Sparse Representation