Distributed Sensing and Perception via Sparse Representation Allen - - PowerPoint PPT Presentation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Distributed Sensing and Perception via Sparse Representation

Allen Y. Yang Department of EECS, UCB yang@eecs.berkeley.edu University of Texas, Austin, 2011

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Distributed Sensing and Perception

Centralized Perception Up: powerful processors Up: unlimited memory Up: unlimited bandwidth Down: single modality Distributed Perception Down: mobile processors Down: limited onboard memory Down: band-limited communications Up: distributed, multi-modality

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Distributed Sensing and Perception

Centralized Perception Up: powerful processors Up: unlimited memory Up: unlimited bandwidth Down: single modality Distributed Perception Down: mobile processors Down: limited onboard memory Down: band-limited communications Up: distributed, multi-modality When the sensing resources are limited or scarce:

1

What is the optimal strategy to deploy these agents?

2

How to effectively take measurements of the events?

3

How to properly tally the local observations to reach a global consensus? An intelligent system over a sensor network shall perform better than the sum of its parts?

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Challenges

1

Making real-time decisions on mobile devices is difficult.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Challenges

1

Making real-time decisions on mobile devices is difficult.

2

Applications demand extremely high accuracy: 99% Precision, 99% Recall?

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Challenges

1

Making real-time decisions on mobile devices is difficult.

2

Applications demand extremely high accuracy: 99% Precision, 99% Recall?

3

Scenarios demand the ability to reconstruct 3-D models.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Outline

1

Robust Face Recognition

A sparse representation framework via ℓ1-min. x∗ = arg min

x

x1

• subj. to b = Ax.

Accelerate ℓ1-min algorithms towards a semi-real time face recognition system.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion 2

Informative Feature Selection for Object Recognition

Informative feature selection via Sparse PCA x∗ = arg max

x

xT ΣAx

• subj. to

x2 = 1, x1 ≤ k. Accelerate Sparse PCA algorithms.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion 3

Reconstruct large-scale 3-D objects by large-baseline feature matching

Extract a new class of low-rank texture regions using Robust PCA A∗ = arg min

A,E,τ A∗ + λE1

• subj. to

I ◦ τ = A + E. Complete pipeline from low-rank texture in single views to 3-D model in multiple views.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Face Recognition

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Classification via Sparse Representation

1

Face-subspace model [Belhumeur et al. ’97, Basri & Jacobs ’03] Assume b belongs to Class i in K classes. b = αi,1vi,1 + αi,2vi,2 + · · · + αi,n1vi,ni , = Aiαi.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Classification via Sparse Representation

1

Face-subspace model [Belhumeur et al. ’97, Basri & Jacobs ’03] Assume b belongs to Class i in K classes. b = αi,1vi,1 + αi,2vi,2 + · · · + αi,n1vi,ni , = Aiαi.

2

Nevertheless, Class i is the unknown label we need to solve: Sparse representation b = [A1, A2, · · · , AK ] 2 4

α1 α2

. . .

αK

3 5 = Ax.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Classification via Sparse Representation

1

Face-subspace model [Belhumeur et al. ’97, Basri & Jacobs ’03] Assume b belongs to Class i in K classes. b = αi,1vi,1 + αi,2vi,2 + · · · + αi,n1vi,ni , = Aiαi.

2

Nevertheless, Class i is the unknown label we need to solve: Sparse representation b = [A1, A2, · · · , AK ] 2 4

α1 α2

. . .

αK

3 5 = Ax.

3

x∗ = [ 0 ··· 0 αT

i

0 ··· 0 ]T ∈ Rn.

Sparse representation x∗ encodes membership!

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Image Corruption

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Image Corruption

1

Sparse representation + sparse error b = Ax + e

2

Cross-and-bouquet model [Wright et al. ’09, ’10] b = `A | I´ „x e « = Bw

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Image Corruption

1

Sparse representation + sparse error b = Ax + e

2

Cross-and-bouquet model [Wright et al. ’09, ’10] b = `A | I´ „x e « = Bw When size of A grows proportionally with the sparsity in x, asymptotically CAB can correct 100% noise.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Performance on the AR database

(ℓ1-min): min x1 + e1

• subj. to

b = Ax + e

Reference: AY, et al. Robust face recognition via sparse representation. IEEE PAMI, 2009.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Question: How to effectively estimate HD sparse signals?

“Black gold” age [Claerbout & Muir 1973, Taylor, Banks & McCoy 1979]

Figure: Deconvolution of spike train.

Basis pursuit / ℓ1-minimization [Chen-Donoho 1999]: (P1) : x∗ = arg min x1, subject to b = Ax The Lasso (least absolute shrinkage and selection operator) [Tibshirani 1996] (P1,2) : x∗ = arg min b − Ax2, subject to x1 ≤ k

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

ℓ1-Minimization via Linear Programming

Using interior-point methods [Karmarkar ’84] Log-Barrier: min

x

1T x − µ

n

X

i=1

log xi,

• subj. to Ax = b, x ≥ 0.

(1) Using the Karush-Kuhn-Tucker (KKT) conditions 1 − µX −11 − AT y = 0. (2) where x ≥ 0 are the primal variables, and y are the dual variables. Update by solving a linear system with O(n3) [Monteiro & Adler ’89] Z (k)∆x + X (k)∆z = ˆ µ1 − X (k)z(k), A∆x = 0, AT ∆y + ∆z = 0, (3)

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

ℓ1-Min Literature

1

Primal-Dual Interior-Point Methods

Log-Barrier [Frisch ’55, Karmarkar ’84, Megiddo ’89, Monteiro-Adler ’89, Kojima-Megiddo-Mizuno ’93]

2

Homotopy Methods:

Homotopy [Osborne-Presnell-Turlach ’00, Malioutov-Cetin-Willsky ’05, Donoho-Tsaig ’06] Polytope Faces Pursuit (PFP) [Plumbley ’06] Least Angle Regression (LARS) [Efron-Hastie-Johnstone-Tibshirani ’04]

3

Gradient Projection Methods

Gradient Projection Sparse Representation (GPSR) [Figueiredo-Nowak-Wright ’07] Truncated Newton Interior-Point Method (TNIPM) [Kim-Koh-Lustig-Boyd-Gorinevsky ’07]

4

Iterative Thresholding Methods

Soft Thresholding [Donoho ’95] Sparse Reconstruction by Separable Approximation (SpaRSA) [Wright-Nowak-Figueiredo ’08]

5

Proximal Gradient Methods [Nesterov ’83, Nesterov ’07]

FISTA [Beck-Teboulle ’09] Nesterov’s Method (NESTA) [Becker-Bobin-Cand´ es ’09]

6

Augmented Lagrangian Methods [Yang-Zhang ’09, AY et al ’10]

Bergman [Yin et al. ’08] YALL1 [Yang-Zhang ’09] SALSA [Figueiredo et al. ’09] Primal ALM, Dual ALM [AY et al ’10]

Reference: AY, et al., A review of fast ℓ1-minimization algorithms for robust face recognition. ICIP, 2010.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

New algorithms significantly improves the speed of ℓ1-min

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Augmented Lagrangian Method (ALM)

ℓ1-Min: x∗ = arg min x1

• subj. to

b = Ax (adding a penalty term for the equality constraint) Lµ(x) = x1 + µ 2 b − Ax2

2

• subj. to

b = Ax.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Augmented Lagrangian Method (ALM)

ℓ1-Min: x∗ = arg min x1

• subj. to

b = Ax (adding a penalty term for the equality constraint) Lµ(x) = x1 + µ 2 b − Ax2

2

• subj. to

b = Ax. Augmented Lagrange Function: Lµ(x, y) = x1 + y, b − Ax + µ 2 b − Ax2

2,

where y is the Lagrange multipliers for the constraint b = Ax.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Convergence of ALM [Hestenes ’69, Powell ’69, Bertsekas ’03]

Lµ(x, y) = x1 + y, b − Ax + µ 2 b − Ax2

2,

1

When y close to y∗, by Lagrange Multiplier Theory, arg min

x

Lµ(x, y∗) = arg min

x

Lµ(x).

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Convergence of ALM [Hestenes ’69, Powell ’69, Bertsekas ’03]

Lµ(x, y) = x1 + y, b − Ax + µ 2 b − Ax2

2,

1

When y close to y∗, by Lagrange Multiplier Theory, arg min

x

Lµ(x, y∗) = arg min

x

Lµ(x).

2

When µ is very large, high cost of infeasibility implies Lµ(x, y) ≈ (P1).

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Convergence of ALM [Hestenes ’69, Powell ’69, Bertsekas ’03]

Lµ(x, y) = x1 + y, b − Ax + µ 2 b − Ax2

2,

1

When y close to y∗, by Lagrange Multiplier Theory, arg min

x

Lµ(x, y∗) = arg min

x

Lµ(x).

2

When µ is very large, high cost of infeasibility implies Lµ(x, y) ≈ (P1). Theorem: Convergence of ALM [Bertsekas ’03] When optimize Lµ(x, y) w.r.t. a sequence µk → ∞, and {yk} is bounded, then the limit point of {xk} is the global minimum of the original problem, namely, ℓ1-min.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Minimize Augmented Lagrangian

Update yk+1: The Method of Multipliers [Rockafellar ’73] Assume (xk, µk) fixed, yk+1 = yk + µk∇yLµk (xk, yk) with complexity O(dn).

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Minimize Augmented Lagrangian

Update yk+1: The Method of Multipliers [Rockafellar ’73] Assume (xk, µk) fixed, yk+1 = yk + µk∇yLµk (xk, yk) with complexity O(dn). Update xk+1: Nesterov’s Method [Nesterov ’07, Becker et al. ’09]

Let f (x) = µ

2 b − Ax2 2 + yk, b − Ax and g(x) = x1:

Lµk (x, yk) = f (x) + g(x) Form a second-order upper bound of Lµk (x, yk) based on two step history (xk, xk−1): zk = α1xk + α2xk−1 Q(x, z) . = f (z) + ∇f (z), x − z + L

2 x − z2 + g(x).

(4) Minimize Q(x, z) via soft-thresholding: soft(x, a) = sgn(x) max(|x| − a, 0)

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Simulation: Speed of ℓ1-Min Solvers

Table: Source signal in 1000-D: sparsity = 200; random projection = 600-D.

Algorithm Estimate Runtime PDIPA 63 s Homotopy 1.7 s ALM 0.16 s

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Simulation: Speed of ℓ1-Min Solvers

Table: Source signal in 1000-D: sparsity = 200; random projection = 600-D.

Algorithm Estimate Runtime PDIPA 63 s Homotopy 1.7 s ALM 0.16 s First-order methods, including ALM, are more suitable for parallel implementation.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Sparse Feature Selection via Sparse PCA

The Problem of Object Recognition 3-D object recognition is a more difficult problem?

1

Every object has 360 degree appearance, top/down views, and different scales.

2

The appearance is determined by shape, surface texture, and illumination.

3

Objects in urban scene appear with cluttered background.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Bag-of-Word Approach

1

Gradient of image pixels expressed as (SIFT) vectors for object features The space of feature vectors is quantized to form a dictionary.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Bag-of-Word Approach

1

Gradient of image pixels expressed as (SIFT) vectors for object features The space of feature vectors is quantized to form a dictionary.

2

Image appearance is captured by the frequency of SIFT features as a histogram.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Informative Feature Selection in Multiple Views

One drawback of BoW approach: uninformative features

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Informative Feature Selection in Multiple Views

One drawback of BoW approach: uninformative features Selecting informative features via geometric approach [Turcot & Lowe ‘09]

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Informative Feature Selection in Multiple Views

One drawback of BoW approach: uninformative features Selecting informative features via geometric approach [Turcot & Lowe ‘09] Geometric approach often is ill-conditioned:

Objects may not be rigid. Correspondence (x1, x2) may be ambiguous. High percentage of outliers. High cost of pairwise matching.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Informative Feature Selection via Sparse PCA

Given a training set Ai = [v1, · · · , vn] in Class i: [Jolliffe et al. ’03, Zou et al. ’06] (SPCA): x1 = arg max

x2=1 xT ΣAi x

• subj. to

x1 ≤ k. (5) A LASSO formulation: min

n

X

i=1

vi − ¯ v −

d

X

j=1

xjaij2 + λ

d

X

j=1

xj1

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Feature Selection Results

(Original, Geometric, Sparse PCA)

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Accelerating Sparse PCA

The semi-definite program [d’Aspremont et al. ’07] Let X = xxT . Since Tr(xT Σx) = Tr(ΣxxT ) and Tr(X) = x2

2

maxx2=1 xT ΣAi x

• subj. to

x1 ≤ k ⇔ maxX Tr(ΣX)

• subj. to

Tr(X) = 1, X0 ≤ k2, Rank(X) = 1, X 0 ⇒ maxX Tr(ΣX) − ρX1

• subj. to

Tr(X) = 1, X 0

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Accelerating Sparse PCA

The semi-definite program [d’Aspremont et al. ’07] Let X = xxT . Since Tr(xT Σx) = Tr(ΣxxT ) and Tr(X) = x2

2

maxx2=1 xT ΣAi x

• subj. to

x1 ≤ k ⇔ maxX Tr(ΣX)

• subj. to

Tr(X) = 1, X0 ≤ k2, Rank(X) = 1, X 0 ⇒ maxX Tr(ΣX) − ρX1

• subj. to

Tr(X) = 1, X 0 SDP can be simplified by SAFE feature elimination [El Ghaoui et al. ’10, Zhang et al. ’11]

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Accelerating Sparse PCA

The semi-definite program [d’Aspremont et al. ’07] Let X = xxT . Since Tr(xT Σx) = Tr(ΣxxT ) and Tr(X) = x2

2

maxx2=1 xT ΣAi x

• subj. to

x1 ≤ k ⇔ maxX Tr(ΣX)

• subj. to

Tr(X) = 1, X0 ≤ k2, Rank(X) = 1, X 0 ⇒ maxX Tr(ΣX) − ρX1

• subj. to

Tr(X) = 1, X 0 SDP can be simplified by SAFE feature elimination [El Ghaoui et al. ’10, Zhang et al. ’11] Acceleration via an approximate dual formulation: [Nesterov ’07] minU∈Sn λmax(Σ + U)

• subj. to

− ρ ≤ Uij ≤ ρ, (Smoothing + ALM) ≈ minU∈Sn{fµ(U) + P

1≤i,j≤n P(Uij, Yij, c)}

fµ(U) = µ log(Tr exp((Σ + U)/µ))

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

ALM as an extension to Nesterov’s Method

(a) Accuracy (b) Run time Reference: Naikal, AY, Sastry. “Informative feature selection for object recognition via Sparse PCA,” (submitted) ICCV 2011.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Object Recognition Performance on BMW Database

Category Baseline SPCA SPCA SfM SfM Rate(%) Rate(%) # Feat Rate(%) # Feat 1 98.61 94.44 35 83.33 2 90.27 91.66 23 90.27 35 3 56.94 66.66 15 58.33 4 70.83 81.94 12 65.27 30 5 77.77 91.66 56 81.94 6 95.83 88.88 23 87.50 7 79.16 93.05 34 86.11 8 77.77 91.66 30 72.22 9 56.94 73.61 45 63.88 11 10 51.38 65.27 9 61.11 11 83.33 76.38 76 69.44 13 12 81.94 83.33 28 70.83 13 62.50 72.22 43 52.77 14 98.61 93.05 20 90.27 37 15 69.44 80.55 36 75.00 16 58.33 79.16 53 80.55 66 17 100.00 90.27 17 84.72 18 98.61 93.05 45 100.00 56 19 97.22 83.33 24 86.11 20 98.61 100 46 95.83 Avg. 80.02 84.51 77.77

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Reconstruct Large-Scale 3D Models in Multiple Views

Traditional approach: Structure-from-Motion xT

2 Fx1 = 0

Small-baseline matching generates dense point clouds

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Reconstruct Large-Scale 3D Models in Multiple Views

Traditional approach: Structure-from-Motion xT

2 Fx1 = 0

Small-baseline matching generates dense point clouds Large-baseline matching is prone to ambiguities

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Introduce a New Class of Robust Features for 3-D Modeling

Degeneracy of Image Features Challenges in large-baseline matching stem from the fact that traditional features (points, lines, patches, etc) do NOT possess any 3-D geometric information alone. Need to extract new features that contain more 3-D information.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Introduce a New Class of Robust Features for 3-D Modeling

Degeneracy of Image Features Challenges in large-baseline matching stem from the fact that traditional features (points, lines, patches, etc) do NOT possess any 3-D geometric information alone. Need to extract new features that contain more 3-D information. Questions:

1

How to properly define large image regions when 3-D structures are rich?

2

How to matching these regions in two camera views?

3

Is the process robust to image corruption and occlusion?

Answer: Image texture sparsity in rank provides a principled solution.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

TILT: Transform Invariant Low-rank Texture [Zhang et al. ’10]

Many object images are low-rank

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

TILT: Transform Invariant Low-rank Texture [Zhang et al. ’10]

Many object images are low-rank Low-rank texture (TILT) features possess rich 3-D information

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Estimating Low-Rank Texture under Transformation and Corruption

Objective function [Zhang et al. ’10] min

A,E,τ rank(A) + λE0

• subj. to

I ◦ τ = A + E, where A is low-rank and E is sparse, τ parametrizes an image transformation.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Estimating Low-Rank Texture under Transformation and Corruption

Objective function [Zhang et al. ’10] min

A,E,τ rank(A) + λE0

• subj. to

I ◦ τ = A + E, where A is low-rank and E is sparse, τ parametrizes an image transformation. An iterative solution using Robust PCA [Candes et al. ’10] min

A,E,∆τ A∗ + λE1

• subj. to

I ◦ τk + ∇I∆τ = A + E, which also has a corresponding ALM formulation [Lin et al. ’10] A∗ + λE1 + Y , I ◦ τk + ∇I∆τ − A − E + µ 2 I ◦ τk + ∇I∆τ − A − E2

F

Its cost is approximately equal to a small constant times the cost of SVD.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

More Examples for Canonical Form of TILT

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Large-Baseline Matching I: Two Features in One Image

Detection of intersection of two TILT patterns in one image: min [A1, A2]∗ + λ[E1, E2]1

• subj. to

[I1 ◦ τ1, I2 ◦ τ2] = [A1, A2] + [E1, E2] More Examples

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Large-Baseline Matching II: One Feature in Two Images

(a) View 1 (b) View 2 (c) Alignment result

Pixel-wise alignment can be achieved up to subpixel accuracy A∗

2 = arg

max

φ=(x,y,u,v)

vec(A1)T vec(A2 ◦ φ) vec(A1) · vec(A2 ◦ φ)

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Complete 3-D Reconstruction Pipeline

Partial 3-D reconstruction from a single image without extracting any points and lines. Full 3-D reconstruction from multiple images. 3-D Reconstruction

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Sparse Representation for Distributed Sensing and Perception

Sparsity-based classification framework for robust face recognition Informative feature selection via Sparse PCA Large-scale 3-D reconstruction via Robust PCA

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Take-Home Message

Nesterov, 2005 The worst-case estimate is valid only for the black-box model of the objective function. Yet in practice, we never meet a pure black box model. We always know something about the structure

• f the underlying objects. And the proper use of the structure of the problem can and does

help in finding the solution.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

References

Acknowledgments UC Berkeley

Dr S. Sastry, N. Naikal

• Univ. Illinois
• A. Ganesh, Z. Zhou, A. Wagner

MSR Asia

Dr Y. Ma, Dr J. Wright

Publications

Wright, Yang, Ganesh, Sastry, Ma. “Robust face recognition via sparse representation,” IEEE PAMI, 2009. Yang, Ganesh, Zhou, Sastry, Ma.“A review of fast ℓ1-minimization algorithms in robust face recognition”, arXiv, 2010. Naikal, Yang, Sastry. “Towards an efficient distributed object recognition system in wireless smart camera networks.” Information Fusion, 2010. Naikal, Yang, Sastry. “Informative feature selection for objection recognition via Sparse PCA.” (submitted) ICCV, 2011. Mobahi, Zhou, Yang, Ma. “Holistic 3D reconstruction of urban structures from low-rank textures.” (submitted) ICCV, 2011.

Funding Support

ARO MURI: Heterogeneous Sensor Networks in Urban Terrains ARL-CTA: Micro Autonomous Systems and Technology

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

ℓ0/ℓ1 Equivalence Relationship

ℓ0-Minimization over an underdetermined system (NP-Hard) x∗ = arg min

x

x0

• subj. to b = Ax.

· 0 simply counts the number of nonzero terms. ℓ1-Minimization (Linear Program) [Candes & Tao 2006, Donoho 2006] x∗ = arg min

x

x1

• subj. to b = Ax.

x1 = |x1| + |x2| + · · · + |xn|.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Feasibility and Uniqueness: ℓ0-Minimization

Spark Condition Spark(A): smallest number of columns that are linearly dependent

1

Example I: Identity matrix I ∈ Rd×d, Spark(A) = d+1;

2

Example II: » 1 1 1 1 – , Spark(A) = 2;

3

Example III: Random matrix [v1, v2, · · · , vn] ∈ Rd×n, Spark(A) = d+1 (with high probability);

Sparse signal x can be uniquely recovered by ℓ0-min if x0 < Spark(A) 2 Proof.

1

Suppose x1 = x2 both satisfy the spark condition, and b = Ax1, b = Ax2.

2

A(x1 − x2) . = Ay = b − b = 0.

3

But y0 < Spark(A)

2

+ Spark(A)

2

= Spark(A). Contradiction.

Estimating Spark(A) is as expensive as ℓ0-min itself!

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Feasibility and Uniqueness: ℓ1-Minimization

k-Neighborliness Condition Define cross polytope C and quotient polytope P such that P = AC. x is k-sparse ⇔ x lie in a unique (k − 1)-face of C. Necessary and Sufficient:

1

If the (k − 1)-face where x lies maps to a face of P, then ℓ1/ℓ0 holds for this specific x.

2

If all (k − 1)-faces of C map to the faces of P on the boundary, ℓ1/ℓ0 holds for all k-sparse signals.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation

Introduction Sparsity-based Classification Sparse Feature Selection Low-Rank Texture Conclusion

Face Alignment

Seek a 2-D transformation b ◦ τi = Aix + e. (6) Although x1 is no longer penalized, the problem becomes nonlinear. Local linear approximation: b ◦ τi + ∇τ(b ◦ τi) · ∆τi ≈ Aix + e. (7) Convert to a linear equation: Enforce sparsity in e. min e1

• subj. to

b ◦ τ (k)

i

= [Ai, −J(k)

i

] » x ∆τi – + e. (8)

Reference: Wagner, et al. Towards a Practical Face Recognition System: Robust Registration and Illumination via Sparse Representation. CVPR, 2009.

http://www.eecs.berkeley.edu/~yang Distributed Sensing and Perception via Sparse Representation