Near-Optimal Joint Object Matching via Convex Relaxation Yuxin - - PowerPoint PPT Presentation

▶
near optimal joint object matching via convex relaxation
SMART_READER_LITE
LIVE PREVIEW

Near-Optimal Joint Object Matching via Convex Relaxation Yuxin - - PowerPoint PPT Presentation

Feb 23, 2024 429 likes •1.09k views

October 22, 2014 Near-Optimal Joint Object Matching via Convex Relaxation Yuxin Chen, Stanford University Joint Work with Qixing Huang (TTIC), Leonidas Guibas (Stanford) Page 1 Assembling Fractured Pieces Computer Assembly (Fig. credit:

slide-1
SLIDE 1

October 22, 2014

Near-Optimal Joint Object Matching via Convex Relaxation

Yuxin Chen, Stanford University

Joint Work with Qixing Huang (TTIC), Leonidas Guibas (Stanford)

Page 1

slide-2
SLIDE 2

Manual Assembly (Ephesus, Turkey) Computer Assembly (Fig. credit: Huang et al 06)

Assembling Fractured Pieces

Page 2

slide-3
SLIDE 3

Structure from Motion from Internet Images

Page 3

slide-4
SLIDE 4

Example: Joint Segmentation

Data-Driven Shape Analysis

Page 4

slide-5
SLIDE 5
  • Given: n objects (graphs), each containing a few elements (vertices)
  • Goal: consistently match all similar elements across all objects

Joint Object/Graph Matching

Page 5

slide-6
SLIDE 6

Naive Approach: Pairwise Matching

  • Naive Approach
  • Compute pairwise matching across all pairs in isolation
  • pairwise matching: extensively explored

Page 6

slide-7
SLIDE 7

Are Pairwise Methods Perfect?

Page 7

slide-8
SLIDE 8

Are Pairwise Methods Perfect?

Page 7

slide-9
SLIDE 9

Additional Object Helps!

Page 8

slide-10
SLIDE 10

Additional Object Helps!

Page 9

slide-11
SLIDE 11

Popular Approach: 2-Stage Method

  • Stage 1: Pairwise Matching
  • Compute pairwise matching across a few pairs in isolation
  • Use off-the-shelf pairwise methods

Page 10

slide-12
SLIDE 12

Popular Approach: 2-Stage Method

  • Stage 1: Pairwise Matching
  • Compute pairwise matching across a few pairs in isolation
  • Use off-the-shelf pairwise methods
  • Stage 2: Global Refinement
  • Jointly refine all provided maps
  • Criterion: exploit global consistency

Page 10

slide-13
SLIDE 13
  • Object
  • a set of points
  • drawn from the same universe
  • Map
  • point-to-point correspondence

Object Representation

Page 11

slide-14
SLIDE 14

Problem Formulation

  • Input: a few pairwise matches computed in isolation

Page 12

slide-15
SLIDE 15

Problem Formulation

  • Input: a few pairwise matches computed in isolation
  • Output:

a collection of maps that are

  • close to the input matches
  • globally consistent
  • NP-Hard! [Huber 02]

Page 12

slide-16
SLIDE 16

spanning tree optimization [Huber’02] detecting inconsistent cycles [Zach’10, Ngu’11] spectral technique [Kim’12, Huang’12]

  • Pros: empirical success
  • Cons:
  • little fundamental understanding (except [HuangGuibas’13])
  • rely on hyper-parameter tuning

Prior Art

Page 13

slide-17
SLIDE 17

Advances in Fundamental Understanding

  • Semidefinite Relaxation (HuangGuibas’13):
  • theoretical guarantees under a basic setup
  • tolerate 50% input errors

Page 14

slide-18
SLIDE 18

Advances in Fundamental Understanding

  • Semidefinite Relaxation (HuangGuibas’13):
  • theoretical guarantees under a basic setup
  • tolerate 50% input errors
  • Spectral Method (Pachauri et al’13):
  • recovery ability improves with # objects
  • Gaussian-Wigner noise (not realistic though...)

Page 14

slide-19
SLIDE 19

Advances in Fundamental Understanding

  • Semidefinite Relaxation (HuangGuibas’13):
  • theoretical guarantees under a basic setup
  • tolerate 50% input errors
  • Spectral Method (Pachauri et al’13):
  • recovery ability improves with # objects
  • Gaussian-Wigner noise (not realistic though...)
  • Several important challenges remain unaddressed...

Page 14

slide-20
SLIDE 20

Advances in Fundamental Understanding

  • Semidefinite Relaxation (HuangGuibas’13):
  • theoretical guarantees under a basic setup
  • tolerate 50% input errors
  • Spectral Method (Pachauri et al’13):
  • recovery ability improves with # objects
  • Gaussian-Wigner noise (not realistic though...)
  • Several important challenges remain unaddressed...
  • Relevant problems:
  • rotation sync (Wang et al), multiway alignment (Bandeira et al)

Page 14

slide-21
SLIDE 21

Input Maps Ground Truth

Challenge 1: Dense Input Errors

  • Input Errors
  • A significant fraction of inputs are corrupted

Page 15

slide-22
SLIDE 22

Input Maps Ground Truth

Challenge 1: Dense Input Errors

  • Input Errors
  • A significant fraction of inputs are corrupted
  • Prior art:

— tolerate 50% input errors [HuangGuibas’2013]

Page 15

slide-23
SLIDE 23

Subgraph Matching Input Maps

Challenge 2: Partial Similarity

  • Partial Similarity
  • Objects might only be partially similar to each other.

— e.g. restricted views at different camera positions

Page 16

slide-24
SLIDE 24

Challenge 3: Incomplete Input

  • Partial Input Matches
  • pairwise matching across all object pairs is

— computationally expensive — sometimes inadmissible

Page 17

slide-25
SLIDE 25

tolerate dense errors handle partial similarity fill in missing matches

Our Goal

  • Develop an effective joint recovery method
  • strong theoretical guarantee (address the 3 challenges)
  • parameter free
  • computationally feasible

Page 18

slide-26
SLIDE 26

(Partial) Maps

  • One-to-one maps between (sub)-sets of elements
  • subgraph matching / isomophism

Page 19

slide-27
SLIDE 27

(Partial) Maps

  • One-to-one maps between (sub)-sets of elements
  • subgraph matching / isomophism
  • Encode the maps across 2 objects by a 0-1 matrix

X12 :=   1 1 1  

Page 19

slide-28
SLIDE 28

X12 :=   1 1 1  

Matrix Representation

  • Consider n objects

Page 20

slide-29
SLIDE 29

X12 :=   1 1 1  

Matrix Representation

  • Consider n objects
  • Matrix representation for a collection of maps

X =     I X12 · · · X1n X21 I · · · X2n . . . . . . ... . . . Xn1 Xn2 · · · I    

  • Diagonal blocks: identity matrices (self-isomophism)
  • Sparse

Page 20

slide-30
SLIDE 30

X12 :=   1 1 1  

Alternative Representation: Augmented Universe

  • All objects / sets are sub-sampled from the same universe (of size m).

Page 21

slide-31
SLIDE 31

X12 :=   1 1 1  

Alternative Representation: Augmented Universe

  • All objects / sets are sub-sampled from the same universe (of size m).
  • Map matrix Yi between object i and the universe

Y 1 :=   1 1 1  

  • m columns

,

Y 2 :=     1 1 1 1    

  • m columns

⇒ X12 = Y 1Y ⊤

2

Page 21

slide-32
SLIDE 32

P.S.D. and Low-Rank Structure

  • Alternative Representation:

X :=     I X12 · · · X1n X21 I · · · X2n . . . . . . ... . . . Xn1 Xn2 · · · I     =     Y 1 Y 2 . . . Y n    

  • m columns
  • Y ⊤

1

Y ⊤

2

· · · Y ⊤

n

  • Page 22
slide-33
SLIDE 33

P.S.D. and Low-Rank Structure

  • Alternative Representation:

X :=     I X12 · · · X1n X21 I · · · X2n . . . . . . ... . . . Xn1 Xn2 · · · I     =     Y 1 Y 2 . . . Y n    

  • m columns
  • Y ⊤

1

Y ⊤

2

· · · Y ⊤

n

  • positive semidefinite and low rank:

rank(X) ≤ m.

  • m: universe size

Page 22

slide-34
SLIDE 34

A consistent map matrix X

  • 1. X 0
  • 2. low-rank
  • 3. sparse (0-1 matrix)
  • 4. Xii = I

X

=

Y Y ⊤

Summary of Matrix Structure

Page 23

slide-35
SLIDE 35

Input map matrix Xin

  • a noisy version of X

— input errors

  • missing entries

— incomplete inputs

ground truth X

⇒

input maps Xin

A consistent map matrix X

  • 1. X 0
  • 2. low-rank
  • 3. sparse (0-1 matrix)
  • 4. Xii = I

X

=

Y Y ⊤

Summary of Matrix Structure

Page 23

slide-36
SLIDE 36

input maps: Xin

⇐

ground truth: X

+

additive errors: Xin −X

Low Rank + Sparse Matrix Separation?

  • Robust PCA / Matrix Completion?
  • Candes et al
  • Chandrasekahran et al

minimizeL,S L∗

(low rank)

+ S1

(sparse)

, s.t. Xin = L ⇓ estimate of X + S

Page 24

slide-37
SLIDE 37

input maps: Xin

⇐

ground truth: X

+

additive errors: Xin −X

Outlier Component is Highly Biased

  • Robust PCA can handle dense corruption if
  • the sparse component exhibits random sign patterns

Page 25

slide-38
SLIDE 38

input maps: Xin

⇐

ground truth: X

+

additive errors: Xin −X

Outlier Component is Highly Biased

  • Robust PCA can handle dense corruption if
  • the sparse component exhibits random sign patterns
  • Our Case?

E

  • Xin − X
  • = ptrueX +(1 − ptrue)
  • corruption rate

· 1 m1·1⊤−X = (1 − ptrue) 1 m1 · 1⊤ − X

  • highly biased

spectral norm: (1 − ptrue) n

Page 25

slide-39
SLIDE 39

Original Form

X :=     Y 1 Y 2 . . . Y n    

  • Y ⊤

1

Y ⊤

2

· · · Y ⊤

n

  • Augmented Form
  • m

1⊤ 1 X

  • :=

   

1⊤ Y 1 Y 2 . . . Y n

    [ 1

Y ⊤

1

· · · Y ⊤

n ] 0

Debias the Error Components

  • Equivalently,

X − 1 m11⊤

debiasing Page 26

slide-40
SLIDE 40

Original Form

X :=     Y 1 Y 2 . . . Y n    

  • Y ⊤

1

Y ⊤

2

· · · Y ⊤

n

  • Augmented Form
  • m

1⊤ 1 X

  • :=

   

1⊤ Y 1 Y 2 . . . Y n

    [ 1

Y ⊤

1

· · · Y ⊤

n ] 0

Debias the Error Components

  • Equivalently,

X − 1 m11⊤

debiasing

  • rank
  • X − 1

m11⊤

= rank(X) − 1 ⇒

  • ne more degree of freedom

Page 26

slide-41
SLIDE 41

X ≥ 0, X 0

Objective Function

  • Ecourage consistency with provided maps

X, Xin (to maximize)

Page 27

slide-42
SLIDE 42

X ≥ 0, X 0

Objective Function

  • Ecourage consistency with provided maps

X, Xin (to maximize)

  • Promote Sparsity

X1= X, 11⊤ (to minimize)

Page 27

slide-43
SLIDE 43

X ≥ 0, X 0

Objective Function

  • Ecourage consistency with provided maps

X, Xin (to maximize)

  • Promote Sparsity

X1= X, 11⊤ (to minimize)

  • Encourage Low-Rank Structure?
  • minimize nuclear norm? – not necessary (X∗ is fixed)

Page 27

slide-44
SLIDE 44

X ≥ 0, X 0

Objective Function (to minimize) f (X) := −

  • X, Xin

+ λ

  • X, 11⊤

Objective Function

  • Ecourage consistency with provided maps

X, Xin (to maximize)

  • Promote Sparsity

X1= X, 11⊤ (to minimize)

  • Encourage Low-Rank Structure?
  • minimize nuclear norm? – not necessary (X∗ is fixed)

Page 27

slide-45
SLIDE 45

MatchLift minimizeX −

  • X, Xin

+ λ

  • X, 11⊤

subject to X ≥ 0,

  • m

1⊤ 1 X

  • 0,

Xii = I.

MatchLift: tractable convex program

  • Efficient Semidefinite Program

Page 28

slide-46
SLIDE 46

MatchLift minimizeX −

  • X, Xin

+ λ

  • X, 11⊤

subject to X ≥ 0,

  • m

1⊤ 1 X

  • 0,

Xii = I.

MatchLift: tractable convex program

  • Efficient Semidefinite Program
  • Caveat: m is usually unkonwn!

Page 28

slide-47
SLIDE 47

Spectral Method

  • 1. Trim Xin
  • 2. m

← − # dominant eigenvalues of Xin n = 50, m = 5

Pre-Estimate m: Spectral Method

  • The eigenvalues λi experience a sharp decrease around λm

Page 29

slide-48
SLIDE 48

Spectral Method

  • 1. Trim Xin
  • 2. m

← − # dominant eigenvalues of Xin

Convex Programming

minimizeX −

  • X, Xin

+ λ

  • X, 11⊤

subject to X ≥ 0, m 1⊤ 1 X

  • 0,

Xii = I.

Two-Step Procedure: MatchLift

  • 1. Pre-Estimate m:
  • 2. Joint Matching via Convex Relaxation:

Page 30

slide-49
SLIDE 49

Exact Recovery via MatchLift

  • Randomized Model: n objects, universe size m
  • Each object contains a fraction

pset

  • undersampling factor: partial similarity
  • f m elements

Page 31

slide-50
SLIDE 50

Exact Recovery via MatchLift

  • Randomized Model: n objects, universe size m
  • Each object contains a fraction

pset

  • undersampling factor: partial similarity
  • f m elements
  • Each pair Xin

ij is observed w.p.

pobs

  • bservation ratio: missing entries

Page 31

slide-51
SLIDE 51

Exact Recovery via MatchLift

  • Randomized Model: n objects, universe size m
  • Each object contains a fraction

pset

  • undersampling factor: partial similarity
  • f m elements
  • Each pair Xin

ij is observed w.p.

pobs

  • bservation ratio: missing entries
  • Each observed Xin

ij is randomly corrupted w.p. 1−

ptrue

  • non-corruption rate

Page 31

slide-52
SLIDE 52

Theorem (ChenGuibasHuang’14). MatchLift with λ = √pobs is exact with high probability if ptrue log2 (mn) p2

set

√pobsn

Exact Recovery via MatchLift

  • Randomized Model: n objects, universe size m
  • Each object contains a fraction

pset

  • undersampling factor: partial similarity
  • f m elements
  • Each pair Xin

ij is observed w.p.

pobs

  • bservation ratio: missing entries
  • Each observed Xin

ij is randomly corrupted w.p. 1−

ptrue

  • non-corruption rate

Page 31

slide-53
SLIDE 53

minimizeX − X, Xin + λX, 11⊤, s.t. feasible

Theorem (ChenGuibasHuang’14). MatchLift with λ = √pobs is exact with high probability if ptrue log2 (mn) p2

set

√pobsn

Exact Recovery via MatchLift

  • Parameter-free
  • MatchLift is insensitive to λ (λ ∈

pobs

m , √pobs

  • )

Page 32

slide-54
SLIDE 54

minimizeX − X, Xin + λX, 11⊤, s.t. feasible

Theorem (ChenGuibasHuang’14). MatchLift with λ = √pobs is exact with high probability if ptrue log2 (mn) p2

set

√pobsn

Exact Recovery via MatchLift

  • Parameter-free
  • MatchLift is insensitive to λ (λ ∈

pobs

m , √pobs

  • )
  • Dense Error Correction

error correction ability ≈ 1 − 1/√n when pset and pobs are constants.

Page 32

slide-55
SLIDE 55

minimizeX − X, Xin + λX, 11⊤, s.t. feasible

Theorem (ChenGuibasHuang’14). MatchLift with λ = √pobs is exact with high probability if ptrue log2 (mn) p2

set

√pobsn

Exact Recovery via MatchLift

  • Incomplete Input Matches
  • Error correction ability decays at rate 1/√pobs

Page 33

slide-56
SLIDE 56

minimizeX − X, Xin + λX, 11⊤, s.t. feasible

Theorem (ChenGuibasHuang’14). MatchLift with λ = √pobs is exact with high probability if ptrue log2 (mn) p2

set

√pobsn

Exact Recovery via MatchLift

  • Incomplete Input Matches
  • Error correction ability decays at rate 1/√pobs
  • Partial Similarity
  • Error correction ability decays at rate 1/p2

set Page 33

slide-57
SLIDE 57

Theorem (ChenGuibasHuang’14). MatchLift with λ = √pobs is exact with high probability if ptrue log2 (mn) p2

set

√pobsn

Optimality of MatchLift

  • Is MatchLift Optimal?

Page 34

slide-58
SLIDE 58

Theorem (ChenGoldsmith’14). If the universe size m is a constant, then No method works if ptrue 1 √pobsn (≈ 1 √avg-degree) Theorem (ChenGuibasHuang’14). MatchLift with λ = √pobs is exact with high probability if ptrue log2 (mn) p2

set

√pobsn

Optimality of MatchLift

  • Is MatchLift Optimal?
  • Information Theoretic Limits under Random Measurement Graphs
  • Fano’s inequality

Page 34

slide-59
SLIDE 59

n: number of objects

pfalse

50 70 90 110 130 150 0.8 0.7 0.6 0.5 0.4 0.3 0.2

MatchLift

n: number of objects

pfalse

50 70 90 110 130 150 0.8 0.7 0.6 0.5 0.4 0.3 0.2

Robust PCA

Phase Transitions in Empirical Success Probability

  • Synthetic Data

(input error rate v.s. # objects)

Page 35

slide-60
SLIDE 60

benchmark

(c)

initial maps

(c)

  • ptimized maps

0.02 0.04 0.06 0.08 0.1 0.2 0.4 0.6 0.8 1

Distance threshold ( ε) % Correspondences

Chair Input RPCA Matchlift

Benchmark: Chairs

Page 36

slide-61
SLIDE 61

benchmark initial maps

  • ptimized maps

Benchmark: CMU Hotel

Page 37

slide-62
SLIDE 62

Concluding Remarks

  • MatchLift
  • Dense error correction (near-optimal when m is constant)
  • Allow partial similarity
  • Incomplete inputs

Page 38

slide-63
SLIDE 63

Concluding Remarks

  • MatchLift
  • Dense error correction (near-optimal when m is constant)
  • Allow partial similarity
  • Incomplete inputs
  • Future direction
  • Pairwise matching and joint refinement all at once
  • More scalable algorithm

– e.g. via non-convex optimization?

Page 38

slide-64
SLIDE 64

Paper and Code

  • Near-Optimal Joint Object Matching via Convex Relaxation
  • Yuxin Chen, Leonidas J. Guibas, and Qixing Huang

– International Conference on Machine Learning (ICML), 2014

  • Arxiv: http://arxiv.org/abs/1402.1473
  • Code: http://web.stanford.edu/~yxchen/codes/code_MatchLift.

zip

Thank You! Questions?

Page 39