Feature Matching via Sparse Relaxation Models jiangbo@ahu.edu.cn - - PowerPoint PPT Presentation

▶
feature matching via sparse relaxation models
SMART_READER_LITE
LIVE PREVIEW

Feature Matching via Sparse Relaxation Models jiangbo@ahu.edu.cn - - PowerPoint PPT Presentation

Mar 08, 2024 43 likes •675 views

Feature Matching via Sparse Relaxation Models jiangbo@ahu.edu.cn 2018-8-8 Content 1 Introduction 2 Problem formulation Related works 3 Sparse models for matching 4 Conclusion and

slide-1
SLIDE 1

江 波

jiangbo@ahu.edu.cn

安徽大学 计算机科学与技术学院 2018-8-8

Feature Matching via Sparse Relaxation Models

slide-2
SLIDE 2

Content

Problem formulation 2 Related works 3 Sparse models for matching 4 Conclusion and future works 5 Introduction 1

slide-3
SLIDE 3

Introduction

slide-4
SLIDE 4

Introduction

slide-5
SLIDE 5

Introduction

Object detection

slide-6
SLIDE 6

Introduction

Object detection Person ReID, Zhou et al. AAAI 2018

slide-7
SLIDE 7

Introduction

Object detection Person ReID, Zhou et al. AAAI 2018

***

Luo et al. PAMI 2001

slide-8
SLIDE 8

Introduction

Object detection Person ReID, Zhou et al. AAAI 2018

***

Object tracking, CVPR 2016 Luo et al. PAMI 2001 Object tracking, Nebehay et al. CVPR15

slide-9
SLIDE 9

Introduction

Object detection Person ReID, Zhou et al. AAAI 2018

***

Object tracking, Nebehay et al. CVPR15 Shape matching, Bai et al. PAMI2008 Luo et al. PAMI 2001

slide-10
SLIDE 10

Introduction

Object detection Person ReID, Zhou et al. AAAI 2018

***

Object tracking, CVPR 2016 Common Visual Pattern Discovery Shape matching, Bai et al. PAMI2008 Luo et al. PAMI 2001 Object tracking, Nebehay et al. CVPR15

slide-11
SLIDE 11

Problem Formulation

slide-12
SLIDE 12

Problem Formulation

slide-13
SLIDE 13

Problem Formulation

slide-14
SLIDE 14

Problem Formulation

slide-15
SLIDE 15

Problem Formulation

slide-16
SLIDE 16

Problem Formulation

slide-17
SLIDE 17

Integer Quadratic Programming (IQP) problem  NP-hard problem  Approximate solution

Problem Formulation

slide-18
SLIDE 18

Continuous Relaxation

Related works

slide-19
SLIDE 19

Continuous Relaxation

Related works

slide-20
SLIDE 20

Continuous Relaxation

Related works

Continuous relaxation Continuous solution Discrete solution

Continuous

  • ptimization

Post-discretization Local optimal for the relaxed problem Not a local optima for the original problem

slide-21
SLIDE 21

Continuous Relaxation

Related works

Spectral matching-ICCV 2005 Spectral matching with affine constraint-NIPS 2006 Doubly stochastic relaxation

  • GA-PAMI 1996
  • POCS-PAMI 2004
  • RRWM-ICCV 2010
  • SCGA-ECCV 2012
  • Probabilistic Models

……

slide-22
SLIDE 22

Discrete Methods

Related works

Integer Projected Fixed Point (IPFP) -NIPS 2009 Factorized Graph Matching (FGM) - CVPR 2012 Discrete Tabu Search –ICCV 2015 Hungarian-BP-CVPR 2016

slide-23
SLIDE 23

Sparse Relaxation

Related works

Discrete constraint

Nonnegative sparse Nonnegative sparse model

relaxation

slide-24
SLIDE 24

Sparse Relaxation

Related works

Spectral matching (SM)-ICCV 2005 Game-theoretic matching (GameM)-ICCV 2009, IJCV 2011 Elastic net matching (EnetM)-ICCV 2013 Sparse nonnegative matrix factorization (SNMF)-PR 2014

slide-25
SLIDE 25

Local sparse model for matching

Motivation

slide-26
SLIDE 26

Local sparse model for matching

Motivation

slide-27
SLIDE 27

Local sparse model for matching

Motivation

slide-28
SLIDE 28

Local sparse model for matching

Motivation

slide-29
SLIDE 29

Local sparse model for matching

Motivation

slide-30
SLIDE 30
  • Each row of solution matrix X is sparse
  • There is no zero row in solution matrix X

Local Sparse Model

Observations Motivation

Local sparse model for matching

slide-31
SLIDE 31

Local sparse matching L12 norm

  • L1 norm on each row encourages sparsity
  • L2 norm on rows encourages that there is no zero row

Local sparse

Local sparse model for matching

Bo Jiang, et al., A Local sparse model for matching problem, AAAI 20 2015

slide-32
SLIDE 32

Algorithm

is the matrix form of

Properties

Local sparse model for matching

Bo Jiang, et al., A Local sparse model for matching problem, AAAI 20 2015

slide-33
SLIDE 33

Local sparse model for matching

Illustration

slide-34
SLIDE 34

Local sparse model for matching

Bo Jiang, et al., A Local sparse model for matching problem, AAAI 20 2015

slide-35
SLIDE 35

Local sparse model for matching

Bo Jiang, et al., A Local sparse model for matching problem, AAAI 20 2015

slide-36
SLIDE 36

Integer Quadratic Programming (IQP) problem

Binary constraint preserving matching

Motivation

slide-37
SLIDE 37

BPGM formulation

  • As ϒ becomes larger, the more closely x approximates to discrete
  • It provides a series of relaxation models

Binary constraint preserving

Binary constraint preserving matching

Bo Jiang, et al., Binary constraint preserving graph matching, CVPR 20 2017

slide-38
SLIDE 38

Theoretical analysis

Pro Prope perty rty 1. When ϒ = n, where n is the number of features, BPGM model is equivalent to original matching problem

Binary constraint preserving matching

Pro rope perty rty 2. When ϒ = ||x*||, where x* is the optimal solution of problem (2), BPGM model is equivalent to the matching problem (2)

Balanced model between (1) and (2)

Bo Jiang, et al., Binary constraint preserving graph matching, CV CVPR 20 2017

slide-39
SLIDE 39

Theoretical analysis

Binary constraint preserving matching

Lemma Lemma 3. There exists a parameter ϒ0 such that BPGM with ϒ=ϒ0 has a global optimal solution

Path-following strategy

Starting from global optimal solution and aims to obtain the discrete solution

Bo Jiang, et al., Binary constraint preserving graph matching, CV CVPR 20 2017

slide-40
SLIDE 40

Algorithm

Binary constraint preserving matching

Bo Jiang, et al., Binary constraint preserving graph matching, CV CVPR 20 2017

slide-41
SLIDE 41

Binary constraint preserving matching

slide-42
SLIDE 42

Binary constraint preserving matching

slide-43
SLIDE 43

Multiplicative update matching

Doubly stochastic relaxation Doubly stochastic relaxation Continuous solution Discrete solution Discrete solution

Continuous

  • ptimization

Post-discretization Continuous

  • ptimization

Hungarian algorithm Local optimal for the relaxed problem Local optimal for the relaxed problem

Traditional methods Our method

Not a local

  • ptima for the
  • riginal problem

A local optima for the original problem

slide-44
SLIDE 44

Multiplicative update matching

Doubly-stochastic Relaxation

slide-45
SLIDE 45

Multiplicative update matching

Doubly-stochastic Relaxation

slide-46
SLIDE 46

Multiplicative update matching

Doubly-stochastic Relaxation

slide-47
SLIDE 47

Multiplicative update matching

Doubly-stochastic Relaxation Multipliers

slide-48
SLIDE 48

Multiplicative update matching

Doubly-stochastic Relaxation Multipliers

slide-49
SLIDE 49

Multiplicative update matching

Doubly-stochastic Relaxation Multipliers

slide-50
SLIDE 50

Multiplicative update matching

Doubly-stochastic Relaxation Multipliers

slide-51
SLIDE 51

Multiplicative update matching

Doubly-stochastic Relaxation Multipliers

slide-52
SLIDE 52

Multiplicative update matching

Doubly-stochastic Relaxation Solution update

slide-53
SLIDE 53

Multiplicative update matching

Doubly-stochastic Relaxation Solution update

slide-54
SLIDE 54

Multiplicative update matching

slide-55
SLIDE 55

Multiplicative update matching

Algorithm

slide-56
SLIDE 56

Multiplicative update matching

Algorithm

slide-57
SLIDE 57

Multiplicative update matching

Convergence Optimality Bo Jiang, et al., Graph Matching via Multiplicative Update Algorithm, NI NIPS 2017

slide-58
SLIDE 58

Multiplicative update matching

Top: start from uniform solution Middle: start from Spectral Matching (SM) solution Bottom: start from Random Walk (RRWM) solution Bo Jiang, et al., Graph Matching via Multiplicative Update Algorithm, NI NIPS 2017

slide-59
SLIDE 59

Multiplicative update matching

Synthetic data

slide-60
SLIDE 60

Multiplicative update matching

slide-61
SLIDE 61

Reference

 Bo Jiang, Jin Tang, Chris Ding, Yihong Gong and Bin Luo, Graph Matching via Multiplicative Update Algorithm, Neural Information Processing Systems (NIPS-2017)  Bo Jiang, Jin Tang, Bin Luo and Chris Ding, Binary constraint preserving graph matching, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.4402-4409, 2017  Bo Jiang, Jin Tang, Chris Ding and Bin Luo, Nonnegative Orthogonal Graph Matching, AAAI Conference on Artificial Intelligence (AAAI), pp.4089-4095, 2017  Bo Jiang, Jin Tang, Xiaochun Cao, Bin Luo, Lagrangian relaxation graph matching, Pattern Recognition, 61: 255-265, 2017  Bo Jiang, Jin Tang, Chris Ding and Bin Luo, A local sparse model for matching problem, AAAI Conference on Artificial Intelligence (AAAI), pp. 3790-3796, 2015  Bo Jiang, Jin Tang, Bin Luo and Liang Lin, Robust feature point matching with sparse model, IEEE Transactions on Image Processing, 23(12):5175-5186, 2014

slide-62
SLIDE 62

Conclusion and Future works

 Sparse relaxation model for matching problem  Binary constraint preserving model for matching  Multiplicative update algorithm for matching  More theoretical analysis on Multiplicative matching  More effective algorithm to solve sparse matching model  Matching objective relaxation

Future works Conclusion

slide-63
SLIDE 63

Thank you !

jiangbo@ahu.edu.cn