Non-Rigid Registration Hao Li Local Shape Matching: Non-Rigid - - PowerPoint PPT Presentation

▶
non rigid registration
SMART_READER_LITE
LIVE PREVIEW

Non-Rigid Registration Hao Li Local Shape Matching: Non-Rigid - - PowerPoint PPT Presentation

May 16, 2023 424 likes •1.88k views

Non-Rigid Registration Hao Li Local Shape Matching: Non-Rigid Registration Correspondence Problem Classification How many meshes? Two: Pairwise registration Initial registration available? Yes: Local optimization methods Class of

slide-1
SLIDE 1

Local Shape Matching: Non-Rigid Registration

Non-Rigid Registration

Hao Li

slide-2
SLIDE 2

Local Shape Matching: Non-Rigid Registration

Correspondence Problem Classification

How many meshes?

Two: Pairwise registration

Initial registration available?

Yes: Local optimization methods

Class of transformations?

Non-rigid deformations

2

slide-3
SLIDE 3

Local Shape Matching: Non-Rigid Registration

Pair of Scans

3

slide-4
SLIDE 4

Local Shape Matching: Non-Rigid Registration

Pair of Scans

3

source

slide-5
SLIDE 5

Local Shape Matching: Non-Rigid Registration

Pair of Scans

3

target source

slide-6
SLIDE 6

Local Shape Matching: Non-Rigid Registration

Correspondences are lost

4

slide-7
SLIDE 7

Local Shape Matching: Non-Rigid Registration

Correspondences are lost

4

slide-8
SLIDE 8

Local Shape Matching: Non-Rigid Registration

Correspondences are lost

4

?

slide-9
SLIDE 9

Local Shape Matching: Non-Rigid Registration

What are the regions of overlap?

5

slide-10
SLIDE 10

Local Shape Matching: Non-Rigid Registration

What are the regions of overlap?

5

  • verlapping regions
slide-11
SLIDE 11

Local Shape Matching: Non-Rigid Registration

What are the regions of overlap?

5

missing data

  • verlapping regions
slide-12
SLIDE 12

Local Shape Matching: Non-Rigid Registration

What are the regions of overlap?

6

slide-13
SLIDE 13

Local Shape Matching: Non-Rigid Registration

What are the regions of overlap?

6

slide-14
SLIDE 14

Local Shape Matching: Non-Rigid Registration

Non-Rigid Registration

7

slide-15
SLIDE 15

Local Shape Matching: Non-Rigid Registration

Non-Rigid Registration

7

slide-16
SLIDE 16

Local Shape Matching: Non-Rigid Registration

Ingredients

8

slide-17
SLIDE 17

Local Shape Matching: Non-Rigid Registration

Ingredients

8

source

slide-18
SLIDE 18

Local Shape Matching: Non-Rigid Registration

Ingredients

8

source target

slide-19
SLIDE 19

Local Shape Matching: Non-Rigid Registration

Ingredients

8

source target

slide-20
SLIDE 20

Local Shape Matching: Non-Rigid Registration

Ingredients

8

source target

detect

  • verlap
slide-21
SLIDE 21

Local Shape Matching: Non-Rigid Registration

Ingredients

8

source target

detect

  • verlap
slide-22
SLIDE 22

Local Shape Matching: Non-Rigid Registration

Ingredients

8

source target

detect

  • verlap

correspond

slide-23
SLIDE 23

Local Shape Matching: Non-Rigid Registration

Ingredients

8

source target

detect

  • verlap

correspond

slide-24
SLIDE 24

Local Shape Matching: Non-Rigid Registration

Ingredients

8

source target

detect

  • verlap

correspond deform

slide-25
SLIDE 25

Local Shape Matching: Non-Rigid Registration

Ingredients

8

target registration

detect

  • verlap

correspond deform

slide-26
SLIDE 26

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

9

detect

  • verlap

correspond deform

slide-27
SLIDE 27

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

9

detect

  • verlap

correspond deform

slide-28
SLIDE 28

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

9

detect

  • verlap

correspond deform

slide-29
SLIDE 29

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

9

detect

  • verlap

correspond deform

deformation

slide-30
SLIDE 30

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

9

detect

  • verlap

correspond deform

slide-31
SLIDE 31

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

9

detect

  • verlap

correspond deform

slide-32
SLIDE 32

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

9

detect

  • verlap

correspond deform

ambiguity

slide-33
SLIDE 33

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

9

detect

  • verlap

correspond deform

slide-34
SLIDE 34

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

9

detect

  • verlap

correspond deform

?

incompleteness

slide-35
SLIDE 35

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

10

detect

  • verlap

correspond deform

slide-36
SLIDE 36

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

10

detect

  • verlap

correspond deform

slide-37
SLIDE 37

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

10

detect

  • verlap

correspond deform

slide-38
SLIDE 38

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

10

detect

  • verlap

correspond deform

scattered source

slide-39
SLIDE 39

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

10

detect

  • verlap

correspond deform

slide-40
SLIDE 40

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

10

detect

  • verlap

correspond deform

incomplete correspondences

slide-41
SLIDE 41

Local Shape Matching: Non-Rigid Registration

What’s so difficult?

10

detect

  • verlap

correspond deform

incomplete correspondences

slide-42
SLIDE 42

Local Shape Matching: Non-Rigid Registration

Challenges

11

detect

  • verlap

correspond deform

slide-43
SLIDE 43

Local Shape Matching: Non-Rigid Registration

Challenges

11

detect

  • verlap

correspond deform

slide-44
SLIDE 44

Local Shape Matching: Non-Rigid Registration

Observation

12

detect

  • verlap

correspond deform

slide-45
SLIDE 45

Local Shape Matching: Non-Rigid Registration

Observation

12

detect

  • verlap

correspond deform

slide-46
SLIDE 46

Local Shape Matching: Non-Rigid Registration

Observation

12

detect

  • verlap

correspond deform

helps

slide-47
SLIDE 47

Local Shape Matching: Non-Rigid Registration

Observation

12

detect

  • verlap

correspond deform

helps helps

slide-48
SLIDE 48

Local Shape Matching: Non-Rigid Registration

Observation

12

detect

  • verlap

correspond deform

slide-49
SLIDE 49

Local Shape Matching: Non-Rigid Registration

Observation

12

detect

  • verlap

correspond deform

slide-50
SLIDE 50

Local Shape Matching: Non-Rigid Registration

Observation

12

detect

  • verlap

correspond deform

global optimization via local refinement

slide-51
SLIDE 51

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

detect

  • verlap

correspond deform

slide-52
SLIDE 52

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

detect

  • verlap

correspond deform

slide-53
SLIDE 53

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

detect

  • verlap

correspond deform

slide-54
SLIDE 54

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

detect

  • verlap

correspond deform

slide-55
SLIDE 55

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

detect

  • verlap

deform closest point

slide-56
SLIDE 56

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

detect

  • verlap

deform closest point

slide-57
SLIDE 57

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

deform closest point pruning

slide-58
SLIDE 58

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

deform closest point pruning

slide-59
SLIDE 59

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

deform closest point pruning

slide-60
SLIDE 60

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

deform closest point pruning

slide-61
SLIDE 61

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

deform closest point pruning

slide-62
SLIDE 62

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

closest point pruning global

  • ptimization
slide-63
SLIDE 63

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

closest point pruning global

  • ptimization
slide-64
SLIDE 64

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

closest point pruning global

  • ptimization

converges?

slide-65
SLIDE 65

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

closest point pruning global

  • ptimization

converges? no

slide-66
SLIDE 66

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

closest point pruning global

  • ptimization

converges? no

slide-67
SLIDE 67

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

closest point pruning global

  • ptimization

converges? no

slide-68
SLIDE 68

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

closest point pruning global

  • ptimization

converges? yes relax stiffness no

slide-69
SLIDE 69

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

closest point pruning global

  • ptimization

converges? yes relax stiffness no

slide-70
SLIDE 70

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

closest point pruning global

  • ptimization

converges? yes relax stiffness no

slide-71
SLIDE 71

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

closest point pruning global

  • ptimization

Robust Non-Rigid ICP

converges? yes relax stiffness no

slide-72
SLIDE 72

Local Shape Matching: Non-Rigid Registration

Iterative Global Optimization

13

closest point pruning global

  • ptimization

Robust Non-Rigid ICP

converges? yes relax stiffness no

slide-73
SLIDE 73

Local Shape Matching: Non-Rigid Registration

Deformation Model

14

slide-74
SLIDE 74

Local Shape Matching: Non-Rigid Registration

Deformation Model

14

Embedded Deformation

slide-75
SLIDE 75

Local Shape Matching: Non-Rigid Registration

Deformation Model

14

Embedded Deformation incomplete scan

slide-76
SLIDE 76

Local Shape Matching: Non-Rigid Registration

Deformation Model

14

Embedded Deformation incomplete scan space deformation

slide-77
SLIDE 77

Local Shape Matching: Non-Rigid Registration

Deformation Model

14

Embedded Deformation incomplete scan high-resolution space deformation

slide-78
SLIDE 78

Local Shape Matching: Non-Rigid Registration

Deformation Model

14

Embedded Deformation incomplete scan high-resolution space deformation decoupled complexity

slide-79
SLIDE 79

Local Shape Matching: Non-Rigid Registration

Deformation Model

14

Embedded Deformation incomplete scan high-resolution deformation graph

slide-80
SLIDE 80

Local Shape Matching: Non-Rigid Registration

Deformation Model

14

Embedded Deformation incomplete scan high-resolution deformation graph

slide-81
SLIDE 81

Local Shape Matching: Non-Rigid Registration

Deformation Model

14

Embedded Deformation incomplete scan high-resolution deformation graph unknown warp

slide-82
SLIDE 82

Local Shape Matching: Non-Rigid Registration

Deformation Model

14

Embedded Deformation incomplete scan high-resolution deformation graph unknown warp incomplete match

slide-83
SLIDE 83

Local Shape Matching: Non-Rigid Registration

Deformation Model

14

Embedded Deformation incomplete scan high-resolution deformation graph unknown warp incomplete match detail preservation global consistency

slide-84
SLIDE 84

Local Shape Matching: Non-Rigid Registration

Deformation Model

14

Embedded Deformation incomplete scan high-resolution deformation graph unknown warp incomplete match detail preservation global consistency

slide-85
SLIDE 85

Local Shape Matching: Non-Rigid Registration

Deformation Model

14

Embedded Deformation incomplete scan high-resolution deformation graph unknown warp incomplete match detail preservation global consistency Linear Blend Skinning

slide-86
SLIDE 86

Local Shape Matching: Non-Rigid Registration

Deformation Model

14

Embedded Deformation incomplete scan high-resolution deformation graph unknown warp incomplete match detail preservation global consistency Linear Blend Skinning

slide-87
SLIDE 87

Local Shape Matching: Non-Rigid Registration

Deformation Model

14

Embedded Deformation incomplete scan high-resolution deformation graph unknown warp incomplete match detail preservation global consistency Linear Blend Skinning

slide-88
SLIDE 88

Local Shape Matching: Non-Rigid Registration

Deformation Model

15

detail preservation global consistency

slide-89
SLIDE 89

Local Shape Matching: Non-Rigid Registration

Deformation Model

15

detail preservation global consistency

slide-90
SLIDE 90

Local Shape Matching: Non-Rigid Registration

Deformation Model

15

detail preservation global consistency

slide-91
SLIDE 91

Local Shape Matching: Non-Rigid Registration

Deformation Model

15

detail preservation global consistency

slide-92
SLIDE 92

Local Shape Matching: Non-Rigid Registration

Deformation Model

15

detail preservation global consistency

slide-93
SLIDE 93

Local Shape Matching: Non-Rigid Registration

Deformation Model

15

detail preservation global consistency

slide-94
SLIDE 94

Local Shape Matching: Non-Rigid Registration

Deformation Model

15

detail preservation global consistency

slide-95
SLIDE 95

Local Shape Matching: Non-Rigid Registration

Deformation Model

15

detail preservation global consistency

Erigid

slide-96
SLIDE 96

Local Shape Matching: Non-Rigid Registration

Deformation Model

15

detail preservation global consistency

Erigid Esmooth

slide-97
SLIDE 97

Local Shape Matching: Non-Rigid Registration

Non-Linear Energy Minimization

16

ci vi

Erigid Esmooth

slide-98
SLIDE 98

Local Shape Matching: Non-Rigid Registration

Non-Linear Energy Minimization

16

ci vi

Erigid Esmooth

slide-99
SLIDE 99

Local Shape Matching: Non-Rigid Registration

Non-Linear Energy Minimization

16

ci vi

Erigid Esmooth

slide-100
SLIDE 100

Local Shape Matching: Non-Rigid Registration

Non-Linear Energy Minimization

16

ci vi

Erigid Esmooth

slide-101
SLIDE 101

Local Shape Matching: Non-Rigid Registration

Non-Linear Energy Minimization

16

ci vi

Epoint Erigid Esmooth

slide-102
SLIDE 102

Local Shape Matching: Non-Rigid Registration

Non-Linear Energy Minimization

16

ci vi

Eplane Epoint Erigid Esmooth

slide-103
SLIDE 103

Local Shape Matching: Non-Rigid Registration

Non-Linear Energy Minimization

16

ci vi

Eplane Etot

+αpoint +αrigid +αsmooth =

Epoint Erigid Esmooth

slide-104
SLIDE 104

Local Shape Matching: Non-Rigid Registration

Non-Linear Energy Minimization

16

ci vi

Etot

+αpoint +αrigid +αsmooth =

Epoint Eplane Erigid Esmooth

slide-105
SLIDE 105

Local Shape Matching: Non-Rigid Registration

Non-Linear Energy Minimization

16

ci vi

Etot

+αpoint +αrigid +αsmooth =

Epoint Eplane Erigid Esmooth

non-linear least squares minimization

slide-106
SLIDE 106

Local Shape Matching: Non-Rigid Registration

Non-Linear Energy Minimization

16

ci vi

Etot

+αpoint +αrigid +αsmooth =

Epoint Eplane Erigid Esmooth

non-linear least squares minimization Gauss-Newton method

slide-107
SLIDE 107

Local Shape Matching: Non-Rigid Registration

Non-Linear Energy Minimization

16

ci vi

Etot

+αpoint +αrigid +αsmooth =

Epoint Eplane Erigid Esmooth

non-linear least squares minimization Gauss-Newton method Jacobian is sparse

slide-108
SLIDE 108

Local Shape Matching: Non-Rigid Registration

Non-Linear Energy Minimization

16

sparse Cholesky factorization

ci vi

Etot

+αpoint +αrigid +αsmooth =

Epoint Eplane Erigid Esmooth

non-linear least squares minimization Gauss-Newton method Jacobian is sparse

slide-109
SLIDE 109

Local Shape Matching: Non-Rigid Registration

Non-Linear Energy Minimization

16

sparse Cholesky factorization

ci vi

Etot

+αpoint +αrigid +αsmooth =

Epoint Eplane Erigid Esmooth

non-linear least squares minimization Gauss-Newton method Jacobian is sparse

that’s it!

slide-110
SLIDE 110

Local Shape Matching: Non-Rigid Registration

Synthetic Model

17

slide-111
SLIDE 111

Local Shape Matching: Non-Rigid Registration

Comparison to Ground-Truth

18

slide-112
SLIDE 112

Local Shape Matching: Non-Rigid Registration

Comparison with other N-ICP

19

N-ICP

  • ur method

[Pauly et al. ‘05] [Pottmann et al. ‘06]

slide-113
SLIDE 113

Local Shape Matching: Non-Rigid Registration

Comparison with other N-ICP

19

N-ICP

  • ur method

texturing

[Pauly et al. ‘05] [Pottmann et al. ‘06]

slide-114
SLIDE 114

Local Shape Matching: Non-Rigid Registration

Comparison with other N-ICP

19

N-ICP

  • ur method

texturing

[Pauly et al. ‘05] [Pottmann et al. ‘06]

slide-115
SLIDE 115

Local Shape Matching: Non-Rigid Registration

Comparison with other N-ICP

20

[Pauly et al. ‘05] [Pottmann et al. ‘06]

N-ICP

slide-116
SLIDE 116

Local Shape Matching: Non-Rigid Registration

Comparison with other N-ICP

20

[Pauly et al. ‘05] [Pottmann et al. ‘06]

N-ICP

  • ur method
slide-117
SLIDE 117

Local Shape Matching: Non-Rigid Registration

Comparison with other N-ICP

20

[Pauly et al. ‘05] [Pottmann et al. ‘06]

N-ICP

  • ur method
slide-118
SLIDE 118

Local Shape Matching: Non-Rigid Registration

Draping Table Cloth

21

source

slide-119
SLIDE 119

Local Shape Matching: Non-Rigid Registration

Draping Table Cloth

21

source target initial alignment

slide-120
SLIDE 120

Local Shape Matching: Non-Rigid Registration

Suitable for Isometric Deformations

22

N-ICP

  • ur method
slide-121
SLIDE 121

Local Shape Matching: Non-Rigid Registration

Suitable for Isometric Deformations

22

N-ICP

  • ur method
slide-122
SLIDE 122

Local Shape Matching: Non-Rigid Registration

Suitable for Isometric Deformations

22

N-ICP

  • ur method
slide-123
SLIDE 123

Local Shape Matching: Non-Rigid Registration

Suitable for Isometric Deformations

22

N-ICP

  • ur method
slide-124
SLIDE 124

Local Shape Matching: Non-Rigid Registration

Some Examples

23

slide-125
SLIDE 125

Local Shape Matching: Non-Rigid Registration

Face Capture Pipeline

24

acquisition

data provided by Paramount Pictures and Aguru Images

slide-126
SLIDE 126

Local Shape Matching: Non-Rigid Registration

Face Capture Pipeline

24

acquisition initial alignment

data provided by Paramount Pictures and Aguru Images

slide-127
SLIDE 127

Local Shape Matching: Non-Rigid Registration

Face Capture Pipeline

24

acquisition initial alignment registration

data provided by Paramount Pictures and Aguru Images

slide-128
SLIDE 128

Local Shape Matching: Non-Rigid Registration

Face Capture Pipeline

24

acquisition initial alignment registration merging

data provided by Paramount Pictures and Aguru Images

slide-129
SLIDE 129

Local Shape Matching: Non-Rigid Registration

Digital Face Cloning (scan-to-scan)

25

data provided by Paramount Pictures and Aguru Images

slide-130
SLIDE 130

Local Shape Matching: Non-Rigid Registration

Digital Face Cloning (scan-to-scan)

25

data provided by Paramount Pictures and Aguru Images

slide-131
SLIDE 131

Local Shape Matching: Non-Rigid Registration

Digital Face Cloning (scan-to-scan)

25

data provided by Paramount Pictures and Aguru Images

slide-132
SLIDE 132

Local Shape Matching: Non-Rigid Registration

Digital Face Cloning

26

data provided by Paramount Pictures and Aguru Images

slide-133
SLIDE 133

Local Shape Matching: Non-Rigid Registration

Digital Face Cloning

26

data provided by Paramount Pictures and Aguru Images

slide-134
SLIDE 134

Local Shape Matching: Non-Rigid Registration

It’s pretty robust

27

Caesar Data Set – Test with Michael Black

slide-135
SLIDE 135

Local Shape Matching: Non-Rigid Registration

It’s pretty robust

27

Caesar Data Set – Test with Michael Black custom scan

slide-136
SLIDE 136

Local Shape Matching: Non-Rigid Registration

It’s pretty robust

27

Caesar Data Set – Test with Michael Black generic template custom scan

slide-137
SLIDE 137

Local Shape Matching: Non-Rigid Registration

It’s pretty robust

27

Caesar Data Set – Test with Michael Black

slide-138
SLIDE 138

Local Shape Matching: Non-Rigid Registration

Application: Statistical Database

28

Caesar Data Set – Test with Michael Black

slide-139
SLIDE 139

Local Shape Matching: Non-Rigid Registration

Application: Statistical Database

28

Caesar Data Set – Test with Michael Black

slide-140
SLIDE 140

Local Shape Matching: Non-Rigid Registration

What’s Next?

29

slide-141
SLIDE 141

Local Shape Matching: Non-Rigid Registration

What’s Next?

29

slide-142
SLIDE 142

Local Shape Matching: Non-Rigid Registration

What’s Next?

29

slide-143
SLIDE 143

Local Shape Matching: Non-Rigid Registration

30

www.hao-li.com

slide-144
SLIDE 144

Local Shape Matching: Non-Rigid Registration

30

?

www.hao-li.com