Selection for Feature-Based Image Registration F. Brunet 1,2 , A. - - PowerPoint PPT Presentation

▶
selection for feature based
SMART_READER_LITE
LIVE PREVIEW

Selection for Feature-Based Image Registration F. Brunet 1,2 , A. - - PowerPoint PPT Presentation

Aug 07, 2023 97 likes •374 views

Pixel-Based Hyperparameter Selection for Feature-Based Image Registration F. Brunet 1,2 , A. Bartoli 1 , N. Navab 2 , and R. Malgouyres 3 1 ISIT, Universit dAuvergne, Clermont -Ferrand, France 2 CAMPAR, Technische Universitt Mnchen,

slide-1
SLIDE 1

Pixel-Based Hyperparameter Selection for Feature-Based Image Registration

  • F. Brunet1,2, A. Bartoli1, N. Navab2, and R. Malgouyres3

1ISIT, Université d’Auvergne, Clermont-Ferrand, France 2CAMPAR, Technische Universität München, Munich, Germany 3LIMOS, Université d’Auvergne, Clermont-Ferrand, France

slide-2
SLIDE 2
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab
  • What is image registration?

– General principle – Standard approaches

  • Problem: choice of the hyperparameters
  • Our approach
  • Experimental results

Outline

slide-3
SLIDE 3
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab

Find the geometric transformation that aligns a source image and a target image

What is image registration?

W(¢;p)

Source image S

Target image T ¯nd p such that:

slide-4
SLIDE 4
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab
  • The feature-based approach
  • The direct approach (photometric approach)

Two standard approaches

slide-5
SLIDE 5
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab

The feature-based approach

Source image S

Target image T

slide-6
SLIDE 6
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab

Extracting point correspondences

The feature-based approach fqi $ q0

ign i=1

qi

q0

i

[ Methods: SIFT, SURF, … ]

slide-7
SLIDE 7
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab

The parameters p* of the transformation are computed from the point correspondences

The feature-based approach

p¤ = arg min

p n

X

i=1

kW(qi; p) ¡ q0

ik2

W(¢; p¤)

slide-8
SLIDE 8
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab
  • Variants

– Robustness – Regularization

The feature-based approach

min

p n

X

i=1

½ (W(qi; p) ¡ q0

i; °)

min

p n

X

i=1

½ (W(qi; p) ¡ q0

i; °) + ¸ 2

X

i=1

Z

Ð

° ° ° ° @2Wi @q2 (q; p) ° ° ° °

2

dq

Error Response Error Response

slide-9
SLIDE 9
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab

The direct approach

p¤ = arg min

p

X

q2R

kS(q) ¡ T(W(q; p))k2

Color S(q) Color T(W(q; p))

slide-10
SLIDE 10
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab
  • What are the hyperparameters?

Problem: the hyperparameters

min

p n

X

i=1

½ (W(qi; p) ¡ q0

i; °) + ¸R(p)

slide-11
SLIDE 11
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab

Determining the hyperparameters is mandatory!

The hyperparameters

Source image Target image

Number of control points

Too much Not enough

slide-12
SLIDE 12
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab

Determining the hyperparameters is mandatory!

The hyperparameters

Source image Target image

M-estimator threshold

Too high Too low

slide-13
SLIDE 13
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab

Determining the hyperparameters is mandatory!

The hyperparameters

Source image Target image

Regularization parameter

Too high Too low

slide-14
SLIDE 14
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab
  • How can we determine the hyperparameters?

Determining the hyperparameters

min

p;¸;°;::: n

X

i=1

½ (W(qi; p) ¡ q0

i; °) + ¸R(p)

slide-15
SLIDE 15
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab
  • Classical approach

Determining the hyperparameters

¸?; °?; : : : = arg min

¸;°;::: C(¸; °; : : :)

min

p n

X

i=1

½ (W(qi; p) ¡ q0

i; °?) + ¸?R(p)

slide-16
SLIDE 16
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab
  • A common approach : Cross Validation

– Measures the ability of « generalizing the data » – Divide the dataset into a training set and test set

  • Drawbacks

– Computation time – Only use the data of the problem (here, the point correspondences)

Cross Validation

slide-17
SLIDE 17
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab
  • Mallow’s Cp
  • Akaike Information Criterion (AIC)
  • Bayesian Information Criterion (BIC)
  • Minimum Description Length (MDL)
  • …
  • Always the same problem: only use the point

correspondences

Other criteria

slide-18
SLIDE 18
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab
  • Use all the available information:

– The point correspondences – And the pixel colors

  • Point correspondences: training set
  • Colors: test set

Our approach

slide-19
SLIDE 19
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab

Our approach

p¤ = arg min

p

X

q2R

kS(q) ¡ T(W(q; p))k2

C(¸; °; : : :) = 1 jRj

n

X

i=1

kS(q) ¡ T (W(q; p¸;°;:::))k2

slide-20
SLIDE 20
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab

Experimental results

slide-21
SLIDE 21
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab

Geometric error (with respect to the ground truth)

Experimental results

Geometric error Geometric error

slide-22
SLIDE 22

Experimental results

slide-23
SLIDE 23
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab

Criterion Geometric error VCV 1,852% VCV (robust) 0,675% Our criterion 0,190% Our criterion (robust) 0,197%

Experimental results

slide-24
SLIDE 24

Experimental results

slide-25
SLIDE 25
  • F. Brunet, A. Bartoli, R. Malgouyres, and N. Navab
  • Importance of the hyperparameters and of their

selection

  • New criterion that uses all the available information

– May be seen as a combination of the feature-based and the direct approaches to image registration

  • What’s next?

– How can we optimize the proposed criterion?

Conclusion

slide-26
SLIDE 26

Thank you!