Registration Deformation models Marcel Lthi Graphics and Vision - - PowerPoint PPT Presentation

β–Ά
registration deformation models
SMART_READER_LITE
LIVE PREVIEW

Registration Deformation models Marcel Lthi Graphics and Vision - - PowerPoint PPT Presentation

May 27, 2023 414 likes β€’1.1k views

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Registration Deformation models Marcel Lthi Graphics and Vision Research Group Department of Mathematics and Computer Science University of Basel University of Basel > DEPARTMENT OF

slide-1
SLIDE 1

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

Registration – Deformation models

Marcel LΓΌthi

Graphics and Vision Research Group Department of Mathematics and Computer Science University of Basel

slide-2
SLIDE 2

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Registration as analysis by synthesis

Parameters πœ„

Comparison: π‘ž π½π‘ˆ πœ„, 𝐽𝑆) Update using π‘ž(πœ„|π½π‘ˆ, 𝐽𝑆) Synthesis πœ’[πœ„]

Prior πœ’[πœ„] ∼ π‘ž(πœ„) π½π‘ˆ 𝐽𝑆 ∘ πœ’[πœ„]

slide-3
SLIDE 3

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Priors

Define the Gaussian process 𝑣 ∼ 𝐻𝑄 𝜈, 𝑙 with mean function 𝜈: Ξ© β†’ ℝ2 and covariance function 𝑙: Ξ© Γ— Ξ© β†’ ℝ2Γ—2 .

Characteristics

  • f deformation

fields

slide-4
SLIDE 4

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Zero mean: 𝜈 𝑦 = 0 Squared exponential covariance function (Gaussian kernel) 𝑙 𝑦, 𝑦′ = s1exp βˆ’ 𝑦 βˆ’ 𝑦′ 2 𝜏1

2

s2exp βˆ’ 𝑦 βˆ’ 𝑦′ 2 𝜏2

2

Example prior: Smooth 2D deformations

slide-5
SLIDE 5

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

𝑑1 = 𝑑2 small, 𝜏1 = 𝜏2 large

Example prior: Smooth 2D deformations

slide-6
SLIDE 6

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

𝑑1 = 𝑑2 small, 𝜏1 = 𝜏2 small

Example prior: Smooth 2D deformations

slide-7
SLIDE 7

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

𝑑1 = 𝑑2 large, 𝜏1 = 𝜏2 large

Example prior: Smooth 2D deformations

slide-8
SLIDE 8

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Why are priors interesting?

πœ„

πœ„βˆ— = arg max

πœ„

π‘ž πœ’ πœ„ π‘ž(π½π‘ˆ|𝐽𝑆, πœ’[πœ„])

slide-9
SLIDE 9

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Why are priors interesting?

πœ„βˆ— = arg max

πœ„

π‘ž πœ’ πœ„ π‘ž(π½π‘ˆ|𝐽𝑆, πœ’[πœ„])

πœ„

slide-10
SLIDE 10

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Intermezzo – The space of samples

10

slide-11
SLIDE 11

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Gaussian processes - Deeper Insights

slide-12
SLIDE 12

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Scalar-valued Gaussian processes

Vector-valu lued (th (this is cou

  • urse)
  • Samples u are deformation fields:

𝑣: β„π‘œ β†’ ℝ𝑒 Sc Scalar-valu lued (m (more common)

  • Samples f are real-valued functions

𝑔 ∢ β„π‘œ β†’ ℝ

slide-13
SLIDE 13

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

The space of samples

Argument:

  • Covariance function 𝑙 is symmetric and positive definite
  • For any finite sample it holds that:

=> the covariance matrix is symmetric => rowspace = columnspace = eigenspace

18

𝑣 ∼ 𝜈 + σ𝑗 𝛽𝑗 πœ‡π‘— πœšπ‘— = σ𝑗 𝛾𝑗𝑙(𝑦𝑗,β‹…) for some 𝛾 Samples are linear combinations of the β€œrows” of 𝑙

slide-14
SLIDE 14

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Example: Gaussian kernel

19

  • Click to edit Master text styles
  • Second level
  • Third level
  • Fourth level
  • Fifth level

𝑙 𝑦, 𝑦′ = exp βˆ’ 𝑦 βˆ’ 𝑦′ 2 𝜏2

slide-15
SLIDE 15

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Example: Gaussian kernel

20

𝑙 𝑦, 𝑦′ = exp βˆ’ 𝑦 βˆ’ 𝑦′ 2 𝜏2

Οƒ = 3

slide-16
SLIDE 16

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Multi-scale signals

  • k x, xβ€² = exp βˆ’ 𝑦 βˆ’

𝑦′ 1 2

+ 0.1 exp βˆ’ 𝑦 βˆ’

𝑦′ 0.1 2

21

slide-17
SLIDE 17

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Periodic kernels

  • Define 𝑣 𝑦 =

cos 𝑦 sin(𝑦)

  • 𝑙 𝑦, 𝑦′ = exp(βˆ’β€–(𝑣 𝑦 βˆ’ 𝑣 𝑦′ β€–2= exp(βˆ’4 sin2

‖𝑦 βˆ’π‘¦β€²β€– 𝜏2

)

22

slide-18
SLIDE 18

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Symmetric kernels

  • Enforce that f(x) = f(-x)
  • 𝑙 𝑦, 𝑦′ = 𝑙 βˆ’π‘¦, 𝑦′ + 𝑙(𝑦, 𝑦′)

23

slide-19
SLIDE 19

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Changepoint kernels

  • 𝑙 𝑦, 𝑦′ = 𝑑 𝑦 𝑙1 𝑦, 𝑦′ 𝑑 𝑦′ + (1 βˆ’ 𝑑 𝑦 )𝑙2(𝑦, 𝑦′)(1 βˆ’ 𝑑 𝑦′ )
  • s 𝑦 =

1 1+exp( βˆ’π‘¦)

24

slide-20
SLIDE 20

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

f x = x

25

Combining existing functions

𝑙 𝑦, 𝑦′ = 𝑔 𝑦 𝑔 𝑦′

slide-21
SLIDE 21

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel 26

Combining existing functions

𝑙 𝑦, 𝑦′ = 𝑔 𝑦 𝑔 𝑦′ f x = sin(x)

slide-22
SLIDE 22

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

{f1 x = x, f2 x = sin(x)}

27

𝑙 𝑦, 𝑦′ = ෍

𝑗

𝑔

𝑗 𝑦 𝑔 𝑗(𝑦′)

Combining existing functions

slide-23
SLIDE 23

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Statistical models

Statistical shape models are linear combinations of example deformations 𝑣1, … π‘£π‘œ.

𝜈 𝑦 = 𝑣 𝑦 = 1 π‘œ ෍

π‘—βˆ’1 π‘œ

𝑣𝑗 (𝑦) 𝑙𝑇𝑁 𝑦, 𝑦′ = 1 π‘œ βˆ’ 1 ෍

𝑗 π‘œ

(𝑣𝑗 𝑦 βˆ’ 𝑣(𝑦)) 𝑣𝑗 𝑦′ βˆ’ 𝑣(𝑦′)

π‘ˆ

𝑣1 ∢ Ξ© β†’ ℝ2 𝑣2 ∢ Ξ© β†’ ℝ2 … π‘£π‘œ ∢ Ξ© β†’ ℝ2

slide-24
SLIDE 24

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Gaussian process regression

  • Given: observations {(𝑦1, 𝑧1), … , π‘¦π‘œ, π‘§π‘œ }
  • Model: 𝑧𝑗 = 𝑔 𝑦𝑗 + πœ—, 𝑔 ∼ 𝐻𝑄(𝜈, 𝑙)
  • Goal: compute p(π‘§βˆ—|π‘¦βˆ—, 𝑦1, … , π‘¦π‘œ, 𝑧1, … , π‘§π‘œ)

29

𝑦1 𝑦2 π‘¦π‘œ π‘¦βˆ— π‘§βˆ— 𝑧1 𝑧2 π‘§π‘œ

slide-25
SLIDE 25

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Gaussian process regression

  • Solution given by posterior process 𝐻𝑄 πœˆπ‘ž, π‘™π‘ž with

πœˆπ‘ž(π‘¦βˆ—) = 𝐿 π‘¦βˆ—, π‘Œ 𝐿 π‘Œ, π‘Œ + 𝜏2𝐽 βˆ’1𝑧 π‘™π‘ž π‘¦βˆ—, π‘¦βˆ—β€² = 𝑙 π‘¦βˆ—, π‘¦βˆ—β€² βˆ’ 𝐿 π‘¦βˆ—, π‘Œ 𝐿 π‘Œ, π‘Œ + 𝜏2𝐽 βˆ’1𝐿 π‘Œ, π‘¦βˆ—

β€²

  • The covariance is independent of the value at the training points
  • Structure of posterior GP determined solely by kernel.
  • The most likely solution is a linear combination of kernels evaluated at the training points
  • This is known as the Rep

epresenter er Th Theorem in machine learning.

  • Structure of solution determined solely by kernel.

30

slide-26
SLIDE 26

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Illustration: Representer theorem

31

slide-27
SLIDE 27

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Examples

32

slide-28
SLIDE 28

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Examples

33

  • Gaussian kernel (𝜏 = 1)
slide-29
SLIDE 29

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Examples

34

  • Gaussian kernel (𝜏 = 5)
slide-30
SLIDE 30

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Examples

35

  • Periodic kernel
slide-31
SLIDE 31

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Examples

36

  • Changepoint kernel
slide-32
SLIDE 32

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Examples

37

  • Symmetric kernel
slide-33
SLIDE 33

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Examples

38

  • Linear kernel
slide-34
SLIDE 34

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Deformation models for registration

39

slide-35
SLIDE 35

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Basic assumption: Deformation fields are smooth

  • Typical assumption:
  • Deformation field is smooth
  • GP approach
  • Choose smooth kernel functions

𝑙 𝑦, 𝑦′ = 𝑑 exp(βˆ’ 𝑦 βˆ’ 𝑦′ 2 𝜏2 )

  • Regularization operators
  • Penalize large derivatives

β„› 𝑣 = 𝑆𝑣 2 = ෍

𝑗=0 π‘œ

𝛽𝑗 𝐸𝑗𝑣 2

slide-36
SLIDE 36

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Green’s functions and covariance functions

β„› 𝑣 = 𝑆𝑣 2 = ෍

𝑗=0 π‘œ

𝛽𝑗 𝐸𝑗𝑣 2 Corresponding covariance function for GP is the Greens function G: π‘†βˆ—π‘†π» 𝑦, 𝑧 = πœ€(𝑦 βˆ’ 𝑧)

  • We can define Gaussian processes, which mimic typical regularization operators.
  • T. Poggio and F. Girosi; Networks for Approximation and Learning, Proceedings of the IEEE, 1990
slide-37
SLIDE 37

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Example: Gaussian kernel

𝑙 𝑦, 𝑦′ = exp(βˆ’ 𝑦 βˆ’ 𝑦′ 2 𝜏2 )

β„› 𝑣 =

𝑆𝑣 2 = ෍

𝑗=0 ∞ 𝜏2𝑗

𝑗! 2𝑗 𝐸𝑗𝑣 2

  • Non-zero functions are penalized
  • pushes functions to zero away from data

Yuille, A. and Grzywacz M. A mathematical analysis of the motion coherence theory. International Journal of Computer vision

slide-38
SLIDE 38

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Example: Exponential kernel (1D case)

𝑙 𝑦, 𝑦′ = 1 2𝛽 exp(βˆ’π›½ 𝑦 βˆ’ 𝑦′ ) β„› 𝑣 = 𝑆𝑣 2 = 𝛽2𝑣 + 𝐸1𝑣 2

Rasmussen, Carl Edward, and Christopher KI Williams. Gaussian processes for machine learning. Vol. 1. Cambridge: MIT press, 2006.

slide-39
SLIDE 39

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

MatΓ©rn class of kernels

𝑙 𝑦, 𝑦′ = 𝑑 21βˆ’πœ‰ Ξ“ πœ‰ 2 2πœ‰ 𝑦 βˆ’ 𝑦′ 𝜍

πœ‰

𝐿𝑀( 2πœ‰ 𝑦 βˆ’ 𝑦′ 𝜍 )

  • Ξ“ is the Ξ“ function, π‘™πœ‰ the modified Bessel function and πœ‰, 𝜍 are parameters
  • Process 𝑣~𝐻𝑄 0, 𝑙 is πœ‰ βˆ’ 1 times m.s. differentiable
  • Special cases:
  • πœ‰ =

1 2 : 𝑙 𝑦, 𝑦′ = 𝑑 exp(βˆ’ π‘¦βˆ’π‘¦β€² 𝜍

)

  • πœ‰ =

3 2 : k x, xβ€² = 𝑑(1 + 3 π‘¦βˆ’π‘¦β€² 𝜍

) exp(βˆ’

3 π‘¦βˆ’π‘¦β€² 𝜍

)

  • πœ‰ β†’ ∞ Gaussian kernel
slide-40
SLIDE 40

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Thin-plate splines

  • Minimizes the bending energy of a metal sheet

𝑆 𝑣 = π›Όπ‘ˆπ›Όπ‘£

2

  • Corresponding covariance function

𝑙 𝑦, 𝑦′ = 1 12 2 𝑦 βˆ’ 𝑦′ 3 βˆ’ 3𝑆( 𝑦 βˆ’ 𝑦′ 2 + 𝑆3 where 𝑆 = max

𝑦,π‘¦β€²βˆˆΞ© ‖𝑦 βˆ’ 𝑦′‖ Rohr, Karl, et al. "Landmark-based elastic registration using approximating thin-plate splines." IEEE Transactions on

medical imaging 20.6 (2001): 526-534. Williams, Oliver and Fitzgibbon Andrew, β€œGaussian process implicit surfaces”

slide-41
SLIDE 41

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

B-Splines

  • We can build a covariance function from B-Spline basis functions 𝛾

(𝑑 is a scaling constant) 𝑙 𝑦, 𝑧 = ෍

π‘™βˆˆβ„€π‘’

𝛾 𝑑𝑦 βˆ’ 𝑙 𝛾 𝑑𝑧 βˆ’ 𝑙

  • Corresponding deformation model often called β€œfree form deformations”
  • Rueckert, Daniel, et al. "Nonrigid registration using free-form deformations: application to breast MR images."

IEEE transactions on medical imaging 18.8 (1999): 712-721.

  • Klein, Stefan, et al. "Elastix: a toolbox for intensity-based medical image registration." IEEE transactions on

medical imaging 29.1 (2010): 196-205.

slide-42
SLIDE 42

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Many standard models for registration can be formulated using Gaussian processes

  • Yields probabilistic interpretation
  • We can sample and visualize deformation fields
  • Can use them as building blocks for more complicated priors
slide-43
SLIDE 43

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

  • 1. 𝑙 𝑦, 𝑦′ = 𝑔 𝑦 𝑔 𝑦’ π‘ˆ, 𝑔: π‘Œ β†’ ℝ𝑒
  • 2. 𝑙 𝑦, 𝑦′ = 𝛽𝑙1 𝑦, 𝑦′ , 𝛽 ∈ ℝ+

(scaling)

  • 3. k 𝑦, 𝑦′ = πΆπ‘ˆπ‘™1 𝑦, 𝑦′ 𝐢, B ∈ ℝ𝑠×𝑒 (lifting)
  • 4. 𝑙 𝑦, 𝑦′ = 𝑙1 𝑦, 𝑦′ + 𝑙2 𝑦, 𝑦′

(or relationship)

  • 5. 𝑙 𝑦, 𝑦′ = 𝑙1 𝑦, 𝑦′ β‹… 𝑙2(𝑦, 𝑦′)

(and relationship)

Constructing s.p.d. kernels

slide-44
SLIDE 44

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Multi-scale kernels

Add kernels that act on different scales: 𝑙 𝑦, 𝑦′ = ෍

𝑗=0 π‘œ

෍

π‘™βˆˆβ„€π‘’

𝛾 2βˆ’π‘—π‘¦ βˆ’ 𝑙 𝛾 2βˆ’π‘—π‘§ βˆ’ 𝑙

  • Wavelet like multiscale representation

Opfer, Roland. "Multiscale kernels." Advances in computational mathematics 25.4 (2006): 357-380.

slide-45
SLIDE 45

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Multi-scale kernel

slide-46
SLIDE 46

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Anisotropic priors

Scale deformations differently in each direction k 𝑦, 𝑦′ = π‘†π‘ˆ 𝑑1 𝑑2 𝑙 𝑦, 𝑦′ 𝑑1 𝑑2 𝑆

  • R is a rotation matrix
  • 𝑙 is scalar valued
  • 𝑑1, s2 scaling factors
slide-47
SLIDE 47

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Anisotropic priors

slide-48
SLIDE 48

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Spatially-varying priors

Use different models for different regions 𝑙 𝑦, 𝑦′ = πœ“ 𝑦 πœ“ 𝑦′ 𝑙1 𝑦, 𝑦′ + 1 βˆ’ πœ“ 𝑦 (1 βˆ’ πœ“ 𝑦′ ) 𝑙2(𝑦, 𝑦′) Ο‡ 𝑦 = α‰Š1 if 𝑦 ∈ thumb region

  • therwise

πœ“ 𝑦 = 1 πœ“ 𝑦 = 0 Freiman, Moti, Stephan D. Voss, and Simon K. Warfield. "Demons registration with local affine adaptive regularization: application to registration of abdominal structures." Biomedical Imaging: From Nano to Macro, 2011 IEEE International Symposium on. IEEE, 2011.

slide-49
SLIDE 49

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Spatially-varying priors

slide-50
SLIDE 50

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Statistical deformation models

Estimate mean and covariance function from data:

𝜈 𝑦 = 𝑣 𝑦 = 1 π‘œ ෍

π‘—βˆ’1 π‘œ

𝑣𝑗 (𝑦) 𝑙𝑇𝑁 𝑦, 𝑦′ = 1 π‘œ βˆ’ 1 ෍

𝑗 π‘œ

(𝑣𝑗 𝑦 βˆ’ 𝑣(𝑦)) 𝑣𝑗 𝑦′ βˆ’ 𝑣(𝑦′)

π‘ˆ

𝑣1 ∢ Ξ© β†’ ℝ2 𝑣2 ∢ Ξ© β†’ ℝ2 … π‘£π‘œ ∢ Ξ© β†’ ℝ2

slide-51
SLIDE 51

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Example 5: Statistical deformation models

slide-52
SLIDE 52

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Given:

  • Gaussian process: 𝑣 ∼ 𝐻𝑄(𝜈, 𝑙)
  • Observations: {(π‘šπ‘—

𝑆, ΰ·€

𝑣𝑗), 𝑗 = 1 , … , π‘œ} Assume: ΰ·€ 𝑣𝑗 = 𝑣 π‘šπ‘— + πœ— with πœ— ∼ 𝑂(0, 𝜏2𝐽2Γ—2). Goal:

  • Find posterior distribution

𝑣 | π‘š1

𝑆, … , π‘šπ‘œ 𝑆, ΰ·€

𝑣1, … , ΰ·€ π‘£π‘œ

π‘£π‘œ

Landmark registration using GP Regression

𝑣1 π‘šπ‘†

1

π‘šπ‘†

π‘œ

slide-53
SLIDE 53

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

πœˆπ‘ž(𝑦) = 𝜈 𝑦 + 𝐿 𝑦, 𝑍 (𝐿 𝑍, 𝑍 + 𝜏2𝐽2π‘œΓ—2π‘œ )βˆ’1 ΰ·₯ 𝒗 βˆ’ 𝜈(𝑍) π‘™π‘ž 𝑦, 𝑦′ = 𝑙 𝑦, 𝑦′ βˆ’ 𝐿 𝑦, 𝑍 (𝐿 𝑍, 𝑍 + 𝜏2𝐽2π‘œΓ—2π‘œ )βˆ’1𝐿(𝑍, 𝑦′)

The posterior 𝑣 |π‘š1

𝑆, … , π‘šπ‘œ 𝑆, ΰ·€

𝑣1, … , π‘œ is a Gaussian process

𝐻𝑄 πœˆπ‘ž, π‘™π‘ž

Its parameters are known analytically.

Gaussian process regression

slide-54
SLIDE 54

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Landmark registration using GP Regression

slide-55
SLIDE 55

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Hybrid registration

  • We can now combine landmark registration with intensity:
  • 1. Use Gaussian process regression to obtain posterior from 𝐻𝑄(𝜈, 𝑙) from landmarks
  • 2. Use 𝐻𝑄(πœˆπ‘ž, π‘™π‘ž) as new prior model for registration

WΓΆrz, Stefan, and Karl Rohr. "Hybrid spline-based elastic image registration using analytic solutions of the navier equation." Bildverarbeitung fΓΌr die Medizin 2007. Springer Berlin Heidelberg, 2007. 151-155. Lu, Huanxiang, Philippe C. Cattin, and Mauricio Reyes. "A hybrid multimodal non-rigid registration of MR images based on diffeomorphic demons." Engineering in Medicine and Biology Society (EMBC), 2010 Annual International Conference of the IEEE. IEEE, 2010.

slide-56
SLIDE 56

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Demo: Priors and interactive registration

slide-57
SLIDE 57

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

Skull Segmentation in MRI

Lab-meeting

Slides by Patrick Kahr

slide-58
SLIDE 58

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

  • Mathematical Morphology

(Dogdas et al. 2005)

  • Multi-level model fitting (Lerch, LΓΌthi 2008)
  • Multi-Atlas matching (Torrado-

Carvajal et al. 2015)

  • Mathematical Morphology

(Dogdas et al. 2005) Problem: Bone and air have similar intensities in MRI β†’ unlike CT, no threshold segmentation possible

Skull Segmentation in MRI

slide-59
SLIDE 59

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

SSM sample

deformation field defined on reference mesh

Model deformations of reference shape using an SSM 𝐻𝑄(πœˆπ‘‘π‘‘π‘›, 𝑙𝑑𝑑𝑛) Deformations are only defined on surface of reference shape: β†’ deformation field needs to be interpolated for the rest of the image.

Modeling deformations with SSMs

slide-60
SLIDE 60

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

sample

deformation field defined on complete image domain

SSM + deformation model Hybrid kernel: mix SSM with smooth Gaussian kernel (1 βˆ’ π‘₯ 𝑦 )π‘™π‘•π‘π‘£π‘‘π‘‘π‘—π‘π‘œ 𝑦, 𝑦′ (1 βˆ’ π‘₯ 𝑦′ ) + π‘₯ 𝑦 𝑙𝑇𝑇𝑁 𝑦, 𝑦′ π‘₯ 𝑦′ where x’, y’: closest points to x,y on the surface, w = 1 if x, y on the surface, wβ†’0 for x, y far away from surface.

Modeling deformations with SSMs

slide-61
SLIDE 61

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel 68

slide-62
SLIDE 62

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Register 14 CT templates to 12 MR targets (168 registrations). Measure average distance to a set of 10 anatomical landmarks.

Registration accuracy: KGaussian vs KHybrid

slide-63
SLIDE 63

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Multi-atlas matching: Target segmentation is mean shape obtained from 14 CT registrations.

Mean shape

Segmentation accuracy

slide-64
SLIDE 64

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Segmentation example

slide-65
SLIDE 65

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Segmentation accuracy

slide-66
SLIDE 66

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Lower variance between registrations for each of the 12 targets with Khybrid.

KGaussian KHybrid

Variance

slide-67
SLIDE 67

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Summary

  • GPs provide probabilistic interpretation to classic registration models
  • But, can visualize assumptions
  • New ways to combine priors to individual applications.
  • Modelling and model fitting are separated
  • Change in prior does not lead to change in algorithm
  • No increase in complexity
  • Can tailor model to application without

increase in complexity

  • Can use SSMs of individual organs to

guide image registration