Gaussian processes for non-rigid id regis istration - Con - PowerPoint PPT Presentation

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Gaussian processes for non-rigid id regis istration - Con onnections to o medical im image an analysis is Marcel Lüthi Graphics and Vision Research Group Department of Mathematics and Computer Science University of Basel

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Aims of the talk • Show how Analysis by Synthesis and Gaussian processes lead to a family of methods for non-rigid registration • Provide an understanding of many common algorithms in terms of Gaussian processes • Show how to derive new registration approaches using GPMMs and MCMC

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Outline • The registration problem • Likelihood functions • Problem formulation • Landmark registration • Registration as analysis by synthesis • Image to image registration problem • Surface to image registration • An algorithm using Gaussian process priors • Advancing registration • Priors for registration • Spline-based models, Radial basis functions • Multis-scale and Spatially-varying models • Statistical deformation models

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Outline • The registration problem • Likelihood functions • Problem formulation • Landmark registration • Registration as analysis by synthesis • Image to image registration problem • Surface to image registration • An algorithm using Gaussian process priors • Some ideas where to go from here • Priors for registration • Spline-based models 𝑦 • Radial basis functions • Statistical deformation models

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL The registration problem 𝑦 𝜒: Ω → Ω Ω Ω 𝜒(𝑦) Reference: Target: 𝐽 𝑆 : Ω → ℝ 𝐽 𝑈 : Ω → ℝ 5

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL The registration problem Variational formulation 𝜒 ∗ = arg min 𝜒 𝐸 𝐽 𝑆 , 𝐽 𝑈 ∘ 𝜒 + 𝜇𝑆[𝜒] Mapping 𝜒 ∗ is trade-off that • makes the images look similar (for similarity measure 𝐸 ) • matches the prior assumptions (encoded by regularizer R)

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL The registration problem Variational formulation 𝜒 ∗ = arg min 𝜒 𝐸 𝐽 𝑆 , 𝐽 𝑈 ∘ 𝜒 + 𝜇𝑆[𝜒] 𝜒 Ω Ω 7

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL The registration problem Variational formulation 𝜒 ∗ = arg min 𝜒 𝐸 𝐽 𝑆 , 𝐽 𝑈 ∘ 𝜒 + 𝜇𝑆[𝜒] 𝜒 Ω Ω 8

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL The registration problem Variational formulation 𝜒 ∗ = arg min 𝜒 𝐸 𝐽 𝑆 , 𝐽 𝑈 ∘ 𝜒 + 𝜇𝑆[𝜒] 𝜒 Ω Ω 9

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Representation of the mapping 𝜒

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Representation of the mapping 𝜒

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Representation of the mapping 𝜒 Assumption: Images are rigidly aligned • Mapping can be represented as a displacement vector field: 𝑦 u( 𝑦) 𝜒 𝑦 = 𝑦 + 𝑣 𝑦 𝑣 ∶ Ω → ℝ 𝑒

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Representation of the mapping 𝜒 Assumption: Images are rigidly aligned • Mapping can be represented as a displacement vector field: 𝜒 𝑦 = 𝑦 + 𝑣 𝑦 𝑣 ∶ Ω → ℝ 𝑒 Further assumption: • 𝜒 is parametric : 𝜒 𝜄 𝑦 = 𝑦 + 𝑣[𝜄](𝑦)

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Representation of the mapping 𝜒 Mapping: 𝜒 𝜄 𝑦 = 𝑦 + 𝑣[𝜄](𝑦) Observation: • Knowledge of 𝜄 and 𝐽 𝑆 allows us to synthesize target image 𝐽 𝑈 • (at least up to intensity differences)

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Registration as analysis by synthesis Comparison: 𝑞 𝐽 𝑈 𝜄, 𝐽 𝑆 ) Prior 𝜒[𝜄] ∼ 𝑞(𝜄) 𝐽 𝑈 𝐽 𝑆 ∘ 𝜒[𝜄] Parameters 𝜄 Update using 𝑞(𝜄|𝐽 𝑈 , 𝐽 𝑆 ) Synthesis 𝜔(𝜄)

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Probabilistic formulation of registration 𝑄 𝐽 𝑈 |𝜒[𝜄],𝐽 𝑆 𝑄 𝜒[𝜄],𝐽 𝑆 Using Bayes rule: 𝑄 𝜒[𝜄]|𝐽 𝑈 , 𝐽 𝑆 = 𝑄 𝐽 𝑈 MAP solution 𝜄 ∗ = arg max 𝑞 𝜒[𝜄] 𝐽 𝑈 , 𝐽 𝑆 = arg max 𝑞 𝜒 𝜄 𝑞(𝐽 𝑈 |𝐽 𝑆 ∘ 𝜒[𝜄]) 𝜄 𝜄 Mapping 𝜄 ∗ is trade-off that defines a mapping 𝜒[𝜄 ∗ ] which • explains the data well (likelihood function) • matches the prior assumptions (prior distribution)

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Registration problem 𝜒 ∗ = arg max 𝑞 𝜒 𝜄 𝑞(𝐽 𝑈 |𝐽 𝑆 ∘ 𝜒[𝜄]) 𝜄 = arg max ln 𝑞(𝜒 𝜄 ) + ln(𝑞 𝐽 𝑈 𝐽 𝑆 ∘ 𝜒[𝜄] ) 𝜄 = arg min 𝜄 − ln 𝑞 𝜒 𝜄 − ln(𝑞 𝐽 𝑈 𝐽 𝑆 ∘ 𝜒[𝜄] )

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL The registration problem Variational formulation 𝜒 ∗ = arg min 𝜒 𝐸 𝐽 𝑈 , 𝐽 𝑆 ∘ 𝜒 + 𝜇𝑆[𝜒] Probabilistic formulation 𝜄 ∗ = arg min 𝜄 − ln 𝑞 𝐽 𝑈 𝐽 𝑆 ∘ 𝜒 𝜄 − ln 𝑞 𝜒 𝜄 Take home message: Registration is model fitting!!!

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Gaussian processes Define the Gaussian process 𝑣 ∼ 𝐻𝑄 𝜈, 𝑙 with mean function 𝜈: Ω → ℝ 2 and covariance function 𝑙: Ω × Ω → ℝ 2×2 .

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Parametric representation of Gaussian process Represent GP using only the first 𝑠 components of its KL-Expansion 𝑠 𝑣 = 𝜈 + ෍ 𝛽 𝑗 𝜇 𝑗 𝜚 𝑗 , 𝛽 𝑗 ∼ 𝑂(0, 1) 𝑗=1 • We have a finite, parametric representation of the process. • We know the pdf for a deformation 𝑣 𝑠 1 2 /2) = 1 𝑎 exp(− 1 2 𝛽 2 ) 𝑞 𝑣 = 𝑞 𝛽 = ෑ exp(−𝛽 𝑗 2𝜌 𝑗=1

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Registration problem 𝜒 ∗ = arg min 𝜄 − ln 𝑞 𝜒 𝜄 − ln 𝑞(𝐽 𝑈 |𝐽 𝑆 ∘ 𝜒[𝜄]) 𝜄 −ln 1 Z exp(− 1 2 𝜄 2 ) − ln(𝑞 𝐽 𝑈 𝐽 𝑆 ∘ 𝜒[𝜄] = arg min −ln 1 𝑎 + 1 2 𝜄 2 − ln(𝑞 𝐽 𝑈 𝐽 𝑆 ∘ 𝜒[𝜄] = arg min 𝜄 1 2 𝜄 2 − ln(𝑞 𝐽 𝑈 𝐽 𝑆 ∘ 𝜒[𝜄] = arg min 𝜄

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Summary: registration problem 1 2 𝜄 2 + ln(𝑞 𝐽 𝑈 𝐽 𝑆 ∘ 𝜒[𝜄] ) arg min 𝜄 • Variational and probabilistic formulation are closely related • Prior can be seen as regularizer • Likelihood term is an image similarity • For a low-rank Gaussian process prior, the problem becomes parametric since 𝑠 𝜒[𝜄](𝑦) = 𝑦 + 𝜈(𝑦) + ෍ 𝜄 𝑗 𝜇 𝑗 𝜚 𝑗 (𝑦) 𝑗=1 • Can be optimized using gradient-descent schemes. • All the regularization assumptions are encoded in the eigenfunctions 𝜚 𝑗

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Outline • The registration problem • Likelihood functions • Problem formulation • Landmark registration • Registration as analysis by synthesis • Image to image registration problem • Surface to image registration • An algorithm using Gaussian process priors • Advancing registration • Priors for registration • Spline-based models, Radial basis functions • Multis-scale and Spatially-varying models • Statistical deformation models

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Why are priors interesting? 𝜄 ∗ = arg max 𝑞 𝜒 𝜄 𝑞(𝐽 𝑈 |𝐽 𝑆 ∘ 𝜒[𝜄]) 𝜄 𝜄

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Why are priors interesting? 𝜄 ∗ = arg max 𝑞 𝜒 𝜄 𝑞(𝐽 𝑈 |𝐽 𝑆 ∘ 𝜒[𝜄]) 𝜄 𝜄

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Defining a Gaussian process A Gaussian process 𝐻𝑄 𝜈, 𝑙 ex is completely specified by a mean function 𝜈 and covariance function (or kernel) 𝑙 . • 𝜈: Ω → ℝ 𝑒 defines how the average deformation looks like • 𝑙: Ω × Ω → ℝ 𝑒×𝑒 defines how it can deviate from the mean • Must be positive semi-definite

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL The mean function • Usual assumption: 𝜈 1 (𝑦) 0 ⋮ 𝜈 𝑦 = = ⋮ 𝜈 𝑒 (𝑦) 0 • The reference shape is an average shape.

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Scalar-valued Gaussian kernel 𝑙 𝑦, 𝑦 ′ = 𝑡 exp(− 𝑦 − 𝑦 ′ 2 ) 𝜏 2 𝑡 = 1, 𝜏 = 3 𝑡 = 1, 𝜏 = 5 𝑡 = 2, 𝜏 = 3