Registration Deformation models Marcel Lthi Graphics and Vision - PowerPoint PPT Presentation

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Registration – Deformation models Marcel Lüthi Graphics and Vision Research Group Department of Mathematics and Computer Science University of Basel

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Registration as analysis by synthesis Comparison: 𝑞 𝐽 𝑈 𝜄, 𝐽 𝑆 ) Prior 𝜒[𝜄] ∼ 𝑞(𝜄) 𝐽 𝑈 𝐽 𝑆 ∘ 𝜒[𝜄] Parameters 𝜄 Update using 𝑞(𝜄|𝐽 𝑈 , 𝐽 𝑆 ) Synthesis 𝜒[𝜄]

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Priors Define the Gaussian process 𝑣 ∼ 𝐻𝑄 𝜈, 𝑙 with mean function Characteristics 𝜈: Ω → ℝ 2 of deformation fields and covariance function 𝑙: Ω × Ω → ℝ 2×2 .

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Example prior: Smooth 2D deformations Zero mean: 𝜈 𝑦 = 0 0 Squared exponential covariance function (Gaussian kernel) s 1 exp − 𝑦 − 𝑦 ′ 2 0 2 𝜏 1 𝑙 𝑦, 𝑦 ′ = s 2 exp − 𝑦 − 𝑦 ′ 2 0 2 𝜏 2

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Example prior: Smooth 2D deformations 𝑡 1 = 𝑡 2 small, 𝜏 1 = 𝜏 2 large

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Example prior: Smooth 2D deformations 𝑡 1 = 𝑡 2 small, 𝜏 1 = 𝜏 2 small

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Example prior: Smooth 2D deformations 𝑡 1 = 𝑡 2 large, 𝜏 1 = 𝜏 2 large

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Why are priors interesting? 𝜄 ∗ = arg max 𝑞 𝜒 𝜄 𝑞(𝐽 𝑈 |𝐽 𝑆 , 𝜒[𝜄]) 𝜄 𝜄

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Why are priors interesting? 𝜄 ∗ = arg max 𝑞 𝜒 𝜄 𝑞(𝐽 𝑈 |𝐽 𝑆 , 𝜒[𝜄]) 𝜄 𝜄

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Intermezzo – The space of samples 10

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Gaussian processes - Deeper Insights

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Scalar-valued Gaussian processes Sc Scalar-valu lued (m (more common) Vector-valu lued (th (this is cou ourse) • Samples f are real-valued functions • Samples u are deformation fields: 𝑔 ∶ ℝ 𝑜 → ℝ 𝑣: ℝ 𝑜 → ℝ 𝑒

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE The space of samples 𝑣 ∼ 𝜈 + σ 𝑗 𝛽 𝑗 𝜇 𝑗 𝜚 𝑗 = σ 𝑗 𝛾 𝑗 𝑙(𝑦 𝑗 ,⋅) for some 𝛾 Argument: • Covariance function 𝑙 is symmetric and positive definite • For any finite sample it holds that: => the covariance matrix is symmetric => rowspace = columnspace = eigenspace Samples are linear combinations of the “ row s” of 𝑙 18

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Example: Gaussian kernel • Click to edit Master text styles 𝑙 𝑦, 𝑦 ′ = exp − 𝑦 − 𝑦 ′ 2 𝜏 2 • Second level • Third level • Fourth level • Fifth level 19

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Example: Gaussian kernel 𝑙 𝑦, 𝑦 ′ = exp − 𝑦 − 𝑦 ′ 2 𝜏 2 σ = 3 20

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Multi-scale signals 2 2 𝑦 ′ 𝑦 ′ • k x, x ′ = exp − 𝑦 − + 0.1 exp − 𝑦 − 1 0.1 21

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Periodic kernels cos 𝑦 • Define 𝑣 𝑦 = sin(𝑦) ‖𝑦 −𝑦 ′ ‖ • 𝑙 𝑦, 𝑦 ′ = exp(−‖(𝑣 𝑦 − 𝑣 𝑦 ′ ‖ 2 = exp(−4 sin 2 ) 𝜏 2 22

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Symmetric kernels • Enforce that f(x) = f(-x) • 𝑙 𝑦, 𝑦 ′ = 𝑙 −𝑦, 𝑦 ′ + 𝑙(𝑦, 𝑦 ′ ) 23

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Changepoint kernels • 𝑙 𝑦, 𝑦 ′ = 𝑡 𝑦 𝑙 1 𝑦, 𝑦 ′ 𝑡 𝑦 ′ + (1 − 𝑡 𝑦 )𝑙 2 (𝑦, 𝑦 ′ )(1 − 𝑡 𝑦 ′ ) 1 • s 𝑦 = 1+exp( −𝑦) 24

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Combining existing functions 𝑙 𝑦, 𝑦 ′ = 𝑔 𝑦 𝑔 𝑦 ′ f x = x 25

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Combining existing functions 𝑙 𝑦, 𝑦 ′ = 𝑔 𝑦 𝑔 𝑦 ′ f x = sin(x) 26

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Combining existing functions 𝑙 𝑦, 𝑦 ′ = ෍ 𝑗 (𝑦 ′ ) 𝑔 𝑗 𝑦 𝑔 𝑗 {f 1 x = x, f 2 x = sin(x)} 27

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Statistical models … 𝑣 𝑜 ∶ Ω → ℝ 2 𝑣 1 ∶ Ω → ℝ 2 𝑣 2 ∶ Ω → ℝ 2 𝑜 𝜈 𝑦 = 𝑣 𝑦 = 1 𝑣 𝑗 (𝑦) 𝑜 ෍ 𝑗−1 𝑜 1 𝑈 𝑙 𝑇𝑁 𝑦, 𝑦 ′ = (𝑣 𝑗 𝑦 − 𝑣(𝑦)) 𝑣 𝑗 𝑦′ − 𝑣(𝑦′) 𝑜 − 1 ෍ 𝑗 Statistical shape models are linear combinations of example deformations 𝑣 1 , … 𝑣 𝑜 .

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Gaussian process regression • Given: observations {(𝑦 1 , 𝑧 1 ), … , 𝑦 𝑜 , 𝑧 𝑜 } • Model: 𝑧 𝑗 = 𝑔 𝑦 𝑗 + 𝜗, 𝑔 ∼ 𝐻𝑄(𝜈, 𝑙) • Goal: compute p( 𝑧 ∗ |𝑦 ∗ , 𝑦 1 , … , 𝑦 𝑜 , 𝑧 1 , … , 𝑧 𝑜 ) 𝑧 𝑜 𝑧 1 𝑧 ∗ 𝑧 2 𝑦 𝑜 𝑦 1 𝑦 2 29 𝑦 ∗

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Gaussian process regression • Solution given by posterior process 𝐻𝑄 𝜈 𝑞 , 𝑙 𝑞 with 𝜈 𝑞 (𝑦 ∗ ) = 𝐿 𝑦 ∗ , 𝑌 𝐿 𝑌, 𝑌 + 𝜏 2 𝐽 −1 𝑧 − 𝐿 𝑦 ∗ , 𝑌 𝐿 𝑌, 𝑌 + 𝜏 2 𝐽 −1 𝐿 𝑌, 𝑦 ∗ ′ 𝑙 𝑞 𝑦 ∗ , 𝑦 ∗ ′ = 𝑙 𝑦 ∗ , 𝑦 ∗ ′ • The covariance is independent of the value at the training points • Structure of posterior GP determined solely by kernel. • The most likely solution is a linear combination of kernels evaluated at the training points • This is known as the Rep epresenter er Th Theorem in machine learning. • Structure of solution determined solely by kernel. 30

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Illustration: Representer theorem 31

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Examples 32

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Examples • Gaussian kernel ( 𝜏 = 1) 33

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Examples • Gaussian kernel ( 𝜏 = 5) 34

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Examples • Periodic kernel 35

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Examples • Changepoint kernel 36

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Examples • Symmetric kernel 37

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Examples • Linear kernel 38

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Deformation models for registration 39

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Basic assumption: Deformation fields are smooth • Typical assumption: • Deformation field is smooth • GP approach • Choose smooth kernel functions 𝑙 𝑦, 𝑦 ′ = 𝑡 exp(− 𝑦 − 𝑦 ′ 2 ) 𝜏 2 • Regularization operators • Penalize large derivatives 𝑜 𝑆𝑣 2 = ෍ 𝛽 𝑗 𝐸 𝑗 𝑣 2 ℛ 𝑣 = 𝑗=0

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Green’s functions and covariance functions 𝑜 𝑆𝑣 2 = ෍ 𝛽 𝑗 𝐸 𝑗 𝑣 2 ℛ 𝑣 = 𝑗=0 Corresponding covariance function for GP is the Greens function G: 𝑆 ∗ 𝑆𝐻 𝑦, 𝑧 = 𝜀(𝑦 − 𝑧) • We can define Gaussian processes, which mimic typical regularization operators. T. Poggio and F. Girosi; Networks for Approximation and Learning, Proceedings of the IEEE, 1990

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Example: Gaussian kernel 𝑙 𝑦, 𝑦 ′ = exp(− 𝑦 − 𝑦 ′ 2 ) 𝜏 2 ∞ 𝜏 2𝑗 𝑆𝑣 2 = ෍ 𝑗! 2 𝑗 𝐸 𝑗 𝑣 2 ℛ 𝑣 = 𝑗=0 • Non-zero functions are penalized • pushes functions to zero away from data Yuille, A. and Grzywacz M. A mathematical analysis of the motion coherence theory. International Journal of Computer vision

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Example: Exponential kernel (1D case) 𝑙 𝑦, 𝑦 ′ = 1 2𝛽 exp(−𝛽 𝑦 − 𝑦 ′ ) 𝑆𝑣 2 = 𝛽 2 𝑣 + 𝐸 1 𝑣 2 ℛ 𝑣 = Rasmussen, Carl Edward, and Christopher KI Williams. Gaussian processes for machine learning . Vol. 1. Cambridge: MIT press, 2006.