Probabilistic model fitting Marcel Lthi Graphics and Vision - PowerPoint PPT Presentation

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Probabilistic model fitting Marcel Lüthi Graphics and Vision Research Group Department of Mathematics and Computer Science University of Basel

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Reminder: Registration as analysis by synthesis Comparison: 𝑞 𝐽 𝑈 𝜄, 𝐽 𝑆 ) Prior 𝜒[𝜄] ∼ 𝑞(𝜄) 𝐽 𝑈 𝐽 𝑆 ∘ 𝜒[𝜄] Parameters 𝜄 Update using 𝑞(𝜄|𝐽 𝑈 , 𝐽 𝑆 ) Synthesis 𝜒[𝜄]

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Reminder: Priors Gaussian process 𝑣 ∼ 𝐻𝑄 𝜈, 𝑙 Represented using first 𝑠 components 𝑠 𝑣 = 𝜈 + ෍ 𝛽 𝑗 𝜇 𝑗 𝜚 𝑗 , 𝛽 𝑗 ∼ 𝑂(0, 1) 𝑗=1 Different GP-s lead to very different deformation models • All of them are parametric 𝑣 ∼ 𝑞(𝜄) .

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Reminder: Likelihood functions Likelihood function: 𝑞 𝐽 𝑈 𝜄, 𝐽 𝑆 ) Surface / surface Image / image Comparison Landmarks / Surface / image landmark Deviation of image (Distance to) exact Stochastic component Noise on landmark Intensity profiles at intensity from position of surface points surface boundary reference image

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Reminder: Obtaining the posterior parameters MAP-Estimate 𝜄 ∗ = arg max 𝑞 𝜄 𝐽 𝑈 , 𝐽 𝑆 = arg max 𝑞 𝜄 𝑞(𝐽 𝑈 |𝜄, 𝐽 𝑆 ) 𝜄 𝜄 MAP Solution 𝜄 ∗ = arg max 𝑞 𝜄 𝑞(𝐽 𝑈 |𝜄, 𝐽 𝑆 ) 𝜄 𝑞(𝜄|𝐽 𝑈 , 𝐽 𝑆 ) 𝜄 - Solving an optimization problem

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Todays Lecture: Obtaining the posterior distribution Full posterior distribution 𝑞 𝜄 𝐽 𝑈 , 𝐽 𝑆 = 𝑞 𝜄 𝑞 𝐽 𝑈 𝜄, 𝐽 𝑆 𝑞(𝐽 𝑈 ) Infeasible to compute: 𝑞(𝜄|𝐽 𝑈 , 𝐽 𝑆 ) p (𝐽 𝑈 ) = ∫ 𝑞 𝜄 𝑞 𝐽 𝑈 𝜄 𝑒𝜄 𝑞 𝜄 𝑞 𝐽 𝑈 𝜄, 𝐽 𝑆 𝑞(𝐽 𝑈 ) 𝜄 - Doing (approximate) Bayesian inference

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Outline • Basic idea: Sampling methods and MCMC • The Metropolis-Hastings algorithm • The Metropolis algorithm • Implementing the Metropolis algorithm • The Metropolis-Hastings algorithm • Example: 3D Landmark fitting

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Approximate Bayesian Inference Samplin Sa ing methods Varia iati tional l meth thods • Numeric approximations through • Function approximation 𝑟(𝜄) simulation arg max KL(𝑟(𝜄)|𝑞(𝜄|𝐸)) 𝑟 KL: Kullback- 𝑞 𝑞 Leibler divergence 𝜄 𝜄

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Sampling Methods • Simulate a distribution 𝑞 through random samples 𝑦 𝑗 • Evaluate expectation (of some function 𝑔 of random variable 𝑌 ) 𝐹 𝑔(𝑌) = න 𝑔 𝑦 𝑞 𝑦 𝑒𝑦 𝑂 𝑔 = 1 𝐹 𝑔(𝑌) ≈ መ 𝑞 𝑂 ෍ 𝑔 𝑦 𝑗 , 𝑦 𝑗 ~ 𝑞 𝑦 𝑗 1 𝑊 መ 𝑔(𝑌) ~ 𝑃 This is s dif diffic icult! 𝑂 • “Independent” of dimensionality of 𝑌 • More samples increase accuracy 𝜄

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Sampling from a Distribution • Easy for standard distributions … is it? Random.nextDouble() Random.nextGaussian() • Uniform • Gaussian • How to sample from more complex distributions? • Beta, Exponential, Chi square, Gamma, … • Posteriors are very often not in a “nice” standard text book form • We need to sample from an unknown posterior with only unnormalized, expensive point- wise evaluation  10

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Markov Chain Monte Carlo Markov Chain Monte Carlo Methods (MCMC) Idea: Design a Markov Chain such that samples 𝑦 obey the target distribution 𝑞 Concept: “Use an already existing sample to produce the next one” • Many successful practical applications • Proven: developed in the 1950/1970ies (Metropolis/Hastings) • Direct mapping of computing power to approximation accuracy 11

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE MCMC: An ingenious mathematical construction … an aperiodic and irreducable induces If Markov Markov chain MCMC Algorithms Chain is a- periodic and irreducable converges to it … Generate samples from Equilibrium is Distribution 𝑞(𝑦) distribution No need to understand this now: more details follow!

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE The Metropolis Algorithm Requirements: • Proposal distribution 𝑅(𝒚 ′ |𝒚) – must generate samples, symmetric • Target distribution 𝑄 𝒚 – with point-wise evaluation Result: • Stream of samples approximately from 𝑄 𝒚 • Initialize with sample 𝒚 • Generate next sample, with current sample 𝒚 Draw a sample 𝒚 ′ from 𝑅(𝒚 ′ |𝒚) (“proposal”) 1. 𝑄 𝒚 ′ accept 𝒚 ′ as new state 𝒚 With probability 𝛽 = min 𝑄 𝒚 , 1 2. 3. Emit current state 𝒚 as sample 13

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

Ƹ University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Example: 2D Gaussian Σ 𝑓 − 1 2 𝒚−𝝂 𝑈 Σ −1 (𝒚−𝝂) 1 • Target: 𝑄 𝒚 = 2𝜌 𝑅 𝒚 ′ 𝒚 = 𝒪(𝒚 ′ | 𝒚, 𝜏 2 𝐽 2 ) • Proposal: Random walk Target Sampled Estimate 𝜈 = 1.5 𝜈 = 1.56 1.5 1.68 Σ = 1.09 0.63 Σ = 1.25 0.75 ෠ 0.75 1.25 0.63 1.07 15

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE 2D Gaussian: Different Proposals 𝜏 = 0.2 𝜏 = 1.0 16

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE The Metropolis-Hastings Algorithm • Initialize with sample 𝒚 • Generate next sample, with current sample 𝒚 Draw a sample 𝒚 ′ from 𝑅(𝒚 ′ |𝒚) (“proposal”) 1. 𝑄 𝑦 ′ 𝑅 𝑦|𝑦 ′ 𝑅 𝑦 ′ |𝑦 , 1 accept 𝒚 ′ as new state 𝒚 With probability 𝛽 = min 2. 𝑄 𝑦 Emit current state 𝒚 as sample 3. • Generalization of Metropolis algorithm to asymmetric Proposal distribution 𝑅 𝒚 ′ 𝒚 ≠ 𝑅 𝒚 𝒚 ′ 𝑅 𝒚 ′ 𝒚 > 0 ⇔ 𝑅 𝒚 𝒚 ′ > 0 17

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Properties • Approximation: Samples 𝑦 1 , 𝑦 2 , … approximate 𝑄(𝑦) Unbiased but correlated (not i.i.d. ) • No Normalization: 𝑄(𝑦) does not need to be normalized Algorithm only considers ratios 𝑄(𝑦′)/𝑄(𝑦) ls: 𝑅 𝑦 ′ 𝑦 depends on current sample 𝑦 • De Dependent Proposals Algorithm adapts to target with simple 1-step memory

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Metropolis - Hastings: Limitations • Highly correlated targets • Serial correlation • Results from rejection Proposal should match target to avoid too many rejections and too small stepping • Subsampling Bishop. PRML, Springer, 2006 19

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Propose-and-Verify Algorithm • Metropolis algorithm formalizes: propose-and-verify • Steps are completely independent. Propose Draw a sample 𝑦 ′ from 𝑅(𝑦 ′ |𝑦) Veri rify fy 𝑄 𝑦 ′ 𝑅 𝑦|𝑦 ′ accept 𝒚 ′ as new sample With probability 𝛽 = min 𝑅 𝑦 ′ |𝑦 , 1 𝑄 𝑦 20

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE MH as Propose and Verify • Decouples the steps of finding the solution from validating a solution • Natural to integrate uncertain proposals Q (e.g. automatically detected landmarks, ...) • Possibility to include “local optimization” (e.g. a ICP or ASM updates, gradient step, …) as proposal Anything more “informed” than random walk should improve convergence.

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Fitting 3D Landmarks 3D Alignment with Shape and Pose 22

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE 3D Fitting Example right.eye.corner_outer left.eye.corner_outer right.lips.corner left.lips.corner 23

University of Basel > DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE 3D Fitting Setup Goa oal: Find posterior distribution for arbitrary pose and shape Shape transformation Observations 𝑠 • Observed positions 𝑚 𝑈 1 , … , 𝑚 𝑈 𝑜 𝜒 𝑡 𝛽 = 𝜈 𝑦 + ෍ 𝛽 𝑗 𝜇 𝑗 𝛸 𝑗 (𝑦) 𝑜 • Correspondence: 𝑚 𝑆 1 , … , 𝑚 𝑆 𝑗=1 Parameters Rigid transformation • 3 angles (pitch, yaw, roll) 𝜒, 𝜔, 𝜘 𝜄 = 𝛽, 𝜒, 𝜔, 𝜘, 𝑢 • Translation 𝑢 = (𝑢 𝑦 , 𝑢 𝑧 , 𝑢 𝑨 ) Posterior distribution: 𝑜 ∝ 𝑞 𝑚 𝑈 1 , … , 𝑚 𝑈 1 , … , 𝑚 𝑈 𝑆 |𝜄 𝑄(𝜄) 𝑄 𝜄 𝑚 𝑈 𝜒 𝑆 𝜒, 𝜔, 𝜘, 𝑢 = 𝑆 𝜘 𝑆 𝜔 𝑆 𝜒 𝒚 + 𝑢 Full transformation 𝜒 𝜄 (𝑦) = (𝜒 𝑆 ∘ 𝜒 𝑇 )[𝜄](𝑦) 24