Probabilistic model fitting Marcel Lthi Graphics and Vision - - PowerPoint PPT Presentation

β–Ά
probabilistic model fitting
SMART_READER_LITE
LIVE PREVIEW

Probabilistic model fitting Marcel Lthi Graphics and Vision - - PowerPoint PPT Presentation

Nov 12, 2022 263 likes β€’590 views

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Probabilistic model fitting Marcel Lthi Graphics and Vision Research Group Department of Mathematics and Computer Science University of Basel University of Basel > DEPARTMENT OF

slide-1
SLIDE 1

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

Probabilistic model fitting

Marcel LΓΌthi

Graphics and Vision Research Group Department of Mathematics and Computer Science University of Basel

slide-2
SLIDE 2

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Reminder: Registration as analysis by synthesis

Parameters πœ„

Comparison: π‘ž π½π‘ˆ πœ„, 𝐽𝑆) Update using π‘ž(πœ„|π½π‘ˆ, 𝐽𝑆) Synthesis πœ’[πœ„]

Prior πœ’[πœ„] ∼ π‘ž(πœ„) π½π‘ˆ 𝐽𝑆 ∘ πœ’[πœ„]

slide-3
SLIDE 3

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Reminder: Priors

Gaussian process 𝑣 ∼ 𝐻𝑄 𝜈, 𝑙

Represented using first 𝑠 components 𝑣 = 𝜈 + ෍

𝑗=1 𝑠

𝛽𝑗 πœ‡π‘— πœšπ‘—, 𝛽𝑗 ∼ 𝑂(0, 1) Different GP-s lead to very different deformation models

  • All of them are parametric 𝑣 ∼ π‘ž(πœ„).
slide-4
SLIDE 4

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Reminder: Likelihood functions

Noise on landmark points Intensity profiles at surface boundary Deviation of image intensity from reference image (Distance to) exact position of surface Stochastic component Likelihood function: π‘ž π½π‘ˆ πœ„, 𝐽𝑆) Comparison Landmarks / landmark Surface / image Image / image Surface / surface

slide-5
SLIDE 5

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Reminder: Obtaining the posterior parameters

MAP-Estimate πœ„βˆ— = arg max

πœ„

π‘ž πœ„ π½π‘ˆ, 𝐽𝑆 = arg max

πœ„

π‘ž πœ„ π‘ž(π½π‘ˆ|πœ„, 𝐽𝑆)

MAP Solution πœ„βˆ— = arg max

πœ„

π‘ž πœ„ π‘ž(π½π‘ˆ|πœ„, 𝐽𝑆)

πœ„ π‘ž(πœ„|π½π‘ˆ, 𝐽𝑆)

  • Solving an optimization problem
slide-6
SLIDE 6

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Todays Lecture: Obtaining the posterior distribution

Full posterior distribution π‘ž πœ„ π½π‘ˆ, 𝐽𝑆 = π‘ž πœ„ π‘ž π½π‘ˆ πœ„, 𝐽𝑆 π‘ž(π½π‘ˆ)

Infeasible to compute: p(π½π‘ˆ)= ∫ π‘ž πœ„ π‘ž π½π‘ˆ πœ„ π‘’πœ„ πœ„ π‘ž(πœ„|π½π‘ˆ, 𝐽𝑆) π‘ž πœ„ π‘ž π½π‘ˆ πœ„, 𝐽𝑆 π‘ž(π½π‘ˆ)

  • Doing (approximate) Bayesian inference
slide-7
SLIDE 7

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Outline

  • Basic idea: Sampling methods and MCMC
  • The Metropolis-Hastings algorithm
  • The Metropolis algorithm
  • Implementing the Metropolis algorithm
  • The Metropolis-Hastings algorithm
  • Example: 3D Landmark fitting
slide-8
SLIDE 8

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Varia iati tional l meth thods

  • Function approximation π‘Ÿ(πœ„)

arg max

π‘Ÿ

KL(π‘Ÿ(πœ„)|π‘ž(πœ„|𝐸)) Sa Samplin ing methods

  • Numeric approximations through

simulation

Approximate Bayesian Inference

KL: Kullback- Leibler divergence

πœ„ π‘ž πœ„ π‘ž

slide-9
SLIDE 9

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

  • Simulate a distribution π‘ž through random samples 𝑦𝑗
  • Evaluate expectation (of some function 𝑔 of random variable π‘Œ)

𝐹 𝑔(π‘Œ) = ΰΆ± 𝑔 𝑦 π‘ž 𝑦 𝑒𝑦 𝐹 𝑔(π‘Œ) β‰ˆ መ 𝑔 = 1 𝑂 ෍

𝑗 𝑂

𝑔 𝑦𝑗 , 𝑦𝑗 ~ π‘ž 𝑦

π‘Š መ 𝑔(π‘Œ) ~ 𝑃 1 𝑂

Sampling Methods

  • β€œIndependent” of dimensionality of π‘Œ
  • More samples increase accuracy

This is s dif diffic icult! πœ„ π‘ž

slide-10
SLIDE 10

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Sampling from a Distribution

  • Easy for standard distributions … is it?
  • Uniform
  • Gaussian
  • How to sample from more complex distributions?
  • Beta, Exponential, Chi square, Gamma, …
  • Posteriors are very often not in a β€œnice” standard text book form
  • We need to sample from an unknown posterior with only unnormalized, expensive point-

wise evaluation 

10

Random.nextDouble() Random.nextGaussian()

slide-11
SLIDE 11

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Markov Chain Monte Carlo

Markov Chain Monte Carlo Methods (MCMC)

Idea: Design a Markov Chain such that samples 𝑦 obey the target distribution π‘ž Concept: β€œUse an already existing sample to produce the next one”

  • Many successful practical applications
  • Proven: developed in the 1950/1970ies (Metropolis/Hastings)
  • Direct mapping of computing power to approximation accuracy

11

slide-12
SLIDE 12

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

MCMC: An ingenious mathematical construction

Markov chain Equilibrium distribution Distribution π‘ž(𝑦) MCMC Algorithms induces converges to Generate samples from is If Markov Chain is a- periodic and irreducable it… … an aperiodic and irreducable No need to understand this now: more details follow!

slide-13
SLIDE 13

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

The Metropolis Algorithm

  • Initialize with sample π’š
  • Generate next sample, with current sample π’š

1. Draw a sample π’šβ€² from 𝑅(π’šβ€²|π’š) (β€œproposal”) 2. With probability 𝛽 = min

𝑄 π’šβ€² 𝑄 π’š , 1

accept π’šβ€² as new state π’š 3. Emit current state π’š as sample

13

Requirements:

  • Proposal distribution 𝑅(π’šβ€²|π’š) – must generate samples, symmetric
  • Target distribution 𝑄 π’š – with point-wise evaluation

Result:

  • Stream of samples approximately from 𝑄 π’š
slide-14
SLIDE 14

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

slide-15
SLIDE 15

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Example: 2D Gaussian

  • Target:

𝑄 π’š =

1 2𝜌 Ξ£ π‘“βˆ’1

2 π’šβˆ’π‚ π‘ˆΞ£βˆ’1(π’šβˆ’π‚)

  • Proposal:

𝑅 π’šβ€² π’š = π’ͺ(π’šβ€²|π’š, 𝜏2𝐽2)

15

Random walk Ƹ 𝜈 = 1.56 1.68 ෠ Σ = 1.09 0.63 0.63 1.07 𝜈 = 1.5 1.5 Σ = 1.25 0.75 0.75 1.25 Sampled Estimate Target

slide-16
SLIDE 16

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

2D Gaussian: Different Proposals

16

𝜏 = 0.2 𝜏 = 1.0

slide-17
SLIDE 17

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

The Metropolis-Hastings Algorithm

  • Initialize with sample π’š
  • Generate next sample, with current sample π’š

1. Draw a sample π’šβ€² from 𝑅(π’šβ€²|π’š) (β€œproposal”) 2. With probability 𝛽 = min

𝑄 𝑦′ 𝑄 𝑦 𝑅 𝑦|𝑦′ 𝑅 𝑦′|𝑦 , 1 accept π’šβ€² as new state π’š

3. Emit current state π’š as sample

17

  • Generalization of Metropolis algorithm to asymmetric Proposal distribution

𝑅 π’šβ€² π’š β‰  𝑅 π’š π’šβ€² 𝑅 π’šβ€² π’š > 0 ⇔ 𝑅 π’š π’šβ€² > 0

slide-18
SLIDE 18

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Properties

  • Approximation: Samples 𝑦1, 𝑦2, … approximate 𝑄(𝑦)

Unbiased but correlated (not i.i.d.)

  • No

Normalization: 𝑄(𝑦) does not need to be normalized

Algorithm only considers ratios 𝑄(𝑦′)/𝑄(𝑦)

  • De

Dependent Proposals ls: 𝑅 𝑦′ 𝑦 depends on current sample 𝑦

Algorithm adapts to target with simple 1-step memory

slide-19
SLIDE 19

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Metropolis - Hastings: Limitations

  • Highly correlated targets

Proposal should match target to avoid too many rejections

  • Serial correlation
  • Results from rejection

and too small stepping

  • Subsampling

19

  • Bishop. PRML, Springer,

2006

slide-20
SLIDE 20

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

  • Metropolis algorithm formalizes: propose-and-verify
  • Steps are completely independent.

Propose Draw a sample 𝑦′ from 𝑅(𝑦′|𝑦) Veri rify fy With probability 𝛽 = min

𝑄 𝑦′ 𝑄 𝑦 𝑅 𝑦|𝑦′ 𝑅 𝑦′|𝑦 , 1

accept π’šβ€² as new sample

Propose-and-Verify Algorithm

20

slide-21
SLIDE 21

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

MH as Propose and Verify

  • Decouples the steps of finding the solution from validating a solution
  • Natural to integrate uncertain proposals Q

(e.g. automatically detected landmarks, ...)

  • Possibility to include β€œlocal optimization” (e.g. a ICP or ASM updates,

gradient step, …) as proposal Anything more β€œinformed” than random walk should improve convergence.

slide-22
SLIDE 22

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Fitting 3D Landmarks

3D Alignment with Shape and Pose

22

slide-23
SLIDE 23

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

3D Fitting Example

23

right.eye.corner_outer left.eye.corner_outer right.lips.corner left.lips.corner

slide-24
SLIDE 24

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

3D Fitting Setup

Observations

  • Observed positions π‘šπ‘ˆ

1, … , π‘šπ‘ˆ π‘œ

  • Correspondence: π‘šπ‘†

1, … , π‘šπ‘† π‘œ

Parameters πœ„ = 𝛽, πœ’, πœ”, 𝜘, 𝑒 Posterior distribution: 𝑄 πœ„ π‘šπ‘ˆ

1, … , π‘šπ‘ˆ π‘œ ∝ π‘ž π‘šπ‘ˆ 1, … , π‘šπ‘ˆ 𝑆|πœ„ 𝑄(πœ„)

Shape transformation πœ’π‘‘ 𝛽 = 𝜈 𝑦 + ෍

𝑗=1 𝑠

𝛽𝑗 πœ‡π‘—π›Έπ‘—(𝑦) Rigid transformation

  • 3 angles (pitch, yaw, roll) πœ’, πœ”, 𝜘
  • Translation 𝑒 = (𝑒𝑦, 𝑒𝑧, 𝑒𝑨)

πœ’π‘† πœ’, πœ”, 𝜘, 𝑒 = π‘†πœ˜π‘†πœ”π‘†πœ’ π’š + 𝑒

Full transformation πœ’ πœ„ (𝑦) = (πœ’π‘†βˆ˜ πœ’π‘‡)[πœ„](𝑦)

24

Goa

  • al: Find posterior distribution for arbitrary pose and shape
slide-25
SLIDE 25

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Proposals

  • Gaussian random walk proposals

"𝑅 πœ„β€²|πœ„ = 𝑂(πœ„β€²|πœ„, Ξ£πœ„)"

  • Update different parameter types block-wise
  • Shape

𝑂(πœ·β€²|𝜷, πœπ‘‡

2𝐽𝑛× 𝑛 )

  • Rotation

𝑂 πœ’β€² πœ’, πœπœ’

2 , 𝑂 πœ”β€² πœ”, πœπœ” 2 , 𝑂 πœ˜β€² 𝜘, 𝜏𝜘 2

  • Translation

𝑂 𝒖′ 𝒖, πœπ‘’

2𝐽3Γ—3

  • Large mixture distributions as proposals
  • Choose proposal 𝑅𝑗 with probability 𝑑𝑗

𝑅 πœ„β€²|πœ„ = βˆ‘π‘‘π‘—π‘…π‘—(πœ„β€²|πœ„)

25

slide-26
SLIDE 26

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

3DMM Landmarks Likelihood

Simple models: In Independent Gau Gaussians Observation of 𝑀 landmark locations π‘šπ‘ˆ

𝑗 in image

  • Single landmark position model:

π‘ž π‘šπ‘ˆ πœ„, π‘šπ‘† = 𝑂 πœ’ πœ„ π‘šπ‘† , 𝐽3Γ—3𝜏2

  • Independent model (conditional independence):

π‘ž π‘šπ‘ˆ

1, … , π‘šπ‘ˆ π‘œ|πœ„ = ΰ·‘ 𝑗=1 𝑀

π‘žπ‘— π‘šπ‘ˆ

𝑗 |πœ„

26

slide-27
SLIDE 27

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

3D Fit to landmarks

  • Influence of landmarks uncertainty on

final posterior?

  • 𝜏LM = 1mm
  • 𝜏LM = 4mm
  • 𝜏LM = 10mm
  • Only 4 landmark observations:
  • Expect only weak shape impact
  • Should still constrain pose
  • Uncertain landmarks should be looser

27

slide-28
SLIDE 28

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Posterior: Pose & Shape, 4mm

28

ΖΈ 𝜈yaw = 0.511 ො 𝜏yaw = 0.073 (4Β°) ΖΈ 𝜈tx = βˆ’1 mm ො 𝜏tx = 4 mm ΖΈ πœˆπ›½1 = 0.4 ො πœπ›½1 = 0.6 (Estimation from samples)

slide-29
SLIDE 29

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Posterior: Pose & Shape, 1mm

30

ΖΈ 𝜈yaw = 0.50 ො 𝜏yaw = 0.041 (2.4Β°) ΖΈ 𝜈tx = βˆ’2 mm ො 𝜏tx = 0.8 mm ΖΈ πœˆπ›½1 = 1.5 ො πœπ›½1 = 0.35

slide-30
SLIDE 30

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Posterior: Pose & Shape, 10mm

31

ΖΈ 𝜈yaw = 0.49 ො 𝜏yaw = 0.11 (7Β°) ΖΈ 𝜈tx = βˆ’5 mm ො 𝜏tx = 10 mm ΖΈ πœˆπ›½1 = 0 ො πœπ›½1 = 0.6

slide-31
SLIDE 31

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Summary: MCMC for 3D Fitting

  • Probabilistic inference for fitting probabilistic models
  • Bayesian inference: posterior distribution
  • Probabilistic inference is often intractable
  • Use approximate inference methods
  • MCMC methods provide a powerful sampling framework
  • Metropolis-Hastings algorithm
  • Propose update step
  • Verify and accept with probability
  • Samples converge to true distribution: More about this later!

32