Gaussian Process Implicit Surfaces Oliver Williams 1 Microsoft - - PowerPoint PPT Presentation

▶
gaussian process implicit surfaces
SMART_READER_LITE
LIVE PREVIEW

Gaussian Process Implicit Surfaces Oliver Williams 1 Microsoft - - PowerPoint PPT Presentation

Dec 22, 2022 232 likes •650 views

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary Gaussian Process Implicit Surfaces Oliver Williams 1 Microsoft Research and Trinity Hall Cambridge, UK Gaussian Processes

slide-1
SLIDE 1

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Gaussian Process Implicit Surfaces

Oliver Williams1

Microsoft Research and Trinity Hall Cambridge, UK

Gaussian Processes in Practice Bletchley Park, 13 June 2006

1Joint work with Andrew Fitzgibbon

slide-2
SLIDE 2

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Talk outline

Implicit surface modelling Spline regularization as a Gaussian process

Covariance function 1D regression demonstration

GPIS for 2D curves

Covariance in 2D Probabilistic interpretation

GPIS for 3D surfaces

Covariance function

Summary

slide-3
SLIDE 3

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Implicit surface Scalar function f (x) defins a surface wherever it passes through a given value (e.g., 0) S0 {x ∈ Rd|f (x) = 0}.

slide-4
SLIDE 4

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Implicit surface Scalar function f (x) defins a surface wherever it passes through a given value (e.g., 0) S0 {x ∈ Rd|f (x) = 0}. Example: Function f (x) for x ∈ R2 defines a closed curve

slide-5
SLIDE 5

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Fitting to data points Our setting (Turk and O’Brien 1999):

◮ Given a set of constraint points in 2D or 3D {xi}, fit f (x) ◮ Have constraints at f (xi) = 0 on the curve and at ±1 off it

e.g.,

◮ Simple interior/exterior case ◮ Control normals to curve

slide-6
SLIDE 6

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Fitting to data points Our setting (Turk and O’Brien 1999):

◮ Given a set of constraint points in 2D or 3D {xi}, fit f (x) ◮ Have constraints at f (xi) = 0 on the curve and at ±1 off it

e.g.,

◮ Simple interior/exterior case ◮ Control normals to curve

−0.5 0.5 −1 −0.5 0.5

slide-7
SLIDE 7

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Fitting to data points Our setting (Turk and O’Brien 1999):

◮ Given a set of constraint points in 2D or 3D {xi}, fit f (x) ◮ Have constraints at f (xi) = 0 on the curve and at ±1 off it

e.g.,

◮ Simple interior/exterior case ◮ Control normals to curve

−0.5 0.5 −1 −0.5 0.5

slide-8
SLIDE 8

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Fitting to data points Our setting (Turk and O’Brien 1999):

◮ Given a set of constraint points in 2D or 3D {xi}, fit f (x) ◮ Have constraints at f (xi) = 0 on the curve and at ±1 off it

e.g.,

◮ Simple interior/exterior case ◮ Control normals to curve

−0.5 0.5 −1 −0.5 0.5

Alternative method: parametric surface: x(t), y(t), [z(t)]

◮ What t to assign to data points? ◮ How to handle different topologies? ◮ Can represent non-closed

curves/surfaces

slide-9
SLIDE 9

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Topology change

0.5 1 −0.4 −0.2 0.2 0.4 0.6 0.8 1 1.2 −2 2 −3 −2 −1 1 2 3

slide-10
SLIDE 10

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Regularization Find function passing through constraint points which minimizes thin-plate spline energy E(f ) =

  • Ω
  • ∇T∇f (x)

2 dx

slide-11
SLIDE 11

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Regularization Find function passing through constraint points which minimizes thin-plate spline energy E(f ) =

  • Ω
  • ∇T∇f (x)

2 dx Fit f (x) with Gaussian process

slide-12
SLIDE 12

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Regularization Find function passing through constraint points which minimizes thin-plate spline energy E(f ) =

  • Ω
  • ∇T∇f (x)

2 dx Fit f (x) with Gaussian process Use covariance function equivalent to thin-plate spline

slide-13
SLIDE 13

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Gaussian distribution

◮ Follow derivation in (MacKay 2003) in 1D

slide-14
SLIDE 14

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Gaussian distribution

◮ Follow derivation in (MacKay 2003) in 1D ◮ consider the energy as a probability and define D as the linear

differential operator E(f ) = − log P(f ) + const =

  • Ω
  • D2f (x)

2 dx.

slide-15
SLIDE 15

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Gaussian distribution

◮ Follow derivation in (MacKay 2003) in 1D ◮ consider the energy as a probability and define D as the linear

differential operator E(f ) = − log P(f ) + const =

  • Ω
  • D2f (x)

2 dx.

◮ Use f (Ω) as vector of function values for all points in Ω:

− log P(f (Ω)) = f (Ω)T[D2]TD2f (Ω),

slide-16
SLIDE 16

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Gaussian distribution

◮ Follow derivation in (MacKay 2003) in 1D ◮ consider the energy as a probability and define D as the linear

differential operator E(f ) = − log P(f ) + const =

  • Ω
  • D2f (x)

2 dx.

◮ Use f (Ω) as vector of function values for all points in Ω:

− log P(f (Ω)) = f (Ω)T[D2]TD2f (Ω),

◮ This is a Gaussian disribution with mean zero and covariance:

C =

  • [D2]TD2−1

=

  • D4−1 .
slide-17
SLIDE 17

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Covariance function

◮ Entries of C indexed by u, v ∈ Ω

  • Ω

D4(u, w)c(w, v) dw = δ(u − v) ⇒ ∂4 ∂r4 c(r) = δ(r) where we impose stationarity with r = u − v.

slide-18
SLIDE 18

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Covariance function

◮ Entries of C indexed by u, v ∈ Ω

  • Ω

D4(u, w)c(w, v) dw = δ(u − v) ⇒ ∂4 ∂r4 c(r) = δ(r) where we impose stationarity with r = u − v.

◮ Interpret as spectral density and solve

F

  • c(r)
  • (ω) = ω−4

⇒ c(r) = 1

6|r|3 + a3r3 + a2r2 + a1r + a0.

slide-19
SLIDE 19

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Covariance function c(r) = 1

6|r|3 + a3r3 + a2r2 + a1r + a0

Find constants using constraints on c(·)

◮ Symmetry: a3 = a1 = 0

slide-20
SLIDE 20

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Covariance function c(r) = 1

6|r|3 + a3r3 + a2r2 + a1r + a0

Find constants using constraints on c(·)

◮ Symmetry: a3 = a1 = 0 ◮ Postive definiteness: simulate by making c(r) → 0 at ∂Ω

c(r) = 1

12

  • 2|r|3 − 3Rr2 + R3

. where R is the largest magnitude of r within Ω.

slide-21
SLIDE 21

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

1D regression demonstration GP predicts function values for set of points U ⊆ Ω P(f (U)|X) = Normal (f (U) | µ, Q) where µ = C T

ux(Cxx + σ2I)−1t

and Q = Cuu − C T

ux(Cxx + σ2I)−1Cux.

The matrices are formed by evaluating c(·, ·) between sets of points: i.e., Cxx = [c(xi, xj)], Cux = [c(ui, xj)], and Cuu = [c(ui, uj)].

slide-22
SLIDE 22

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

1D regression demonstration

−2 −1 1 2 −2 −1 1 2 −2 −1 1 2 −2 −1 1 2

(a) (b)

Figure: Thin plate vs. squared exponential covariance. Mean (solid line) and 3 s.d. error bars (filled region) for GP regression (a) Thin- plate covariance; (b) Squared exponential covariance function c(ui, uj) = e−αui−uj2 with α = 2, 10 and 100; error bars correspond to α = 10.

slide-23
SLIDE 23

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Covariance in 2D

◮ In 2D the Green’s equation is

  • ∇T∇

2 c(r) = δ(r) where now c(u, v) = c(r) with r u − v.

slide-24
SLIDE 24

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Covariance in 2D

◮ In 2D the Green’s equation is

  • ∇T∇

2 c(r) = δ(r) where now c(u, v) = c(r) with r u − v.

◮ Solution (with similar constraints at the boundary of Ω)

c(r) = 2r2 log |r| −

  • 1 + 2 log(R)
  • r2 + R2
slide-25
SLIDE 25

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Demonstration

◮ Set constraint points ◮

−0.5 0.5 −1 −0.8 −0.6 −0.4 −0.2 0.2 0.4 0.6 0.8

X: {⊙ = 0, = +1, = −1}

slide-26
SLIDE 26

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Demonstration

◮ Set constraint points ◮ Fit GP to points

−0.5 0.5 −1 −0.8 −0.6 −0.4 −0.2 0.2 0.4 0.6 0.8

X: {⊙ = 0, = +1, = −1} Mean function

slide-27
SLIDE 27

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Probabilistic interpretation Gaussian process makes probabilistic prediction:

−0.5 0.5 −1 −0.8 −0.6 −0.4 −0.2 0.2 0.4 0.6 0.8

Samples from posterior

slide-28
SLIDE 28

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Probabilistic interpretation Gaussian process makes probabilistic prediction:

−0.5 0.5 −1 −0.8 −0.6 −0.4 −0.2 0.2 0.4 0.6 0.8 −1 −0.5 0.5 1 −1 −0.5 0.5 1

Samples from posterior P

  • x ∈ S0
  • ≡ P
  • f (x) = 0
  • .
slide-29
SLIDE 29

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

With different topology

−0.5 0.5 1 1.5 −0.5 0.5 1 1.5 −2 2 −3 −2 −1 1 2 3

slide-30
SLIDE 30

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Result with squared exponential

−1 −0.5 0.5 1 −1 −0.5 0.5 1

slide-31
SLIDE 31

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Fitting 3D surfaces In 3D the covariance is c(r) = 2|r|3 + 3Rr2 + R3

slide-32
SLIDE 32

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Fitting 3D surfaces In 3D the covariance is c(r) = 2|r|3 + 3Rr2 + R3

◮ Take n points on surface of object

slide-33
SLIDE 33

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Fitting 3D surfaces In 3D the covariance is c(r) = 2|r|3 + 3Rr2 + R3

◮ Take n points on surface of object ◮ Define internal and external points

slide-34
SLIDE 34

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Fitting 3D surfaces In 3D the covariance is c(r) = 2|r|3 + 3Rr2 + R3

◮ Take n points on surface of object ◮ Define internal and external points ◮ Fit Gaussian process

slide-35
SLIDE 35

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Fitting 3D surfaces In 3D the covariance is c(r) = 2|r|3 + 3Rr2 + R3

◮ Take n points on surface of object ◮ Define internal and external points ◮ Fit Gaussian process ◮ Use marching cubes algorithm to find mean surface

slide-36
SLIDE 36

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

(a) (b)

Figure: 3D surfaces. Mean surfaces µ(x) = 0 when x ∈ R3, rendered as an high resolution polygonal mesh generated by the marching cubes

  • algorithm. (a) A simple “blob” defined by 15 points on the surface, one

interior +1 point and 8 exterior -1 points arranged as a cube; (b) Two views of the Stanford bunny defined by 800 surface points, one interior +1 point, and a sphere of 80 exterior -1 points.

slide-37
SLIDE 37

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Summary

◮ Gaussian processes can be used to define curves and surfaces

slide-38
SLIDE 38

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Summary

◮ Gaussian processes can be used to define curves and surfaces ◮ Appropriate covariance must be used to obtain high quality

results

slide-39
SLIDE 39

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Summary

◮ Gaussian processes can be used to define curves and surfaces ◮ Appropriate covariance must be used to obtain high quality

results

◮ By using a GPIS, curves and surfaces have a meaningful

probabilistic interpretation

slide-40
SLIDE 40

Implicit surface modelling Spline regularization as a Gaussian process GPIS for 2D curves GPIS for 3D surfaces Summary

Summary

◮ Gaussian processes can be used to define curves and surfaces ◮ Appropriate covariance must be used to obtain high quality

results

◮ By using a GPIS, curves and surfaces have a meaningful

probabilistic interpretation Shortcomings / ideas for future work:

◮ Exploit probabilistic nature of GPIS in computer vision

problems

◮ More elegant methods for constraining surface normals? ◮ Can this be used to learn a meaningful prior? ◮ Scale/smoothness control?