Justin Solomon David Bommes Princeton University RWTH Aachen - - PowerPoint PPT Presentation

▶
justin solomon david bommes
SMART_READER_LITE
LIVE PREVIEW

Justin Solomon David Bommes Princeton University RWTH Aachen - - PowerPoint PPT Presentation

May 27, 2023 103 likes •862 views

Justin Solomon David Bommes Princeton University RWTH Aachen University Optimization. Synonym(-ish): Variational methods. Optimization. Synonym(-ish): Variational methods. Caveat: Slightly different connotation in ML Client Which

slide-1
SLIDE 1

Justin Solomon

Princeton University

David Bommes

RWTH Aachen University

slide-2
SLIDE 2

Optimization.

Synonym(-ish): Variational methods.

slide-3
SLIDE 3

Optimization.

Synonym(-ish): Variational methods.

Caveat: Slightly different connotation in ML

slide-4
SLIDE 4
slide-5
SLIDE 5

Client

Which optimization tool is relevant?

Designer

Can I design an algorithm for this problem?

slide-6
SLIDE 6

Optimization is a huge field.

Patterns, algorithms, & examples common in geometry processing.

slide-7
SLIDE 7

Vocabulary Simple examples Unconstrained optimization Equality-constrained

  • ptimization

Part I (Justin)

slide-8
SLIDE 8

Inequality constraints Advanced algorithms Discrete problems Conclusion

Part II (David)

slide-9
SLIDE 9

Vocabulary Simple examples Unconstrained optimization Equality-constrained

  • ptimization

Part I (Justin)

slide-10
SLIDE 10

Objective (“Energy Function”)

slide-11
SLIDE 11

Equality Constraints

slide-12
SLIDE 12

Inequality Constraints

slide-13
SLIDE 13

Gradient

https://en.wikipedia.org/?title=Gradient

slide-14
SLIDE 14

Hessian

http://math.etsu.edu/multicalc/prealpha/Chap2/Chap2-5/10-3a-t3.gif

slide-15
SLIDE 15

Jacobian

https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant

slide-16
SLIDE 16

Critical point

(unconstrained)

Saddle point Local min Local max

slide-17
SLIDE 17

Critical points

may not be minima.

slide-18
SLIDE 18
slide-19
SLIDE 19

Numerical Algorithms, Solomon

More later

slide-20
SLIDE 20

Vocabulary Simple examples Unconstrained optimization Equality-constrained

  • ptimization

Part I (Justin)

slide-21
SLIDE 21
slide-22
SLIDE 22

How effective are generic

  • ptimization tools?
slide-23
SLIDE 23

Try the

simplest solver first.

slide-24
SLIDE 24

(assume A is symmetric and positive definite)

slide-25
SLIDE 25

Normal equations (better solvers for this case!)

slide-26
SLIDE 26
  • G. Peyré, mesh processing course slides
slide-27
SLIDE 27

 𝒙𝒋𝒌 ≡ 𝟐: Tutte embedding  𝒙𝒋𝒌 from mesh: Harmonic embedding

Assumption: 𝒙 symmetric.

slide-28
SLIDE 28

Laplacian matrix!

slide-29
SLIDE 29
  • K. Crane, brickisland.net

Leads to famous cotangent weights! Useful for interpolation.

slide-30
SLIDE 30

 Never construct 𝑩−𝟐 explicitly

(if you can avoid it)

 Added structure helps

Sparsity, symmetry, positive definite

slide-31
SLIDE 31

 Direct (explicit matrix)

  • Dense: Gaussian elimination/LU, QR for least-squares
  • Sparse: Reordering (SuiteSparse, Eigen)

 Iterative (apply matrix repeatedly)

  • Positive definite: Conjugate gradients
  • Symmetric: MINRES, GMRES
  • Generic: LSQR
slide-32
SLIDE 32

Induced by the connectivity of the triangle mesh. Iteration of CG has local effect ⇒ Precondition!

slide-33
SLIDE 33

What if 𝑾𝟏 = {}?

slide-34
SLIDE 34

Prevents trivial solution 𝒚 ≡ 𝟏. Extract the smallest eigenvalue.

slide-35
SLIDE 35

Prevents trivial solution 𝒚 ≡ 𝟏. N contains basis for null space of A. Extract the smallest nonzero eigenvalue.

slide-36
SLIDE 36

Mullen et al. “Spectral Conformal Parameterization.” SGP 2008.

slide-37
SLIDE 37

2 3 4 5 6 7 8 9 10

“Laplace-Beltrami Eigenfunctions”

slide-38
SLIDE 38
slide-39
SLIDE 39

Roughly:

1.Extract Laplace-Beltrami eigenfunctions: 2.Find mapping matrix (linear solve!):

Ovsjanikov et al. “Functional Maps.” SIGGRAPH 2012.

slide-40
SLIDE 40

Vocabulary Simple examples Unconstrained optimization Equality-constrained

  • ptimization

Part I (Justin)

slide-41
SLIDE 41
slide-42
SLIDE 42

Unstructured.

slide-43
SLIDE 43

Gradient descent

slide-44
SLIDE 44

Gradient descent

Line search

slide-45
SLIDE 45

Gradient descent

Multiple optima!

slide-46
SLIDE 46

Accelerated gradient descent

Quadratic convergence on convex problems!

(Nesterov 1983)

slide-47
SLIDE 47

Newton’s Method

1 2 3

Line search for stability

slide-48
SLIDE 48

Quasi-Newton: BFGS and friends

Hessian approximation

 (Often sparse) approximation from previous

samples and gradients

 Inverse in closed form!

slide-49
SLIDE 49

Often continuous gradient descent

  • M. Kazhdan
slide-50
SLIDE 50

Fröhlich and Botsch. “Example-Driven Deformations Based on Discrete Shells.” CGF 2011.

slide-51
SLIDE 51

Roughly:

  • 1. Linearly interpolate edge lengths and dihedral

angles.

  • 2. Nonlinear optimization for vertex positions.

Sum of squares: Gauss-Newton

slide-52
SLIDE 52

 Matlab: fminunc or minfunc  C++: libLBFGS, dlib, others

Typically provide functions for function and gradient (and optionally, Hessian).

Try several!

slide-53
SLIDE 53

Vocabulary Simple examples Unconstrained optimization Equality-constrained

  • ptimization

Part I (Justin)

slide-54
SLIDE 54
slide-55
SLIDE 55
  • Decrease f: −𝛂𝒈
  • Violate constraint: ±𝛂𝒉
slide-56
SLIDE 56

Want:

slide-57
SLIDE 57
slide-58
SLIDE 58

Turns constrained optimization into

unconstrained root-finding.

slide-59
SLIDE 59

Reparameterization

Eliminate constraints to reduce to unconstrained case

Newton’s method

Approximation: quadratic function with linear constraint

Penalty method

Augment objective with barrier term, e.g. 𝒈 𝒚 + 𝝇|𝒉 𝒚 |

slide-60
SLIDE 60

(assume A is symmetric and positive definite)

slide-61
SLIDE 61

No longer positive definite!

slide-62
SLIDE 62

Example: Levenberg-Marquardt

Fix (or adjust) damping parameter 𝝁 > 𝟏.

slide-63
SLIDE 63

Huang et al. “L1-Based Construction of Polycube Maps from Complex Shapes.” TOG 2014.

slide-64
SLIDE 64

Align with coordinate axes Preserve area

Note: Final method includes several more terms!

slide-65
SLIDE 65
slide-66
SLIDE 66

Try lightweight options

versus

slide-67
SLIDE 67

Try lightweight options

versus

slide-68
SLIDE 68

Repeatedly solve linear systems

“Geometric median”

slide-69
SLIDE 69

d can be a Bregman divergence

slide-70
SLIDE 70

Decompose as sum of hard part f and easy part g.

https://blogs.princeton.edu/imabandit/2013/04/11/orf523-ista-and-fista/

slide-71
SLIDE 71

Add constraint to objective

Does nothing when constraint is satisfied

slide-72
SLIDE 72

https://web.stanford.edu/~boyd/papers/pdf/admm_slides.pdf

slide-73
SLIDE 73

Want two easy subproblems

Augmented part

Solomon et al. “Earth Mover’s Distances on Discrete Surfaces.” SIGGRAPH 2014.

slide-74
SLIDE 74

https://en.wikipedia.org/wiki/Frank%E2%80%93Wolfe_algorithm

Linearize objective, not constraints