Beyond Convenience: Beyond Convexity Purushottam Kar MINI-SYMPOSIUM - - PowerPoint PPT Presentation

β–Ά
beyond convenience beyond convexity
SMART_READER_LITE
LIVE PREVIEW

Beyond Convenience: Beyond Convexity Purushottam Kar MINI-SYMPOSIUM - - PowerPoint PPT Presentation

Mar 26, 2023 207 likes β€’488 views

Beyond Convenience: Beyond Convexity Purushottam Kar MINI-SYMPOSIUM ON COMPUTATION AND OPTIMIZATION IN THE SCIENCES AND ENGINEERING Outline of the Talk Convex Optimization A Few Contemporary Applications Non-convex Optimization

slide-1
SLIDE 1

Beyond Convenience: Beyond Convexity

Purushottam Kar MINI-SYMPOSIUM ON COMPUTATION AND OPTIMIZATION IN THE SCIENCES AND ENGINEERING

slide-2
SLIDE 2

Outline of the Talk

  • Convex Optimization
  • A Few Contemporary Applications
  • Non-convex Optimization
  • Robust Regression
  • Applications of Robust Regression
  • Robust PCA
slide-3
SLIDE 3

Convex Optimization

slide-4
SLIDE 4

Convex Optimization

Convex function Convex set

slide-5
SLIDE 5

Examples

Linear Programming Quadratic Programming Semidefinite Programming

slide-6
SLIDE 6

Applications

Resource Allocation Classification Regression Clustering/Partitioning Signal Processing Dimensionality Reduction

slide-7
SLIDE 7

Techniques

  • Projected (Sub)gradient Methods
  • Stochastic, mini-batch variants
  • Primal, dual, primal-dual approaches
  • Coordinate update techniques
  • Interior Point Methods
  • Barrier methods
  • Annealing methods
  • Other Methods
  • Cutting plane methods
  • Accelerated routines
  • Proximal methods
  • Distributed optimization
  • Derivative-free optimization
slide-8
SLIDE 8

A Few Contemporary Applications

slide-9
SLIDE 9

Gene Expression Analysis

www.tes.com

DNA micro-array gene expression data

…

slide-10
SLIDE 10

Recommender Systems

=

π‘œ 𝑛 𝑙

slide-11
SLIDE 11

Image Reconstruction and Robust Face Recognition

β‰ˆ β‰ˆ = +

0.05 0.90

+

0.05

+

0.01 0.92

+

0.07

= +

0.15 0.65

+

0.20

=

slide-12
SLIDE 12

Image Denoising and Robust Face Recognition

= + = + + + + β‹―

π‘œ

slide-13
SLIDE 13

Large Scale Surveillance

  • Foreground-background separation

= = + = +

π‘œ 𝑛

www.extremetech.com

slide-14
SLIDE 14

Non Convex Optimization

Sparse Recovery Robust PCA Robust Regression Matrix Completion

slide-15
SLIDE 15

Non-convex Optimization

slide-16
SLIDE 16

Relaxation-based Techniques

  • β€œConvexify” the feasible set
slide-17
SLIDE 17

Alternating Minimization

Matrix Completion Robust PCA … also Robust Regression, coming up

slide-18
SLIDE 18

Projected Gradient Descent

Sparse Recovery

Top 𝑑 elements by magnitude Perform 𝑙-truncated SVD

slide-19
SLIDE 19

Pursuit and Greedy Methods

Set of β€œatoms”

Sparse Recovery

slide-20
SLIDE 20

Applications of NCOpt

slide-21
SLIDE 21

Face Recognition

10% noise 30% noise 50% noise 70% noise [Bhatia et al 2015]

slide-22
SLIDE 22

Image Reconstruction

Input Ordinary LS Alt-Min

[Bhatia et al 2015]

Original

slide-23
SLIDE 23

Foreground-background Separation

23

+ = + =

Convex Relaxation. Runtime: 1700 sec Alt-Proj. Runtime: 70 sec [Netrapalli et al 2014]

slide-24
SLIDE 24

Concluding Comments Non-convex optimization is an exciting area Widespread applications

  • Much better modelling of problems
  • Much more scalable algorithms
  • Provable guarantees

So …

  • Full of opportunities
  • Full of challenges
slide-25
SLIDE 25

Acknowledgements

Kush Bhatia

Microsoft Research

Prateek Jain

Microsoft Research

Ambuj Tewari

  • U. Michigan, Ann Arbor

Portions of this talk were based on joint work with

http://research.microsoft.com/en-us/projects/altmin/default.aspx

slide-26
SLIDE 26

Questions?