Nonlinear Spectral Decomposition Martin Burger Martin Burger - - PowerPoint PPT Presentation

▶
nonlinear spectral decomposition
SMART_READER_LITE
LIVE PREVIEW

Nonlinear Spectral Decomposition Martin Burger Martin Burger - - PowerPoint PPT Presentation

May 25, 2023 288 likes •771 views

Nonlinear Spectral Decomposition Martin Burger Martin Burger Joint work with Martin Benning (Mnster / Cambridge) MB 2 , Meth. Appl. Anal. 2013, Special Issue Osher 70 Benning, PhD thesis, 2011 Guy Gilboa (T echnion), Michael Mller

slide-1
SLIDE 1

Martin Burger

Martin Burger

Nonlinear Spectral Decomposition

slide-2
SLIDE 2

Martin Burger

Joint work with

Martin Benning (Münster / Cambridge)

MB2, Meth. Appl. Anal. 2013, Special Issue Osher 70 Benning, PhD thesis, 2011

Guy Gilboa (T echnion), Michael Möller (Munich), Lina Eckardt (Münster), Daniel Cremers (Munich)

SSVM 2015 / BsC Thesis 2014 Preprint (arXiV) 2016

slide-3
SLIDE 3

Martin Burger

Singular Value Decomposition

Singular values and singular vectors are crucial for the analysis of linear methods for solving inverse problems Singular vectors are obtained as solutions of eigenvalue problem Singular value

slide-4
SLIDE 4

Martin Burger

Convex Variational Regularization

In the last years linear reconstruction methods lost importance Popular approaches (in particular in imaging) are of the form with one-homogeneous J like TV or L1 Are there singular vectors for such ? Are they useful ?

slide-5
SLIDE 5

Martin Burger

Defining Singular Vectors

We need variational characterization for comparison Note: linear case corresponds to Rayleigh-principle: singular vector for smallest singular value minimizes

slide-6
SLIDE 6

Martin Burger

Ground States

Generalize Rayleigh principle: Problem: can yield uninteresting elements minimizing J Example J=TV: Ground states would simply be constant functions

slide-7
SLIDE 7

Martin Burger

Ground States

Choose J to be a seminorm on a dense subspace. Then its kernel is a closed linear subspace Eliminate kernel for improved definition of ground state Existence under standard assumptions

slide-8
SLIDE 8

Martin Burger

Ground States: Examples

1D Total Variation denoising (K=I): ground state = single step function

slide-9
SLIDE 9

Martin Burger

Ground States: Examples

Sparsity ( ): ground state = vector with nonzero entry at index corresponding to column of K with maximal norm Nuclear norm of matrices: ground state = rank one matrix corresponding to classical largest singular value

slide-10
SLIDE 10

Martin Burger

Ground States and Singular Vectors

Ground states are stationary points of Lagrangian Due to nonconvexity of constraint there are multiple stationary points satisfying We call them singular vectors and focus on those with

slide-11
SLIDE 11

Martin Burger

Rayleigh Principle for Higher Singular Vectors

Usual construction for further singular vectors Due to nonconvexity of constraint there are multiple stationary points satisfying where

slide-12
SLIDE 12

Martin Burger

Rayleigh Principle for Singular Vectors

Usual construction for further singular vectors fails ! Using Lagrange multipliers we find with No particular reason for those to vanish !

slide-13
SLIDE 13

Martin Burger

Rayleigh Principle for Higher Singular Vectors

Construction for special cases can still be interesting 1D total variation denoising with appropriate TV definition: Rayleigh principle yields sequence of singular vectors equivalent to the Haar wavelet basis !

slide-14
SLIDE 14

Martin Burger

Use of Singular Vectors ?

Due to nonlinearity, there is no singular value decomposition Other ways of use:

  • Canonical cases and exact solutions for regularization methods,

analysis of bias

  • Definition of scale relative to regularization, scale estimates
slide-15
SLIDE 15

Martin Burger

Exact Solutions of Variational Regularization

Solutions of with are given by if Similar results for noisy perturbations

slide-16
SLIDE 16

Martin Burger

Exact Solutions of Inverse Scale Space Method

Solutions of with are given by Similar results for noisy data and other related methods

slide-17
SLIDE 17

Martin Burger

Exact Solutions of Variational Regularization

Provides systematic way of analyzing exact solutions Includes all examples in literature (most being ground states, some already charaterized as eigenfunctions):

  • TV: Strong-Chan 1996, Meyer 2001, Strong 2003
  • TV-flow: Bellettini et al 2001, Andreu et al 2001, Caselles-Chambolle-

Novaga 2007-2010

  • Higher order TV: Papafitsoros-Bredies 2014, Pöschl-Scherzer 2014

mainly contained as singular values in Benning-Brune-mb-Müller 2013

slide-18
SLIDE 18

Martin Burger

Higher order TV functionals avoiding staircasing

Major idea: combine TV with higher order TV Infimal convolution Dual version

Chambolle-Lions 97

slide-19
SLIDE 19

Martin Burger

TGV / GTV

Decomposition by inf-convolution not optimal, improvement by stronger dual constraint Primal version

Bredies et al 2011

slide-20
SLIDE 20

Martin Burger

TGV vs. ICTV

Equivalence of functionals in 1D Intuitive advantages of TGV in multiple dimensions

Bredies et al 2011 / 2013, Benning-Brune-mb-Müller 2013

Better understanding by constructing eigenfunctions for TGV denoising, which are not eigenfunctions of ICTV . Any eigenfunction of ICTV is eigenfunction of TGV

Müller PhD 2013

slide-21
SLIDE 21

Martin Burger

GTV Origami

slide-22
SLIDE 22

Martin Burger

Bias of Variational Regularization

Error (bias) in solution increasing with size of singular value Does the smallest singular value define minimal bias ? Arbitrary data satisfying only Then where

slide-23
SLIDE 23

Martin Burger

Bias of Variational Regularization

In the same way underestimation of regularization functional then

slide-24
SLIDE 24

Martin Burger

Spectral Decomposition

Consider simpler case of K = Id (eigenfunctions / values) Can we get a spectral decomposition from a seminorm J ? Example: Fourier Decomposition / Laplacian eigenfunctions

slide-25
SLIDE 25

Martin Burger

Laplacian Eigenfunctions

Fourier cosine decomposition in 1D

slide-26
SLIDE 26

Martin Burger

Spectral Decomposition

Natural / geometric spectral definition of an image ?

slide-27
SLIDE 27

Martin Burger

Spectral Decomposition

Natural / geometric spectral definition of an image ?

slide-28
SLIDE 28

Martin Burger

What is a Spectral Decomposition ?

Standard case related to positive semifinite linear operator A in Hilbert space X, respectively seminorm Spectral theorem: there exists a vector valued measure (to the space of linear operators on X) such that for a scalar function In particular decomposition of A and Identity

slide-29
SLIDE 29

Martin Burger

Filtering

We are not interested in the operator, but in action on f Hence we have a vector valued measure into X Spectral decomposition / filtering

slide-30
SLIDE 30

Martin Burger

Nonlinear Spectral Decomposition

Keep basic properties: Wavelength representation:

slide-31
SLIDE 31

Martin Burger

Nonlinear Spectral Decomposition

Polar decomposition of the measure defines spectrum Parseval identity

slide-32
SLIDE 32

Martin Burger

Defining Spectral Decompositions

Different options: variational methods / gradient flows

slide-33
SLIDE 33

Martin Burger

Defining Spectral Decompositions

Spectral representation derived from dynamics of eigenfunctions

slide-34
SLIDE 34

Martin Burger

Connections of Spectral Decompositions

Very similar results for all the spectral decompositions Conjecture: under appropriate conditions all spectral representations are the same GF and VM have same primal variable u(t) VM and IS have same dual variable p(t) / q(s) VM dual variable is Fejer mean of GF dual variable VM primal variable is Fejer mean of IS primal variable

slide-35
SLIDE 35

Martin Burger

Relations of Spectral Decomposition

Consider finite-dimensional polyhedral (crystalline) case

slide-36
SLIDE 36

Martin Burger

Piecewise linear dynamics under (PS)

Gradient flow (related to results by Briani et 2011 for TV flow): Inverse scale space method (related to mb-Möller-Benning-Osher 2012, Möller-mb 2014):

slide-37
SLIDE 37

Martin Burger

Piecewise linear dynamics under (PS)

Variational method:

slide-38
SLIDE 38

Martin Burger

Spectral representation

Well-defined decomposition

slide-39
SLIDE 39

Martin Burger

Equivalence under (MINSUB)

If (PS) and (MINSUB) hold, then for all f:

slide-40
SLIDE 40

Martin Burger

Equivalence under (DD-L1)

Canonical example (MINSUB) is satisfied if and only if KK* is weakly diagonally dominant Satisfied e.g. for 1D TV, K = div (Briani et al 2011) Under this condition we also obtain that VM and IS are equivalent (same dual variable) In particular all three approaches yield the same spectral decomposition

slide-41
SLIDE 41

Martin Burger

Eigenfunction decomposition

Under (DD-L1) the subgradients of the gradient flow are eigenfunctions of J Hence we have a decomposition into eigenfunctions

slide-42
SLIDE 42

Martin Burger

Applications: Filtering

slide-43
SLIDE 43

Martin Burger

Applications: Filtering

slide-44
SLIDE 44

Martin Burger

Applications: Ageing

slide-45
SLIDE 45

Martin Burger

Applications: Ageing

slide-46
SLIDE 46

Martin Burger

Applications: Personalized Avatar

slide-47
SLIDE 47

Martin Burger

Conclusion

Ground states and singular vectors can be generalized nicely to nonlinear setup Yield detailed insight into behaviour of regularization methods and multiple scales Potential for further investigation Computation of singular vectors (explicit / numerical)