infimal convolution type regularization for inverse
play

Infimal-convolution-type regularization for inverse problems in - PowerPoint PPT Presentation

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Infimal-convolution-type regularization for inverse problems in imaging Kristian Bredies Institute for Mathematics and Scientific Computing University of Graz IHP semester The Mathematics


  1. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Infimal convolution for inverse problems Tikhonov regularization: Solve min u ∈ X S ( Ku , f ) + Φ( u ) S discrepancy for Ku = f , Φ regularization functional ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 10 / 51

  2. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Infimal convolution for inverse problems Tikhonov regularization: Solve min u ∈ X S ( Ku , f ) + Φ( u ) S discrepancy for Ku = f , Φ regularization functional Two viewpoints: 1 Monolithic regularization min u ∈ X S ( Ku , f ) + Φ( u ), Φ = Φ 1 � . . . � Φ m 2 Vector-valued regularization � � ( u 1 ,..., u m ) ∈ X m S min K ( u 1 + . . . + u m ) , f + Φ 1 ( u 1 ) + . . . Φ m ( u m ) ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 10 / 51

  3. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Infimal convolution for inverse problems Tikhonov regularization: Solve min u ∈ X S ( Ku , f ) + Φ( u ) S discrepancy for Ku = f , Φ regularization functional Two viewpoints: 1 Monolithic regularization min u ∈ X S ( Ku , f ) + Φ( u ), Φ = Φ 1 � . . . � Φ m 2 Vector-valued regularization � � ( u 1 ,..., u m ) ∈ X m S min K ( u 1 + . . . + u m ) , f + Φ 1 ( u 1 ) + . . . Φ m ( u m ) Monolithic regularization: Intrinsic decomposition & problem-adapted interpretation ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 10 / 51

  4. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING General existence result Theorem: X reflexive Banach space, Y normed space K : X → Y bounded linear S ( · , f ) : Y → [0 , ∞ ] proper, convex, l.s.c. Φ : X ∈ [0 , ∞ ] proper, convex, l.s.c. Φ 1-homogeneous, dim ker(Φ) < ∞ , and coercive, i.e., � u − Pu � X ≤ C Φ( u ) for P : X → ker(Φ) bounded linear projector Then: min u ∈ X S ( Ku , f ) + α Φ( u ) has a solution for each α > 0 ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 11 / 51

  5. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING General regularization properties min u ∈ X S ( Ku , f ) + α Φ( u ) Let: Y Hilbert space, S ( v , f ) = 1 2 � v − f � 2 Y Then: (Subsequential) stability for varying f Noise level δ → 0 and α → 0, δ 2 /α → 0 ⇒ (Subsequential) convergence Source condition K ∗ w ∈ ∂ Φ( u † ), α ∼ δ ⇒ O ( δ ) for the Bregman distance w.r.t. Φ ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 12 / 51

  6. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING General regularization properties min u ∈ X S ( Ku , f ) + α Φ( u ) Let: Y Hilbert space, S ( v , f ) = 1 2 � v − f � 2 Y Then: (Subsequential) stability for varying f Noise level δ → 0 and α → 0, δ 2 /α → 0 ⇒ (Subsequential) convergence Source condition K ∗ w ∈ ∂ Φ( u † ), α ∼ δ ⇒ O ( δ ) for the Bregman distance w.r.t. Φ Difficulties: dim ker(Φ) = ∞ ⇒ K has to be stably invertible on ker(Φ) Φ not coercive ⇒ K has to be stably invertible on X ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 12 / 51

  7. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Infimal convolution regularization Lemma: Φ 1 , Φ 2 : X → [0 , ∞ ] proper, convex, l.s.c. Φ i 1-homogeneous, dim ker(Φ) < ∞ for i = 1 , 2 Then: Φ 1 � Φ 2 : X → [0 , ∞ ] is exact, proper, convex, l.s.c., 1-homogeneous, ker Φ 1 � Φ 2 = ker Φ 1 + ker Φ 2 ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 13 / 51

  8. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Infimal convolution regularization Lemma: Φ 1 , Φ 2 : X → [0 , ∞ ] proper, convex, l.s.c. Φ i 1-homogeneous, dim ker(Φ) < ∞ for i = 1 , 2 Then: Φ 1 � Φ 2 : X → [0 , ∞ ] is exact, proper, convex, l.s.c., 1-homogeneous, ker Φ 1 � Φ 2 = ker Φ 1 + ker Φ 2 Lemma: Additionally: X ֒ − ֒ → Z , Z Banach space and � � � u � X ≤ C � u � Z + Φ i ( u ) for i = 1 , 2 and all u ∈ X Then: Φ 1 � Φ 2 is coercive, i.e., � u − Pu � X ≤ C Φ( u ) for all u ∈ X ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 13 / 51

  9. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Infimal convolution regularization Lemma: Φ 1 , Φ 2 : X → [0 , ∞ ] proper, convex, l.s.c. Φ i 1-homogeneous, dim ker(Φ) < ∞ for i = 1 , 2 Then: Φ 1 � Φ 2 : X → [0 , ∞ ] is exact, proper, convex, l.s.c., 1-homogeneous, ker Φ 1 � Φ 2 = ker Φ 1 + ker Φ 2 Lemma: Additionally: X ֒ − ֒ → Z , Z Banach space and � � � u � X ≤ C � u � Z + Φ i ( u ) for i = 1 , 2 and all u ∈ X Then: Φ 1 � Φ 2 is coercive, i.e., � u − Pu � X ≤ C Φ( u ) for all u ∈ X � sufficient conditions for a regularizer ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 13 / 51

  10. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Examples TV - TV 2 infimal convolution: TV and TV 2 have finite-dimensional kernels + embedding: � � � u � 1 + TV 2 ( u ) � u � BV ≤ � u � BV 2 ≤ C Consequently: TV � β TV 2 is a regularizer ( β > 0) ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 14 / 51

  11. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Examples TV - TV 2 infimal convolution: TV and TV 2 have finite-dimensional kernels + embedding: � � � u � 1 + TV 2 ( u ) � u � BV ≤ � u � BV 2 ≤ C Consequently: TV � β TV 2 is a regularizer ( β > 0) TV - L 1 ◦ ∆ infimal convolution: ker(TV � � · � M ◦ ∆) = { u ∈ BV | u harmonic } Infinite-dimensional kernel ⇒ not a regularizer ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 14 / 51

  12. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Examples TV - TV 2 infimal convolution: TV and TV 2 have finite-dimensional kernels + embedding: � � � u � 1 + TV 2 ( u ) � u � BV ≤ � u � BV 2 ≤ C Consequently: TV � β TV 2 is a regularizer ( β > 0) TV - L 1 ◦ ∆ infimal convolution: ker(TV � � · � M ◦ ∆) = { u ∈ BV | u harmonic } Infinite-dimensional kernel ⇒ not a regularizer TV - G -norm/ TV - H − 1 infimal convolution: � · � TV ∗ / � · � H − 1 not coercive in L p -spaces Infimal convolution not coercive ⇒ not a regularizer ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 14 / 51

  13. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Examples TV - TV 2 infimal convolution: TV and TV 2 have finite-dimensional kernels + embedding: � � � u � 1 + TV 2 ( u ) � u � BV ≤ � u � BV 2 ≤ C Consequently: TV � β TV 2 is a regularizer ( β > 0) TV - L 1 ◦ ∆ infimal convolution: ker(TV � � · � M ◦ ∆) = { u ∈ BV | u harmonic } Infinite-dimensional kernel ⇒ not a regularizer TV - G -norm/ TV - H − 1 infimal convolution: � · � TV ∗ / � · � H − 1 not coercive in L p -spaces Infimal convolution not coercive ⇒ not a regularizer � Develop regularizing cartoon/texture models + complex smoothness-based regularizers ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 14 / 51

  14. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Outline 1 Infimal convolution regularization 2 Total generalized variation Definition and properties Applications 3 Infimal convolution TGV Accelerated dynamic MRI 4 Infimal convolution of oscillation TGV Oscillation TGV and infimal convolution Numerical realization Applications 5 Summary ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 15 / 51

  15. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Total generalized variation Motivation: TV-based first-order regularization favors certain artifacts Total generalized variation: [B./Kunisch/Pock ’10] �� � u div k v d x TGV k � α ( u ) = sup noisy image � Ω c (Ω , Sym k (I R d )) , v ∈ C k � � div l v � ∞ ≤ α l , l = 0 , . . . , k − 1 α = ( α 0 , . . . , α k − 1 ) > 0 weights Second-order version: TGV 2 α ( u )= min α 1 �∇ u − w � M + α 0 �E w � M TV-regularization w ∈ BD(Ω) ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 16 / 51

  16. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Total generalized variation Motivation: TV-based first-order regularization favors certain artifacts Total generalized variation: [B./Kunisch/Pock ’10] �� � u div k v d x TGV k � α ( u ) = sup noisy image � Ω c (Ω , Sym k (I R d )) , v ∈ C k � � div l v � ∞ ≤ α l , l = 0 , . . . , k − 1 α = ( α 0 , . . . , α k − 1 ) > 0 weights Second-order version: TGV 2 α ( u )= min α 1 �∇ u − w � M + α 0 �E w � M TGV 2 α -regularization w ∈ BD(Ω) ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 16 / 51

  17. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Properties of TGV 2 α Basic properties: [B./Kunisch/Pock ’10] TGV 2 α is proper, convex, lower semi-continuous TGV 2 α is translation and rotation invariant TGV 2 α + � · � 1 gives the Banach space BGV 2 α (Ω) α ) = Π 1 affine functions ker(TGV 2 TGV 2 α measures piecewise affine only at the interfaces ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 17 / 51

  18. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Properties of TGV 2 α Basic properties: [B./Kunisch/Pock ’10] TGV 2 α is proper, convex, lower semi-continuous TGV 2 α is translation and rotation invariant TGV 2 α + � · � 1 gives the Banach space BGV 2 α (Ω) α ) = Π 1 affine functions ker(TGV 2 TGV 2 α measures piecewise affine only at the interfaces ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 17 / 51

  19. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Properties of TGV 2 α Basic properties: [B./Kunisch/Pock ’10] TGV 2 α is proper, convex, lower semi-continuous TGV 2 α is translation and rotation invariant TGV 2 α + � · � 1 gives the Banach space BGV 2 α (Ω) α ) = Π 1 affine functions ker(TGV 2 TGV 2 α measures piecewise affine only at the interfaces ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 17 / 51

  20. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Properties of TGV 2 α Basic properties: [B./Kunisch/Pock ’10] TGV 2 α is proper, convex, lower semi-continuous TGV 2 α is translation and rotation invariant TGV 2 α + � · � 1 gives the Banach space BGV 2 α (Ω) α ) = Π 1 affine functions ker(TGV 2 TGV 2 α measures piecewise affine only at the interfaces ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 17 / 51

  21. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Properties of TGV 2 α Basic properties: [B./Kunisch/Pock ’10] TGV 2 α is proper, convex, lower semi-continuous TGV 2 α is translation and rotation invariant TGV 2 α + � · � 1 gives the Banach space BGV 2 α (Ω) α ) = Π 1 affine functions ker(TGV 2 TGV 2 α measures piecewise affine only at the interfaces ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 17 / 51

  22. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Properties of TGV 2 α Basic properties: [B./Kunisch/Pock ’10] TGV 2 α is proper, convex, lower semi-continuous TGV 2 α is translation and rotation invariant TGV 2 α + � · � 1 gives the Banach space BGV 2 α (Ω) α ) = Π 1 affine functions ker(TGV 2 TGV 2 α measures piecewise affine only at the interfaces Advanced properties: [B./Valkonen ’11] BGV 2 α (Ω) = BV(Ω) in the sense of equivalent norms TGV 2 α is coercive in the sense � u − Pu � BV ≤ C TGV 2 α ( u ) for P : L 1 (Ω) → Π 1 continuous projection ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 17 / 51

  23. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Properties of TGV 2 α Basic properties: [B./Kunisch/Pock ’10] TGV 2 α is proper, convex, lower semi-continuous TGV 2 α is translation and rotation invariant TGV 2 α + � · � 1 gives the Banach space BGV 2 α (Ω) α ) = Π 1 affine functions ker(TGV 2 TGV 2 α measures piecewise affine only at the interfaces Advanced properties: [B./Valkonen ’11] BGV 2 α (Ω) = BV(Ω) in the sense of equivalent norms TGV 2 α is coercive in the sense � u − Pu � BV ≤ C TGV 2 α ( u ) for P : L 1 (Ω) → Π 1 continuous projection ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 17 / 51

  24. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Existence and stability Theorem: [B./Valkonen ’11]   1 < p ≤ d / ( d − 1) Optimization problem     K : L p (Ω) → H 1     2 � Ku − f � 2 min linear and continuous, ⇒ u ∈ L p (Ω) + TGV 2 α ( u )   H Hilbert space       K injective on Π 1 possesses a solution ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 18 / 51

  25. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Existence and stability Theorem: [B./Valkonen ’11]   1 < p ≤ d / ( d − 1) Optimization problem     K : L p (Ω) → H 1     2 � Ku − f � 2 min linear and continuous, ⇒ u ∈ L p (Ω) + TGV 2 α ( u )   H Hilbert space       K injective on Π 1 possesses a solution � u n ⇀ u in L p (Ω) (subseq.) Stability: f n → f in H ⇒ TGV 2 α ( u n ) → TGV 2 α ( u ) ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 18 / 51

  26. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Example: Denoising a “cartoon” image � u − f � 2 + TGV 2 Solve: min α ( u ) 2 u ∈ L 2 (Ω) noisy image ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 19 / 51

  27. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Example: Denoising a “cartoon” image � u − f � 2 + TGV 2 Solve: min α ( u ) 2 u ∈ L 2 (Ω) TV regularization ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 19 / 51

  28. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Example: Denoising a “cartoon” image � u − f � 2 + TGV 2 Solve: min α ( u ) 2 u ∈ L 2 (Ω) TGV 2 α regularization ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 19 / 51

  29. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Electron tomography Joint work with Georg Haberfehlner and Richard Huber Scanning Transmission Electron Microscopy (STEM): Focused electron beam High Angle Annular Dark Field (HAADF) � non Bragg-scattered electrons � proportional to mass-thickness � Reconstruction via Radon inversion ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 20 / 51

  30. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Regularized reconstruction Reconstruction of one slice: Sinogram Projections ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 21 / 51

  31. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Regularized reconstruction Reconstruction of one slice: Filtered back-projection Projections ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 21 / 51

  32. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Regularized reconstruction Reconstruction of one slice: TV-regularization Projections ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 21 / 51

  33. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Regularized reconstruction Reconstruction of one slice: TGV-regularization Projections ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 21 / 51

  34. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Outline 1 Infimal convolution regularization 2 Total generalized variation Definition and properties Applications 3 Infimal convolution TGV Accelerated dynamic MRI 4 Infimal convolution of oscillation TGV Oscillation TGV and infimal convolution Numerical realization Applications 5 Summary ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 22 / 51

  35. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Regularization for image sequences Motivation: Straightforward spatio-temporal regularization � � ǫ | ∂ x 1 u | 2 + ǫ | ∂ x 2 u | 2 + | ∂ t u | 2 d t d x TV ǫ x ( u ) = [0 , T ] × Ω ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 23 / 51

  36. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Regularization for image sequences Motivation: Straightforward spatio-temporal regularization � � ǫ | ∂ x 1 u | 2 + ǫ | ∂ x 2 u | 2 + | ∂ t u | 2 d t d x TV ǫ x ( u ) = [0 , T ] × Ω Compressed TV ǫ x ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 23 / 51

  37. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Regularization for image sequences Motivation: Straightforward spatio-temporal regularization � � | ∂ x 1 u | 2 + | ∂ x 2 u | 2 + ǫ | ∂ t u | 2 d t d x TV ǫ t ( u ) = [0 , T ] × Ω ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 23 / 51

  38. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Regularization for image sequences Motivation: Straightforward spatio-temporal regularization � � | ∂ x 1 u | 2 + | ∂ x 2 u | 2 + ǫ | ∂ t u | 2 d t d x TV ǫ t ( u ) = [0 , T ] × Ω Compressed TV ǫ t ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 23 / 51

  39. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Regularization for image sequences Motivation: Straightforward spatio-temporal regularization � � | ∂ x 1 u | 2 + | ∂ x 2 u | 2 + ǫ | ∂ t u | 2 d t d x TV ǫ t ( u ) = [0 , T ] × Ω TV ǫ t ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 23 / 51

  40. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Regularization for image sequences Motivation: Straightforward spatio-temporal regularization � � | ∂ x 1 u | 2 + | ∂ x 2 u | 2 + ǫ | ∂ t u | 2 d t d x TV ǫ t ( u ) = [0 , T ] × Ω ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 23 / 51

  41. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Regularization for image sequences Motivation: Straightforward spatio-temporal regularization � � | ∂ x 1 u | 2 + | ∂ x 2 u | 2 + ǫ | ∂ t u | 2 d t d x TV ǫ t ( u ) = [0 , T ] × Ω Approach: Optimal balancing with infimal convolution (IC) ICTV ǫ = TV ǫ x � TV ǫ t ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 23 / 51

  42. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Regularization for image sequences Motivation: Straightforward spatio-temporal regularization � � | ∂ x 1 u | 2 + | ∂ x 2 u | 2 + ǫ | ∂ t u | 2 d t d x TV ǫ t ( u ) = [0 , T ] × Ω Approach: Optimal balancing with infimal convolution (IC) ICTV ǫ = TV ǫ x � TV ǫ t Compressed ICTV ǫ ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 23 / 51

  43. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Regularization for image sequences Motivation: Straightforward spatio-temporal regularization � � | ∂ x 1 u | 2 + | ∂ x 2 u | 2 + ǫ | ∂ t u | 2 d t d x TV ǫ t ( u ) = [0 , T ] × Ω Approach: Optimal balancing with infimal convolution (IC) ICTV ǫ = TV ǫ x � TV ǫ t ICTV ǫ ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 23 / 51

  44. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Regularization for image sequences Motivation: Straightforward spatio-temporal regularization � � | ∂ x 1 u | 2 + | ∂ x 2 u | 2 + ǫ | ∂ t u | 2 d t d x TV ǫ t ( u ) = [0 , T ] × Ω Approach: Optimal balancing with infimal convolution (IC) ICTV ǫ = TV ǫ x � TV ǫ t u 1 u 2 ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 23 / 51

  45. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Regularization for image sequences Motivation: Straightforward spatio-temporal regularization � � | ∂ x 1 u | 2 + | ∂ x 2 u | 2 + ǫ | ∂ t u | 2 d t d x TV ǫ t ( u ) = [0 , T ] × Ω Approach: Optimal balancing with infimal convolution (IC) ICTV ǫ = TV ǫ x � TV ǫ t � Use infimal convolution of TGV u 1 u 2 ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 23 / 51

  46. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Infimal convolution TGV Definition: Anisotropic Total Generalized Variation �� � u div k v d x TGV k � � v ∈ C k c (Ω , Sym k (I R d )) , β ( u ) = sup � Ω � div l v � ∞ ,β ∗ l ≤ 1, l=0,. . . ,k-1 β = ( | · | β 0 , . . . , | · | β k − 1 ) tensor norms � v l � ∞ ,β ∗ l = sup x ∈ Ω | v l ( x ) | β ∗ l , | · | β ∗ l dual norm ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 24 / 51

  47. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Infimal convolution TGV Definition: Anisotropic Total Generalized Variation �� � u div k v d x TGV k � � v ∈ C k c (Ω , Sym k (I R d )) , β ( u ) = sup � Ω � div l v � ∞ ,β ∗ l ≤ 1, l=0,. . . ,k-1 β = ( | · | β 0 , . . . , | · | β k − 1 ) tensor norms � v l � ∞ ,β ∗ l = sup x ∈ Ω | v l ( x ) | β ∗ l , | · | β ∗ l dual norm Infimal convolution TGV: [Holler/Kunisch ’14] ICTGV k β = TGV k 1 β 1 � . . . � TGV k m β m ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 24 / 51

  48. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Infimal convolution TGV Definition: Anisotropic Total Generalized Variation �� � u div k v d x TGV k � v ∈ C k � c (Ω , Sym k (I R d )) , β ( u ) = sup � Ω � div l v � ∞ ,β ∗ l ≤ 1, l=0,. . . ,k-1 β = ( | · | β 0 , . . . , | · | β k − 1 ) tensor norms � v l � ∞ ,β ∗ l = sup x ∈ Ω | v l ( x ) | β ∗ l , | · | β ∗ l dual norm Infimal convolution TGV: [Holler/Kunisch ’14] ICTGV k β = TGV k 1 β 1 � . . . � TGV k m β m Regularization properties: Each TGV k i β i has finite-dimensional kernel + embedding � � � u � 1 + TGV k i � u � BV ≤ C β i ( u ) � ICTGV is a regularizer ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 24 / 51

  49. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Accelerated dynamic MRI Joint work with Martin Holler and Matthias Schl¨ ogl Dynamic MRI: Acquire image sequences of moving objects Due to limited acquisition speed, only low-resolution images are obtainable sum of squares ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 25 / 51

  50. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Accelerated dynamic MRI Joint work with Martin Holler and Matthias Schl¨ ogl Dynamic MRI: Acquire image sequences of moving objects Due to limited acquisition speed, only low-resolution images are obtainable sum of squares ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 25 / 51

  51. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Accelerated dynamic MRI Joint work with Martin Holler and Matthias Schl¨ ogl Dynamic MRI: Acquire image sequences of moving objects Due to limited acquisition speed, only low-resolution images are obtainable sum of squares Goal: Improve spatio-temporal resolution by undersampling reconstruction ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 25 / 51

  52. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Accelerated dynamic MRI Joint work with Martin Holler and Matthias Schl¨ ogl Dynamic MRI: Acquire image sequences of moving objects Due to limited acquisition speed, only low-resolution images are obtainable sum of squares Goal: Improve spatio-temporal resolution by undersampling reconstruction � Apply ICTGV regularized regularization ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 25 / 51

  53. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING The variational model Minimization problem: λ � 2 � K t , c ( u t ) − d t , c � 2 2 + ICTGV 2 min β ( u ) u t , c K t , c ( u t ) = M t F ( u t σ c ) masked Fourier transform ( σ c complex coil sensitivities) ICTGV 2 β = TGV 2 β 1 � TGV 2 β 2 ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 26 / 51

  54. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING The variational model Minimization problem: λ � 2 � K t , c ( u t ) − d t , c � 2 2 + ICTGV 2 min β ( u ) u t , c K t , c ( u t ) = M t F ( u t σ c ) masked Fourier transform ( σ c complex coil sensitivities) ICTGV 2 β = TGV 2 β 1 � TGV 2 β 2 Primal-dual algorithm: Guaranteed convergence, duality-based stopping criterion GPU-optimized version: ≈ 160 seconds including coil-sensitivity estimation (NVidia GeForce GTX770 with AGILE library) ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 26 / 51

  55. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Numerical test Acceleration factor 8: Reference data Unregularized reconstruction ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 27 / 51

  56. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Numerical test Acceleration factor 8: Reference data Unregularized reconstruction ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 27 / 51

  57. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Numerical test Acceleration factor 8: Low rank + sparse model Difference ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 27 / 51

  58. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Numerical test Acceleration factor 8: ICTGV model Difference ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 27 / 51

  59. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Numerical test Acceleration factor 16: ICTGV model Difference ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 27 / 51

  60. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Numerical test Acceleration factor 8: Slow component Fast component ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 27 / 51

  61. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Numerical test Acceleration factor 8: Slow component Fast component Favorable quantitative comparison 2nd place at the ISMRM 2013 reconstruction challenge ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 27 / 51

  62. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Outline 1 Infimal convolution regularization 2 Total generalized variation Definition and properties Applications 3 Infimal convolution TGV Accelerated dynamic MRI 4 Infimal convolution of oscillation TGV Oscillation TGV and infimal convolution Numerical realization Applications 5 Summary ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 28 / 51

  63. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Motivation: Denoising “barbara” noisy image ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 29 / 51

  64. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Motivation: Denoising “barbara” TGV 2 α regularization ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 29 / 51

  65. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Motivation: Denoising “barbara” TGV 2 α regularization � Capture oscillatory structures � TGV osci α,β, c ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 29 / 51

  66. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Oscillation TGV Joint work with Yiming Gao Idea: Choose differential equation with oscillatory functions as solutions ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 30 / 51

  67. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Oscillation TGV Joint work with Yiming Gao Idea: Choose differential equation with oscillatory functions as solutions R d , ω � = 0 ∇ 2 u + c u = 0 , c = ω ⊗ ω, ω ∈ I Solutions: u ( x ) = C 1 cos( ω · x ) + C 2 sin( ω · x ) , C 1 , C 2 ∈ I R ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 30 / 51

  68. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Oscillation TGV Joint work with Yiming Gao Idea: Choose differential equation with oscillatory functions as solutions R d , ω � = 0 ∇ 2 u + c u = 0 , c = ω ⊗ ω, ω ∈ I Solutions: u ( x ) = C 1 cos( ω · x ) + C 2 sin( ω · x ) , C 1 , C 2 ∈ I R Adapt second-order TGV to the corresponding operator ∇ 2 u + c u = 0 ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 30 / 51

  69. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Oscillation TGV Joint work with Yiming Gao Idea: Choose differential equation with oscillatory functions as solutions ∇ 2 u + c u = 0 , R d , ω � = 0 c = ω ⊗ ω, ω ∈ I Solutions: u ( x ) = C 1 cos( ω · x ) + C 2 sin( ω · x ) , C 1 , C 2 ∈ I R Adapt second-order TGV to the corresponding operator ∇ u − w = 0 , E w + c u = 0 ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 30 / 51

  70. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Oscillation TGV Joint work with Yiming Gao Idea: Choose differential equation with oscillatory functions as solutions ∇ 2 u + c u = 0 , R d , ω � = 0 c = ω ⊗ ω, ω ∈ I Solutions: u ( x ) = C 1 cos( ω · x ) + C 2 sin( ω · x ) , C 1 , C 2 ∈ I R Adapt second-order TGV to the corresponding operator ∇ u − w = 0 , E w + c u = 0 Total generalized variation (second order): � � TGV 2 α,β ( u ) = w ∈ BD(Ω) α min d |∇ u − w | + β d |E w | Ω Ω α > 0, β > 0 ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 30 / 51

  71. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Oscillation TGV Joint work with Yiming Gao Idea: Choose differential equation with oscillatory functions as solutions ∇ 2 u + c u = 0 , R d , ω � = 0 c = ω ⊗ ω, ω ∈ I Solutions: u ( x ) = C 1 cos( ω · x ) + C 2 sin( ω · x ) , C 1 , C 2 ∈ I R Adapt second-order TGV to the corresponding operator ∇ u − w = 0 , E w + c u = 0 Oscillation total generalized variation: � � TGV osci α,β, c ( u ) = w ∈ BD(Ω) α min d |∇ u − w | + β d |E w + c u | Ω Ω R d , ω � = 0 α > 0, β > 0, c = ω ⊗ ω, ω ∈ I ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 30 / 51

  72. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Properties of TGV osci α,β, c Basic properties: [Gao/B. ’17] TGV osci α,β, c is proper, convex, lower semi-continuous TGV osci α,β, c is translation and rotation invariant TGV osci α,β, c + � · � 1 gives the Banach space BGV osci α,β, c (Ω) ker(TGV osci α,β, c ) = span { x �→ cos( ω · x ) , x �→ sin( ω · x ) } TGV osci α,β, c measures piecewise oscillations only at interfaces ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 31 / 51

  73. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Properties of TGV osci α,β, c Basic properties: [Gao/B. ’17] TGV osci α,β, c is proper, convex, lower semi-continuous TGV osci α,β, c is translation and rotation invariant TGV osci α,β, c + � · � 1 gives the Banach space BGV osci α,β, c (Ω) ker(TGV osci α,β, c ) = span { x �→ cos( ω · x ) , x �→ sin( ω · x ) } TGV osci α,β, c measures piecewise oscillations only at interfaces ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 31 / 51

  74. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Properties of TGV osci α,β, c Basic properties: [Gao/B. ’17] TGV osci α,β, c is proper, convex, lower semi-continuous TGV osci α,β, c is translation and rotation invariant TGV osci α,β, c + � · � 1 gives the Banach space BGV osci α,β, c (Ω) ker(TGV osci α,β, c ) = span { x �→ cos( ω · x ) , x �→ sin( ω · x ) } TGV osci α,β, c measures piecewise oscillations only at interfaces ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 31 / 51

  75. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Properties of TGV osci α,β, c Basic properties: [Gao/B. ’17] TGV osci α,β, c is proper, convex, lower semi-continuous TGV osci α,β, c is translation and rotation invariant TGV osci α,β, c + � · � 1 gives the Banach space BGV osci α,β, c (Ω) ker(TGV osci α,β, c ) = span { x �→ cos( ω · x ) , x �→ sin( ω · x ) } TGV osci α,β, c measures piecewise oscillations only at interfaces ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 31 / 51

  76. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Properties of TGV osci α,β, c Basic properties: [Gao/B. ’17] TGV osci α,β, c is proper, convex, lower semi-continuous TGV osci α,β, c is translation and rotation invariant TGV osci α,β, c + � · � 1 gives the Banach space BGV osci α,β, c (Ω) ker(TGV osci α,β, c ) = span { x �→ cos( ω · x ) , x �→ sin( ω · x ) } TGV osci α,β, c measures piecewise oscillations only at interfaces ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 31 / 51

  77. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Properties of TGV osci α,β, c Basic properties: [Gao/B. ’17] TGV osci α,β, c is proper, convex, lower semi-continuous TGV osci α,β, c is translation and rotation invariant TGV osci α,β, c + � · � 1 gives the Banach space BGV osci α,β, c (Ω) ker(TGV osci α,β, c ) = span { x �→ cos( ω · x ) , x �→ sin( ω · x ) } TGV osci α,β, c measures piecewise oscillations only at interfaces Advanced properties: BGV osci α,β, c (Ω) = BV(Ω) in the sense of equivalent norms TGV osci α,β, c is coercive in the sense � u − Pu � BV ≤ C TGV osci α,β, c ( u ) for P : L 1 (Ω) → ker(TGV osci α,β, c ) continuous projection ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 31 / 51

  78. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Properties of TGV osci α,β, c Basic properties: [Gao/B. ’17] TGV osci α,β, c is proper, convex, lower semi-continuous TGV osci α,β, c is translation and rotation invariant TGV osci α,β, c + � · � 1 gives the Banach space BGV osci α,β, c (Ω) ker(TGV osci α,β, c ) = span { x �→ cos( ω · x ) , x �→ sin( ω · x ) } TGV osci α,β, c measures piecewise oscillations only at interfaces Advanced properties: BGV osci α,β, c (Ω) = BV(Ω) in the sense of equivalent norms TGV osci α,β, c is coercive in the sense � u − Pu � BV ≤ C TGV osci α,β, c ( u ) for P : L 1 (Ω) → ker(TGV osci α,β, c ) continuous projection ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 31 / 51

  79. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Infimal convolution of TGV osci Next steps: Separate cartoon components from oscillatory components Allow for multiple directions and frequencies � m -fold infimal convolution ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 32 / 51

  80. INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Infimal convolution of TGV osci Next steps: Separate cartoon components from oscillatory components Allow for multiple directions and frequencies � m -fold infimal convolution Infimal convolution of TGV osci : ICTGV osci c ( u ) = (TGV osci α 1 ,β 1 , c 1 � . . . � TGV osci α m ,β m , c m )( u ) α,� � β,� R d α i > 0, β i > 0, c i = ω i ⊗ ω i , ω i ∈ I ICTGV osci Infimal convolution TGV ICTGV Summary K. Bredies 32 / 51

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend