Infimal-convolution-type regularization for inverse problems in - - PowerPoint PPT Presentation

▶
infimal convolution type regularization for inverse
SMART_READER_LITE
LIVE PREVIEW

Infimal-convolution-type regularization for inverse problems in - - PowerPoint PPT Presentation

Sep 23, 2022 463 likes •2k views

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING Infimal-convolution-type regularization for inverse problems in imaging Kristian Bredies Institute for Mathematics and Scientific Computing University of Graz IHP semester The Mathematics

slide-1
SLIDE 1

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal-convolution-type regularization for inverse problems in imaging

Kristian Bredies

Institute for Mathematics and Scientific Computing University of Graz

IHP semester “The Mathematics of Imaging”

Workshop “Variational methods and optimization in imaging”

February 7th, 2019

  • K. Bredies

1 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-2
SLIDE 2

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Motivation

Goals: Find flexible models for stable solution

  • f inverse problems in function spaces

Establish meaningful combined models In particular: Combine multiple smoothness orders Separate image characteristics (e.g. cartoon/texture) Apply broadly to imaging problems (denoising, inpainting, inverse problems/reconstruction)

  • K. Bredies

2 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-3
SLIDE 3

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Motivation

Goals: Find flexible models for stable solution

  • f inverse problems in function spaces

Establish meaningful combined models In particular: Combine multiple smoothness orders Separate image characteristics (e.g. cartoon/texture) Apply broadly to imaging problems (denoising, inpainting, inverse problems/reconstruction)

  • K. Bredies

2 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-4
SLIDE 4

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Motivation

↓ Goals: Find flexible models for stable solution

  • f inverse problems in function spaces

Establish meaningful combined models In particular: Combine multiple smoothness orders Separate image characteristics (e.g. cartoon/texture) Apply broadly to imaging problems (denoising, inpainting, inverse problems/reconstruction)

  • K. Bredies

2 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-5
SLIDE 5

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Motivation

↓ Goals: Find flexible models for stable solution

  • f inverse problems in function spaces

Establish meaningful combined models In particular: Combine multiple smoothness orders Separate image characteristics (e.g. cartoon/texture) Apply broadly to imaging problems (denoising, inpainting, inverse problems/reconstruction)

  • K. Bredies

2 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-6
SLIDE 6

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Motivation

↓ Goals: Find flexible models for stable solution

  • f inverse problems in function spaces

Establish meaningful combined models In particular: Combine multiple smoothness orders Separate image characteristics (e.g. cartoon/texture) Apply broadly to imaging problems (denoising, inpainting, inverse problems/reconstruction)

  • K. Bredies

2 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-7
SLIDE 7

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Motivation

↓ Goals: Find flexible models for stable solution

  • f inverse problems in function spaces

Establish meaningful combined models In particular: Combine multiple smoothness orders Separate image characteristics (e.g. cartoon/texture) Apply broadly to imaging problems (denoising, inpainting, inverse problems/reconstruction) Here: Infimal-convolution regularizers

  • K. Bredies

2 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-8
SLIDE 8

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Outline

1 Infimal convolution regularization 2 Total generalized variation

Definition and properties Applications

3 Infimal convolution TGV

Accelerated dynamic MRI

4 Infimal convolution of oscillation TGV

Oscillation TGV and infimal convolution Numerical realization Applications

5 Summary

  • K. Bredies

3 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-9
SLIDE 9

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Outline

1 Infimal convolution regularization 2 Total generalized variation

Definition and properties Applications

3 Infimal convolution TGV

Accelerated dynamic MRI

4 Infimal convolution of oscillation TGV

Oscillation TGV and infimal convolution Numerical realization Applications

5 Summary

  • K. Bredies

4 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-10
SLIDE 10

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

The infimal convolution

Definition: Φ1, Φ2 : X → ]−∞, ∞] proper, convex, l.s.c. (Φ1Φ2)(u) = inf

u1+u2=u Φ1(u1) + Φ2(u2)

Infimal convolution of Φ1 and Φ2 Φ1Φ2 is exact if the infimum is always attained

  • K. Bredies

5 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-11
SLIDE 11

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

The infimal convolution

Definition: Φ1, Φ2 : X → ]−∞, ∞] proper, convex, l.s.c. (Φ1Φ2)(u) = inf

u1+u2=u Φ1(u1) + Φ2(u2)

Infimal convolution of Φ1 and Φ2 Φ1Φ2 is exact if the infimum is always attained m-fold infimal convolution: (Φ1 . . . Φm)(u) = inf

u1+...um=u m

  • i=1

Φi(ui)

  • K. Bredies

5 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-12
SLIDE 12

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

The infimal convolution

Definition: Φ1, Φ2 : X → ]−∞, ∞] proper, convex, l.s.c. (Φ1Φ2)(u) = inf

u1+u2=u Φ1(u1) + Φ2(u2)

Infimal convolution of Φ1 and Φ2 Φ1Φ2 is exact if the infimum is always attained m-fold infimal convolution: (Φ1 . . . Φm)(u) = inf

u1+...um=u m

  • i=1

Φi(ui) Optimal balancing of m competing regularization functionals

  • K. Bredies

5 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-13
SLIDE 13

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Basic variational models

↓ Smoothness-based: Total variation Φ(u) =

  • Ω

d|∇u| [Rudin/Osher/Fatemi ’92] Accounts for edges Unaware of higher-order smoothness staircasing effect

  • K. Bredies

6 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-14
SLIDE 14

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Basic variational models

↓ Smoothness-based: Total variation Φ(u) =

  • Ω

d|∇u| [Rudin/Osher/Fatemi ’92] Accounts for edges Unaware of higher-order smoothness staircasing effect Higher-order TV Φ(u) =

  • Ω

d|∇2u| [Lysaker/Lundervold/Tai ’03] [Hinterberger/Scherzer ’04] Favors smooth solutions Edges are not preserved

  • K. Bredies

6 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-15
SLIDE 15

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal-convolution models

↓ Smoothness-based: TV-TV2 infimal convolution Φ(u) = min

u=u1+u2

  • Ω

d|∇u1| + β

  • Ω

d|∇2u2| [Chambolle/Lions ’97] Models piecewise smooth images Staircase effect dominates solutions

  • K. Bredies

7 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-16
SLIDE 16

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal-convolution models

↓ Smoothness-based: TV-TV2 infimal convolution Φ(u) = min

u=u1+u2

  • Ω

d|∇u1| + β

  • Ω

d|∇2u2| [Chambolle/Lions ’97] Models piecewise smooth images Staircase effect dominates solutions TV-L1 ◦ ∆ infimal convolution Φ(u) = min

u=u1+u2

  • Ω

d|∇u1| + β

  • Ω

d|∆u2| [Chan/Esedoglu/Park ’05] Denoising similar to TV-TV2 infimal convolution

  • K. Bredies

7 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-17
SLIDE 17

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal-convolution-type models

↓ Variations: Total generalized variation of order 2 Φ(u) = TGV2

α(u)

= min

∇u=u1+u2 α1

  • Ω

d|u1| + α0

  • Ω

d|Eu2| [B./Kunisch/Pock ’10] Models piecewise smooth images Favors smoothness where given Preserves relevant edges

  • K. Bredies

8 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-18
SLIDE 18

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal-convolution-type models

↓ Variations: Total generalized variation of order 2 Φ(u) = TGV2

α(u)

= min

∇u=u1+u2 α1

  • Ω

d|u1| + α0

  • Ω

d|Eu2| [B./Kunisch/Pock ’10] Models piecewise smooth images Favors smoothness where given Preserves relevant edges Other variants by [Setzer/Steidl/Teuber ’11]

  • K. Bredies

8 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-19
SLIDE 19

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Cartoon/texture decomposition

↓

  • Infimal convolution models:

TV-G-norm decomposition Φ(u) = min

u1+u2=u

  • Ω

d|∇u1| + βu2TV∗ [Osher/Vese ’03], [Aujol et al. ’05] TV-H−1-norm decomposition Φ(u) = min

u1+u2=u

  • Ω

d|∇u1| + βu2H−1 [Osher/Sole/Vese ’03] Separates piecewise smooth and

  • scillatory parts

G-norm/H−1-norm part also captures non-oscillations

  • K. Bredies

9 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-20
SLIDE 20

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Cartoon/texture decomposition

↓

  • Infimal convolution models:

TV-G-norm decomposition Φ(u) = min

u1+u2=u

  • Ω

d|∇u1| + βu2TV∗ [Osher/Vese ’03], [Aujol et al. ’05] TV-H−1-norm decomposition Φ(u) = min

u1+u2=u

  • Ω

d|∇u1| + βu2H−1 [Osher/Sole/Vese ’03] Separates piecewise smooth and

  • scillatory parts

G-norm/H−1-norm part also captures non-oscillations

  • K. Bredies

9 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-21
SLIDE 21

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution for inverse problems

Tikhonov regularization: Solve min

u∈X S(Ku, f ) + Φ(u)

S discrepancy for Ku = f , Φ regularization functional

  • K. Bredies

10 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-22
SLIDE 22

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution for inverse problems

Tikhonov regularization: Solve min

u∈X S(Ku, f ) + Φ(u)

S discrepancy for Ku = f , Φ regularization functional Two viewpoints:

1 Monolithic regularization

min

u∈X S(Ku, f ) + Φ(u),

Φ = Φ1 . . . Φm

2 Vector-valued regularization

min

(u1,...,um)∈X m S

  • K(u1 + . . . + um), f
  • + Φ1(u1) + . . . Φm(um)
  • K. Bredies

10 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-23
SLIDE 23

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution for inverse problems

Tikhonov regularization: Solve min

u∈X S(Ku, f ) + Φ(u)

S discrepancy for Ku = f , Φ regularization functional Two viewpoints:

1 Monolithic regularization

min

u∈X S(Ku, f ) + Φ(u),

Φ = Φ1 . . . Φm

2 Vector-valued regularization

min

(u1,...,um)∈X m S

  • K(u1 + . . . + um), f
  • + Φ1(u1) + . . . Φm(um)

Monolithic regularization: Intrinsic decomposition & problem-adapted interpretation

  • K. Bredies

10 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-24
SLIDE 24

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

General existence result

Theorem: X reflexive Banach space, Y normed space K : X → Y bounded linear S(·, f ) : Y → [0, ∞] proper, convex, l.s.c. Φ : X ∈ [0, ∞] proper, convex, l.s.c. Φ 1-homogeneous, dim ker(Φ) < ∞, and coercive, i.e., u − PuX ≤ CΦ(u) for P : X → ker(Φ) bounded linear projector Then: min

u∈X S(Ku, f ) + αΦ(u)

has a solution for each α > 0

  • K. Bredies

11 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-25
SLIDE 25

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

General regularization properties

min

u∈X S(Ku, f ) + αΦ(u)

Let: Y Hilbert space, S(v, f ) = 1

2v − f 2 Y

Then: (Subsequential) stability for varying f Noise level δ → 0 and α → 0, δ2/α → 0 ⇒ (Subsequential) convergence Source condition K ∗w ∈ ∂Φ(u†), α ∼ δ ⇒ O(δ) for the Bregman distance w.r.t. Φ

  • K. Bredies

12 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-26
SLIDE 26

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

General regularization properties

min

u∈X S(Ku, f ) + αΦ(u)

Let: Y Hilbert space, S(v, f ) = 1

2v − f 2 Y

Then: (Subsequential) stability for varying f Noise level δ → 0 and α → 0, δ2/α → 0 ⇒ (Subsequential) convergence Source condition K ∗w ∈ ∂Φ(u†), α ∼ δ ⇒ O(δ) for the Bregman distance w.r.t. Φ Difficulties: dim ker(Φ) = ∞ ⇒ K has to be stably invertible on ker(Φ) Φ not coercive ⇒ K has to be stably invertible on X

  • K. Bredies

12 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-27
SLIDE 27

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution regularization

Lemma: Φ1, Φ2 : X → [0, ∞] proper, convex, l.s.c. Φi 1-homogeneous, dim ker(Φ) < ∞ for i = 1, 2 Then: Φ1Φ2 : X → [0, ∞] is exact, proper, convex, l.s.c., 1-homogeneous, ker Φ1Φ2 = ker Φ1 + ker Φ2

  • K. Bredies

13 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-28
SLIDE 28

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution regularization

Lemma: Φ1, Φ2 : X → [0, ∞] proper, convex, l.s.c. Φi 1-homogeneous, dim ker(Φ) < ∞ for i = 1, 2 Then: Φ1Φ2 : X → [0, ∞] is exact, proper, convex, l.s.c., 1-homogeneous, ker Φ1Φ2 = ker Φ1 + ker Φ2 Lemma: Additionally: X ֒ − ֒ → Z, Z Banach space and uX ≤ C

  • uZ + Φi(u)
  • for i = 1, 2 and all u ∈ X

Then: Φ1Φ2 is coercive, i.e., u − PuX ≤ CΦ(u) for all u ∈ X

  • K. Bredies

13 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-29
SLIDE 29

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution regularization

Lemma: Φ1, Φ2 : X → [0, ∞] proper, convex, l.s.c. Φi 1-homogeneous, dim ker(Φ) < ∞ for i = 1, 2 Then: Φ1Φ2 : X → [0, ∞] is exact, proper, convex, l.s.c., 1-homogeneous, ker Φ1Φ2 = ker Φ1 + ker Φ2 Lemma: Additionally: X ֒ − ֒ → Z, Z Banach space and uX ≤ C

  • uZ + Φi(u)
  • for i = 1, 2 and all u ∈ X

Then: Φ1Φ2 is coercive, i.e., u − PuX ≤ CΦ(u) for all u ∈ X sufficient conditions for a regularizer

  • K. Bredies

13 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-30
SLIDE 30

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Examples

TV-TV2 infimal convolution: TV and TV2 have finite-dimensional kernels + embedding: uBV ≤ uBV2 ≤ C

  • u1 + TV2(u)
  • Consequently: TV β TV2 is a regularizer (β > 0)
  • K. Bredies

14 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-31
SLIDE 31

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Examples

TV-TV2 infimal convolution: TV and TV2 have finite-dimensional kernels + embedding: uBV ≤ uBV2 ≤ C

  • u1 + TV2(u)
  • Consequently: TV β TV2 is a regularizer (β > 0)

TV-L1 ◦ ∆ infimal convolution: ker(TV · M ◦ ∆) = {u ∈ BV | u harmonic} Infinite-dimensional kernel ⇒ not a regularizer

  • K. Bredies

14 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-32
SLIDE 32

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Examples

TV-TV2 infimal convolution: TV and TV2 have finite-dimensional kernels + embedding: uBV ≤ uBV2 ≤ C

  • u1 + TV2(u)
  • Consequently: TV β TV2 is a regularizer (β > 0)

TV-L1 ◦ ∆ infimal convolution: ker(TV · M ◦ ∆) = {u ∈ BV | u harmonic} Infinite-dimensional kernel ⇒ not a regularizer TV-G-norm/TV-H−1 infimal convolution: · TV∗/ · H−1 not coercive in Lp-spaces Infimal convolution not coercive ⇒ not a regularizer

  • K. Bredies

14 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-33
SLIDE 33

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Examples

TV-TV2 infimal convolution: TV and TV2 have finite-dimensional kernels + embedding: uBV ≤ uBV2 ≤ C

  • u1 + TV2(u)
  • Consequently: TV β TV2 is a regularizer (β > 0)

TV-L1 ◦ ∆ infimal convolution: ker(TV · M ◦ ∆) = {u ∈ BV | u harmonic} Infinite-dimensional kernel ⇒ not a regularizer TV-G-norm/TV-H−1 infimal convolution: · TV∗/ · H−1 not coercive in Lp-spaces Infimal convolution not coercive ⇒ not a regularizer Develop regularizing cartoon/texture models + complex smoothness-based regularizers

  • K. Bredies

14 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-34
SLIDE 34

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Outline

1 Infimal convolution regularization 2 Total generalized variation

Definition and properties Applications

3 Infimal convolution TGV

Accelerated dynamic MRI

4 Infimal convolution of oscillation TGV

Oscillation TGV and infimal convolution Numerical realization Applications

5 Summary

  • K. Bredies

15 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-35
SLIDE 35

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Total generalized variation

noisy image TV-regularization

Motivation: TV-based first-order regularization favors certain artifacts Total generalized variation: [B./Kunisch/Pock ’10] TGVk

α(u) = sup

  • Ω

u divk v dx

  • v ∈ Ck

c (Ω, Symk(I

Rd)), divl v∞ ≤ αl, l = 0, . . . , k − 1

  • α = (α0, . . . , αk−1) > 0 weights

Second-order version:

TGV2

α(u)=

min

w∈BD(Ω)

α1∇u−wM+α0EwM

  • K. Bredies

16 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-36
SLIDE 36

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Total generalized variation

noisy image TGV2

α-regularization

Motivation: TV-based first-order regularization favors certain artifacts Total generalized variation: [B./Kunisch/Pock ’10] TGVk

α(u) = sup

  • Ω

u divk v dx

  • v ∈ Ck

c (Ω, Symk(I

Rd)), divl v∞ ≤ αl, l = 0, . . . , k − 1

  • α = (α0, . . . , αk−1) > 0 weights

Second-order version:

TGV2

α(u)=

min

w∈BD(Ω)

α1∇u−wM+α0EwM

  • K. Bredies

16 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-37
SLIDE 37

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of TGV2

α Basic properties: [B./Kunisch/Pock ’10] TGV2

α is proper, convex, lower semi-continuous

TGV2

α is translation and rotation invariant

TGV2

α + · 1 gives the Banach space BGV2 α(Ω)

ker(TGV2

α) = Π1 affine functions

TGV2

α measures piecewise affine only at the interfaces

  • K. Bredies

17 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-38
SLIDE 38

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of TGV2

α Basic properties: [B./Kunisch/Pock ’10] TGV2

α is proper, convex, lower semi-continuous

TGV2

α is translation and rotation invariant

TGV2

α + · 1 gives the Banach space BGV2 α(Ω)

ker(TGV2

α) = Π1 affine functions

TGV2

α measures piecewise affine only at the interfaces

  • K. Bredies

17 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-39
SLIDE 39

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of TGV2

α Basic properties: [B./Kunisch/Pock ’10] TGV2

α is proper, convex, lower semi-continuous

TGV2

α is translation and rotation invariant

TGV2

α + · 1 gives the Banach space BGV2 α(Ω)

ker(TGV2

α) = Π1 affine functions

TGV2

α measures piecewise affine only at the interfaces

  • K. Bredies

17 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-40
SLIDE 40

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of TGV2

α Basic properties: [B./Kunisch/Pock ’10] TGV2

α is proper, convex, lower semi-continuous

TGV2

α is translation and rotation invariant

TGV2

α + · 1 gives the Banach space BGV2 α(Ω)

ker(TGV2

α) = Π1 affine functions

TGV2

α measures piecewise affine only at the interfaces

  • K. Bredies

17 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-41
SLIDE 41

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of TGV2

α Basic properties: [B./Kunisch/Pock ’10] TGV2

α is proper, convex, lower semi-continuous

TGV2

α is translation and rotation invariant

TGV2

α + · 1 gives the Banach space BGV2 α(Ω)

ker(TGV2

α) = Π1 affine functions

TGV2

α measures piecewise affine only at the interfaces

  • K. Bredies

17 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-42
SLIDE 42

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of TGV2

α Basic properties: [B./Kunisch/Pock ’10] TGV2

α is proper, convex, lower semi-continuous

TGV2

α is translation and rotation invariant

TGV2

α + · 1 gives the Banach space BGV2 α(Ω)

ker(TGV2

α) = Π1 affine functions

TGV2

α measures piecewise affine only at the interfaces

Advanced properties: [B./Valkonen ’11] BGV2

α(Ω) = BV(Ω) in the sense of equivalent norms

TGV2

α is coercive in the sense

u − PuBV ≤ C TGV2

α(u)

for P : L1(Ω) → Π1 continuous projection

  • K. Bredies

17 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-43
SLIDE 43

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of TGV2

α Basic properties: [B./Kunisch/Pock ’10] TGV2

α is proper, convex, lower semi-continuous

TGV2

α is translation and rotation invariant

TGV2

α + · 1 gives the Banach space BGV2 α(Ω)

ker(TGV2

α) = Π1 affine functions

TGV2

α measures piecewise affine only at the interfaces

Advanced properties: [B./Valkonen ’11] BGV2

α(Ω) = BV(Ω) in the sense of equivalent norms

TGV2

α is coercive in the sense

u − PuBV ≤ C TGV2

α(u)

for P : L1(Ω) → Π1 continuous projection

  • K. Bredies

17 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-44
SLIDE 44

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Existence and stability

Theorem: [B./Valkonen ’11] 1 < p ≤ d/(d − 1) K : Lp(Ω) → H linear and continuous, H Hilbert space K injective on Π1          ⇒          Optimization problem min

u∈Lp(Ω)

1 2Ku − f 2 + TGV2

α(u)

possesses a solution

  • K. Bredies

18 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-45
SLIDE 45

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Existence and stability

Theorem: [B./Valkonen ’11] 1 < p ≤ d/(d − 1) K : Lp(Ω) → H linear and continuous, H Hilbert space K injective on Π1          ⇒          Optimization problem min

u∈Lp(Ω)

1 2Ku − f 2 + TGV2

α(u)

possesses a solution Stability: f n → f in H ⇒

  • un ⇀ u in Lp(Ω) (subseq.)

TGV2

α(un) → TGV2 α(u)

  • K. Bredies

18 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-46
SLIDE 46

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Denoising a “cartoon” image

Solve: min

u∈L2(Ω)

u − f 2 2 + TGV2

α(u)

noisy image

  • K. Bredies

19 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-47
SLIDE 47

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Denoising a “cartoon” image

Solve: min

u∈L2(Ω)

u − f 2 2 + TGV2

α(u)

TV regularization

  • K. Bredies

19 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-48
SLIDE 48

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Denoising a “cartoon” image

Solve: min

u∈L2(Ω)

u − f 2 2 + TGV2

α(u)

TGV2

α regularization

  • K. Bredies

19 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-49
SLIDE 49

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Electron tomography

Joint work with Georg Haberfehlner and Richard Huber Scanning Transmission Electron Microscopy (STEM): Focused electron beam High Angle Annular Dark Field (HAADF) non Bragg-scattered electrons proportional to mass-thickness Reconstruction via Radon inversion

  • K. Bredies

20 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-50
SLIDE 50

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Regularized reconstruction

Projections Reconstruction of one slice: Sinogram

  • K. Bredies

21 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-51
SLIDE 51

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Regularized reconstruction

Projections Reconstruction of one slice: Filtered back-projection

  • K. Bredies

21 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-52
SLIDE 52

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Regularized reconstruction

Projections Reconstruction of one slice: TV-regularization

  • K. Bredies

21 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-53
SLIDE 53

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Regularized reconstruction

Projections Reconstruction of one slice: TGV-regularization

  • K. Bredies

21 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-54
SLIDE 54

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Outline

1 Infimal convolution regularization 2 Total generalized variation

Definition and properties Applications

3 Infimal convolution TGV

Accelerated dynamic MRI

4 Infimal convolution of oscillation TGV

Oscillation TGV and infimal convolution Numerical realization Applications

5 Summary

  • K. Bredies

22 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-55
SLIDE 55

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Regularization for image sequences

Motivation: Straightforward spatio-temporal regularization TVǫx(u) =

  • [0,T]×Ω
  • ǫ|∂x1u|2 + ǫ|∂x2u|2 + |∂tu|2 dt dx
  • K. Bredies

23 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-56
SLIDE 56

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Regularization for image sequences

Motivation: Straightforward spatio-temporal regularization TVǫx(u) =

  • [0,T]×Ω
  • ǫ|∂x1u|2 + ǫ|∂x2u|2 + |∂tu|2 dt dx

Compressed TVǫx

  • K. Bredies

23 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-57
SLIDE 57

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Regularization for image sequences

Motivation: Straightforward spatio-temporal regularization TVǫt(u) =

  • [0,T]×Ω
  • |∂x1u|2 + |∂x2u|2 + ǫ|∂tu|2 dt dx
  • K. Bredies

23 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-58
SLIDE 58

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Regularization for image sequences

Motivation: Straightforward spatio-temporal regularization TVǫt(u) =

  • [0,T]×Ω
  • |∂x1u|2 + |∂x2u|2 + ǫ|∂tu|2 dt dx

Compressed TVǫt

  • K. Bredies

23 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-59
SLIDE 59

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Regularization for image sequences

Motivation: Straightforward spatio-temporal regularization TVǫt(u) =

  • [0,T]×Ω
  • |∂x1u|2 + |∂x2u|2 + ǫ|∂tu|2 dt dx

TVǫt

  • K. Bredies

23 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-60
SLIDE 60

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Regularization for image sequences

Motivation: Straightforward spatio-temporal regularization TVǫt(u) =

  • [0,T]×Ω
  • |∂x1u|2 + |∂x2u|2 + ǫ|∂tu|2 dt dx
  • K. Bredies

23 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-61
SLIDE 61

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Regularization for image sequences

Motivation: Straightforward spatio-temporal regularization TVǫt(u) =

  • [0,T]×Ω
  • |∂x1u|2 + |∂x2u|2 + ǫ|∂tu|2 dt dx

Approach: Optimal balancing with infimal convolution (IC) ICTVǫ = TVǫx TVǫt

  • K. Bredies

23 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-62
SLIDE 62

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Regularization for image sequences

Motivation: Straightforward spatio-temporal regularization TVǫt(u) =

  • [0,T]×Ω
  • |∂x1u|2 + |∂x2u|2 + ǫ|∂tu|2 dt dx

Approach: Optimal balancing with infimal convolution (IC) ICTVǫ = TVǫx TVǫt Compressed ICTVǫ

  • K. Bredies

23 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-63
SLIDE 63

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Regularization for image sequences

Motivation: Straightforward spatio-temporal regularization TVǫt(u) =

  • [0,T]×Ω
  • |∂x1u|2 + |∂x2u|2 + ǫ|∂tu|2 dt dx

Approach: Optimal balancing with infimal convolution (IC) ICTVǫ = TVǫx TVǫt ICTVǫ

  • K. Bredies

23 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-64
SLIDE 64

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Regularization for image sequences

Motivation: Straightforward spatio-temporal regularization TVǫt(u) =

  • [0,T]×Ω
  • |∂x1u|2 + |∂x2u|2 + ǫ|∂tu|2 dt dx

Approach: Optimal balancing with infimal convolution (IC) ICTVǫ = TVǫx TVǫt u1 u2

  • K. Bredies

23 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-65
SLIDE 65

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Regularization for image sequences

Motivation: Straightforward spatio-temporal regularization TVǫt(u) =

  • [0,T]×Ω
  • |∂x1u|2 + |∂x2u|2 + ǫ|∂tu|2 dt dx

Approach: Optimal balancing with infimal convolution (IC) ICTVǫ = TVǫx TVǫt u1 u2 Use infimal convolution

  • f TGV
  • K. Bredies

23 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-66
SLIDE 66

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution TGV

Definition: Anisotropic Total Generalized Variation TGVk

β(u) = sup

  • Ω

u divk v dx

  • v ∈ Ck

c (Ω, Symk(I

Rd)), divl v∞,β∗

l ≤ 1, l=0,. . . ,k-1

  • β = (| · |β0, . . . , | · |βk−1) tensor norms

vl∞,β∗

l = supx∈Ω |vl(x)|β∗ l , | · |β∗ l dual norm

  • K. Bredies

24 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-67
SLIDE 67

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution TGV

Definition: Anisotropic Total Generalized Variation TGVk

β(u) = sup

  • Ω

u divk v dx

  • v ∈ Ck

c (Ω, Symk(I

Rd)), divl v∞,β∗

l ≤ 1, l=0,. . . ,k-1

  • β = (| · |β0, . . . , | · |βk−1) tensor norms

vl∞,β∗

l = supx∈Ω |vl(x)|β∗ l , | · |β∗ l dual norm

Infimal convolution TGV: [Holler/Kunisch ’14] ICTGVk

β = TGVk1 β1 . . . TGVkm βm

  • K. Bredies

24 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-68
SLIDE 68

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution TGV

Definition: Anisotropic Total Generalized Variation TGVk

β(u) = sup

  • Ω

u divk v dx

  • v ∈ Ck

c (Ω, Symk(I

Rd)), divl v∞,β∗

l ≤ 1, l=0,. . . ,k-1

  • β = (| · |β0, . . . , | · |βk−1) tensor norms

vl∞,β∗

l = supx∈Ω |vl(x)|β∗ l , | · |β∗ l dual norm

Infimal convolution TGV: [Holler/Kunisch ’14] ICTGVk

β = TGVk1 β1 . . . TGVkm βm

Regularization properties: Each TGVki

βi has finite-dimensional kernel + embedding

uBV ≤ C

  • u1 + TGVki

βi(u)

  • ICTGV is a regularizer
  • K. Bredies

24 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-69
SLIDE 69

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Accelerated dynamic MRI

Joint work with Martin Holler and Matthias Schl¨

  • gl

sum of squares Dynamic MRI: Acquire image sequences of moving objects Due to limited acquisition speed, only low-resolution images are obtainable

  • K. Bredies

25 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-70
SLIDE 70

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Accelerated dynamic MRI

Joint work with Martin Holler and Matthias Schl¨

  • gl

sum of squares Dynamic MRI: Acquire image sequences of moving objects Due to limited acquisition speed, only low-resolution images are obtainable

  • K. Bredies

25 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-71
SLIDE 71

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Accelerated dynamic MRI

Joint work with Martin Holler and Matthias Schl¨

  • gl

sum of squares Dynamic MRI: Acquire image sequences of moving objects Due to limited acquisition speed, only low-resolution images are obtainable Goal: Improve spatio-temporal resolution by undersampling reconstruction

  • K. Bredies

25 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-72
SLIDE 72

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Accelerated dynamic MRI

Joint work with Martin Holler and Matthias Schl¨

  • gl

sum of squares regularized Dynamic MRI: Acquire image sequences of moving objects Due to limited acquisition speed, only low-resolution images are obtainable Goal: Improve spatio-temporal resolution by undersampling reconstruction Apply ICTGV regularization

  • K. Bredies

25 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-73
SLIDE 73

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

The variational model

Minimization problem: min

u

  • t,c

λ 2Kt,c(ut) − dt,c2

2 + ICTGV2 β(u)

Kt,c(ut) = MtF(utσc) masked Fourier transform (σc complex coil sensitivities) ICTGV2

β = TGV2 β1 TGV2 β2

  • K. Bredies

26 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-74
SLIDE 74

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

The variational model

Minimization problem: min

u

  • t,c

λ 2Kt,c(ut) − dt,c2

2 + ICTGV2 β(u)

Kt,c(ut) = MtF(utσc) masked Fourier transform (σc complex coil sensitivities) ICTGV2

β = TGV2 β1 TGV2 β2

Primal-dual algorithm: Guaranteed convergence, duality-based stopping criterion GPU-optimized version: ≈ 160 seconds including coil-sensitivity estimation (NVidia GeForce GTX770 with AGILE library)

  • K. Bredies

26 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-75
SLIDE 75

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Numerical test

Acceleration factor 8: Reference data Unregularized reconstruction

  • K. Bredies

27 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-76
SLIDE 76

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Numerical test

Acceleration factor 8: Reference data Unregularized reconstruction

  • K. Bredies

27 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-77
SLIDE 77

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Numerical test

Acceleration factor 8: Low rank + sparse model Difference

  • K. Bredies

27 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-78
SLIDE 78

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Numerical test

Acceleration factor 8: ICTGV model Difference

  • K. Bredies

27 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-79
SLIDE 79

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Numerical test

Acceleration factor 16: ICTGV model Difference

  • K. Bredies

27 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-80
SLIDE 80

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Numerical test

Acceleration factor 8: Slow component Fast component

  • K. Bredies

27 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-81
SLIDE 81

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Numerical test

Acceleration factor 8: Slow component Fast component Favorable quantitative comparison 2nd place at the ISMRM 2013 reconstruction challenge

  • K. Bredies

27 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-82
SLIDE 82

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Outline

1 Infimal convolution regularization 2 Total generalized variation

Definition and properties Applications

3 Infimal convolution TGV

Accelerated dynamic MRI

4 Infimal convolution of oscillation TGV

Oscillation TGV and infimal convolution Numerical realization Applications

5 Summary

  • K. Bredies

28 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-83
SLIDE 83

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Motivation: Denoising “barbara”

noisy image

  • K. Bredies

29 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-84
SLIDE 84

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Motivation: Denoising “barbara”

TGV2

α regularization

  • K. Bredies

29 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-85
SLIDE 85

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Motivation: Denoising “barbara”

TGV2

α regularization

Capture oscillatory structures TGVosci

α,β,c

  • K. Bredies

29 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-86
SLIDE 86

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Oscillation TGV

Joint work with Yiming Gao Idea: Choose differential equation with

  • scillatory functions as solutions
  • K. Bredies

30 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-87
SLIDE 87

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Oscillation TGV

Joint work with Yiming Gao Idea: Choose differential equation with

  • scillatory functions as solutions

∇2u + cu = 0, c = ω ⊗ ω, ω ∈ I Rd, ω = 0 Solutions: u(x) = C1 cos(ω · x) + C2 sin(ω · x), C1, C2 ∈ I R

  • K. Bredies

30 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-88
SLIDE 88

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Oscillation TGV

Joint work with Yiming Gao Idea: Choose differential equation with

  • scillatory functions as solutions

∇2u + cu = 0, c = ω ⊗ ω, ω ∈ I Rd, ω = 0 Solutions: u(x) = C1 cos(ω · x) + C2 sin(ω · x), C1, C2 ∈ I R Adapt second-order TGV to the corresponding operator ∇2u + cu = 0

  • K. Bredies

30 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-89
SLIDE 89

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Oscillation TGV

Joint work with Yiming Gao Idea: Choose differential equation with

  • scillatory functions as solutions

∇2u + cu = 0, c = ω ⊗ ω, ω ∈ I Rd, ω = 0 Solutions: u(x) = C1 cos(ω · x) + C2 sin(ω · x), C1, C2 ∈ I R Adapt second-order TGV to the corresponding operator ∇u − w = 0, Ew + cu = 0

  • K. Bredies

30 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-90
SLIDE 90

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Oscillation TGV

Joint work with Yiming Gao Idea: Choose differential equation with

  • scillatory functions as solutions

∇2u + cu = 0, c = ω ⊗ ω, ω ∈ I Rd, ω = 0 Solutions: u(x) = C1 cos(ω · x) + C2 sin(ω · x), C1, C2 ∈ I R Adapt second-order TGV to the corresponding operator ∇u − w = 0, Ew + cu = 0 Total generalized variation (second order): TGV2

α,β (u) =

min

w∈BD(Ω) α

  • Ω

d|∇u − w|+β

  • Ω

d|Ew | α > 0, β > 0

  • K. Bredies

30 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-91
SLIDE 91

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Oscillation TGV

Joint work with Yiming Gao Idea: Choose differential equation with

  • scillatory functions as solutions

∇2u + cu = 0, c = ω ⊗ ω, ω ∈ I Rd, ω = 0 Solutions: u(x) = C1 cos(ω · x) + C2 sin(ω · x), C1, C2 ∈ I R Adapt second-order TGV to the corresponding operator ∇u − w = 0, Ew + cu = 0 Oscillation total generalized variation: TGVosci

α,β,c(u) =

min

w∈BD(Ω) α

  • Ω

d|∇u − w|+β

  • Ω

d|Ew + cu| α > 0, β > 0, c = ω ⊗ ω, ω ∈ I Rd, ω = 0

  • K. Bredies

30 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-92
SLIDE 92

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of TGVosci

α,β,c Basic properties: [Gao/B. ’17] TGVosci

α,β,c is proper, convex, lower semi-continuous

TGVosci

α,β,c is translation and rotation invariant

TGVosci

α,β,c + · 1 gives the Banach space BGVosci α,β,c(Ω)

ker(TGVosci

α,β,c) = span{x → cos(ω · x), x → sin(ω · x)}

TGVosci

α,β,c measures piecewise oscillations only at interfaces

  • K. Bredies

31 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-93
SLIDE 93

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of TGVosci

α,β,c Basic properties: [Gao/B. ’17] TGVosci

α,β,c is proper, convex, lower semi-continuous

TGVosci

α,β,c is translation and rotation invariant

TGVosci

α,β,c + · 1 gives the Banach space BGVosci α,β,c(Ω)

ker(TGVosci

α,β,c) = span{x → cos(ω · x), x → sin(ω · x)}

TGVosci

α,β,c measures piecewise oscillations only at interfaces

  • K. Bredies

31 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-94
SLIDE 94

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of TGVosci

α,β,c Basic properties: [Gao/B. ’17] TGVosci

α,β,c is proper, convex, lower semi-continuous

TGVosci

α,β,c is translation and rotation invariant

TGVosci

α,β,c + · 1 gives the Banach space BGVosci α,β,c(Ω)

ker(TGVosci

α,β,c) = span{x → cos(ω · x), x → sin(ω · x)}

TGVosci

α,β,c measures piecewise oscillations only at interfaces

  • K. Bredies

31 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-95
SLIDE 95

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of TGVosci

α,β,c Basic properties: [Gao/B. ’17] TGVosci

α,β,c is proper, convex, lower semi-continuous

TGVosci

α,β,c is translation and rotation invariant

TGVosci

α,β,c + · 1 gives the Banach space BGVosci α,β,c(Ω)

ker(TGVosci

α,β,c) = span{x → cos(ω · x), x → sin(ω · x)}

TGVosci

α,β,c measures piecewise oscillations only at interfaces

  • K. Bredies

31 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-96
SLIDE 96

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of TGVosci

α,β,c Basic properties: [Gao/B. ’17] TGVosci

α,β,c is proper, convex, lower semi-continuous

TGVosci

α,β,c is translation and rotation invariant

TGVosci

α,β,c + · 1 gives the Banach space BGVosci α,β,c(Ω)

ker(TGVosci

α,β,c) = span{x → cos(ω · x), x → sin(ω · x)}

TGVosci

α,β,c measures piecewise oscillations only at interfaces

  • K. Bredies

31 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-97
SLIDE 97

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of TGVosci

α,β,c Basic properties: [Gao/B. ’17] TGVosci

α,β,c is proper, convex, lower semi-continuous

TGVosci

α,β,c is translation and rotation invariant

TGVosci

α,β,c + · 1 gives the Banach space BGVosci α,β,c(Ω)

ker(TGVosci

α,β,c) = span{x → cos(ω · x), x → sin(ω · x)}

TGVosci

α,β,c measures piecewise oscillations only at interfaces

Advanced properties: BGVosci

α,β,c(Ω) = BV(Ω) in the sense of equivalent norms

TGVosci

α,β,c is coercive in the sense

u − PuBV ≤ C TGVosci

α,β,c(u)

for P : L1(Ω) → ker(TGVosci

α,β,c) continuous projection

  • K. Bredies

31 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-98
SLIDE 98

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of TGVosci

α,β,c Basic properties: [Gao/B. ’17] TGVosci

α,β,c is proper, convex, lower semi-continuous

TGVosci

α,β,c is translation and rotation invariant

TGVosci

α,β,c + · 1 gives the Banach space BGVosci α,β,c(Ω)

ker(TGVosci

α,β,c) = span{x → cos(ω · x), x → sin(ω · x)}

TGVosci

α,β,c measures piecewise oscillations only at interfaces

Advanced properties: BGVosci

α,β,c(Ω) = BV(Ω) in the sense of equivalent norms

TGVosci

α,β,c is coercive in the sense

u − PuBV ≤ C TGVosci

α,β,c(u)

for P : L1(Ω) → ker(TGVosci

α,β,c) continuous projection

  • K. Bredies

31 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-99
SLIDE 99

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution of TGVosci

Next steps: Separate cartoon components from oscillatory components Allow for multiple directions and frequencies m-fold infimal convolution

  • K. Bredies

32 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-100
SLIDE 100

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution of TGVosci

Next steps: Separate cartoon components from oscillatory components Allow for multiple directions and frequencies m-fold infimal convolution Infimal convolution of TGVosci: ICTGVosci

  • α,

β, c(u) = (TGVosci α1,β1,c1 . . . TGVosci αm,βm,cm)(u)

αi > 0, βi > 0, ci = ωi ⊗ ωi, ωi ∈ I Rd

  • K. Bredies

32 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-101
SLIDE 101

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution of TGVosci

Infimal convolution of TGVosci: ICTGVosci

  • α,

β, c(u) = (TGVosci α1,β1,c1 . . . TGVosci αm,βm,cm)(u)

αi > 0, βi > 0, ci = ωi ⊗ ωi, ωi ∈ I Rd Choice of parameters: αi, βi regularization parameters, similar to TGV2 ωi directions and frequencies, e.g., in 2D: ωi = f sin( iπ

k )

cos( iπ

k )

  • ,

k > 0 f > 0 frequency

  • K. Bredies

33 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-102
SLIDE 102

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution of TGVosci

Infimal convolution of TGVosci: ICTGVosci

  • α,

β, c(u) = (TGVosci α1,β1,c1 . . . TGVosci αm,βm,cm)(u)

αi > 0, βi > 0, ci = ωi ⊗ ωi, ωi ∈ I Rd Choice of parameters: αi, βi regularization parameters, similar to TGV2 ωi directions and frequencies, e.g., in 2D: ωi = f sin( iπ

k )

cos( iπ

k )

  • ,

k > 0 f > 0 frequency

  • K. Bredies

33 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-103
SLIDE 103

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution of TGVosci

Infimal convolution of TGVosci: ICTGVosci

  • α,

β, c(u) = (TGVosci α1,β1,c1 . . . TGVosci αm,βm,cm)(u)

αi > 0, βi > 0, ci = ωi ⊗ ωi, ωi ∈ I Rd Choice of parameters: αi, βi regularization parameters, similar to TGV2 ωi directions and frequencies, e.g., in 2D: ωi = f sin( iπ

k )

cos( iπ

k )

  • ,

k > 0 f > 0 frequency Remark: For ω = 0 ⇒ TGVosci = TGV2 cartoon component

  • K. Bredies

33 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-104
SLIDE 104

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution of TGVosci

Typical choice for cartoon/texture models: ω1 = 0 cartoon component ω2, . . . , ω9 8 texture directions, fixed frequency Optionally: ω10, . . . , ω17 8 texture directions, higher frequency

  • K. Bredies

34 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-105
SLIDE 105

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Infimal convolution of TGVosci

Typical choice for cartoon/texture models: ω1 = 0 cartoon component ω2, . . . , ω9 8 texture directions, fixed frequency Optionally: ω10, . . . , ω17 8 texture directions, higher frequency Variant of ICTGVosci: ICTGVosci

  • α,

β, c, γ(u) =

  • (γ1 · 1 + TGVosci

α1,β1,c1)

. . . (γm · 1 + TGVosci

αm,βm,cm)

  • (u)

γi ≥ 0 sparsifying parameter sparser textures

  • K. Bredies

34 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-106
SLIDE 106

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of ICTGVosci

Basic properties: ICTGVosci

  • α,

β, c, γ is proper and convex

ICTGVosci

  • α,

β, c, γ is translation and rotation invariant

ICTGVosci

  • α,

β, c, γ + · 1 gives a Banach space

ker(ICTGVosci

  • α,

β, c, γ) =

  • γi=0

ker(TGVosci

αi,βi,ci)

  • K. Bredies

35 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-107
SLIDE 107

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of ICTGVosci

Basic properties: ICTGVosci

  • α,

β, c, γ is proper and convex

ICTGVosci

  • α,

β, c, γ is translation and rotation invariant

ICTGVosci

  • α,

β, c, γ + · 1 gives a Banach space

ker(ICTGVosci

  • α,

β, c, γ) =

  • γi=0

ker(TGVosci

αi,βi,ci)

  • K. Bredies

35 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-108
SLIDE 108

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of ICTGVosci

Basic properties: ICTGVosci

  • α,

β, c, γ is proper and convex

ICTGVosci

  • α,

β, c, γ is translation and rotation invariant

ICTGVosci

  • α,

β, c, γ + · 1 gives a Banach space

ker(ICTGVosci

  • α,

β, c, γ) =

  • γi=0

ker(TGVosci

αi,βi,ci)

  • K. Bredies

35 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-109
SLIDE 109

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of ICTGVosci

Basic properties: ICTGVosci

  • α,

β, c, γ is proper and convex

ICTGVosci

  • α,

β, c, γ is translation and rotation invariant

ICTGVosci

  • α,

β, c, γ + · 1 gives a Banach space

ker(ICTGVosci

  • α,

β, c, γ) =

  • γi=0

ker(TGVosci

αi,βi,ci)

  • K. Bredies

35 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-110
SLIDE 110

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of ICTGVosci

Advanced properties: ICTGVosci

  • α,

β, c, γ is lower semi-continuous and exact, i.e.,

the infimum in the definition is attained ICTGVosci

  • α,

β, c, γ + · 1 is an equivalent norm on BV(Ω)

ICTGVosci

  • α,

β, c, γ is coercive in the sense

u − PuBV ≤ C ICTGVosci

  • α,

β, c, γ(u)

for P : L1(Ω) → ker(ICTGVosci

  • α,

β, c, γ) continuous projection

  • K. Bredies

36 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-111
SLIDE 111

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of ICTGVosci

Advanced properties: ICTGVosci

  • α,

β, c, γ is lower semi-continuous and exact, i.e.,

the infimum in the definition is attained ICTGVosci

  • α,

β, c, γ + · 1 is an equivalent norm on BV(Ω)

ICTGVosci

  • α,

β, c, γ is coercive in the sense

u − PuBV ≤ C ICTGVosci

  • α,

β, c, γ(u)

for P : L1(Ω) → ker(ICTGVosci

  • α,

β, c, γ) continuous projection

  • K. Bredies

36 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-112
SLIDE 112

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Properties of ICTGVosci

Advanced properties: ICTGVosci

  • α,

β, c, γ is lower semi-continuous and exact, i.e.,

the infimum in the definition is attained ICTGVosci

  • α,

β, c, γ + · 1 is an equivalent norm on BV(Ω)

ICTGVosci

  • α,

β, c, γ is coercive in the sense

u − PuBV ≤ C ICTGVosci

  • α,

β, c, γ(u)

for P : L1(Ω) → ker(ICTGVosci

  • α,

β, c, γ) continuous projection

  • K. Bredies

36 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-113
SLIDE 113

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Solution of imaging problems

Theorem: 1 < p ≤ d/(d − 1) K : Lp(Ω) → Y linear and continuous, Y Banach space f ∈ Y , 1 ≤ q < ∞          ⇒            Optimization problem min

u∈Lp(Ω)

1 qKu − f q

Y

+ ICTGVosci

  • α,

β, c, γ(u)

possesses a solution

  • K. Bredies

37 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-114
SLIDE 114

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Solution of imaging problems

Theorem: 1 < p ≤ d/(d − 1) K : Lp(Ω) → Y linear and continuous, Y Banach space f ∈ Y , 1 ≤ q < ∞          ⇒            Optimization problem min

u∈Lp(Ω)

1 qKu − f q

Y

+ ICTGVosci

  • α,

β, c, γ(u)

possesses a solution Remarks: Ensures applicability to a variety of imaging problems (denoising, deblurring, image reconstruction (CT, MRI), . . .) Intrinsic cartoon/texture decomposition for these problems

  • K. Bredies

37 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-115
SLIDE 115

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Solution of imaging problems

Theorem: 1 < p ≤ d/(d − 1) K : Lp(Ω) → Y linear and continuous, Y Banach space f ∈ Y , 1 ≤ q < ∞          ⇒            Optimization problem min

u∈Lp(Ω)

1 qKu − f q

Y

+ ICTGVosci

  • α,

β, c, γ(u)

possesses a solution Remarks: Ensures applicability to a variety of imaging problems (denoising, deblurring, image reconstruction (CT, MRI), . . .) Intrinsic cartoon/texture decomposition for these problems

  • K. Bredies

37 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-116
SLIDE 116

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Numerical realization

Discretization in 2D and for Y = L2(Ω′): Discretize TGVosci

α,β,c with forward/backward differences

Maintain kernel, i.e., ker(TGVosci

α,β,c) = span{x → cos(ω · x), x → sin(ω · x)}

c =

  • 2 − 2 cos(ω1)

1 + cos(ω1 − ω2) − cos(ω1) − cos(ω2) 1 + cos(ω1 − ω2) − cos(ω1) − cos(ω2) 2 − 2 cos(ω2)

  • for ω = (ω1, ω2) ∈ I

R2 \ πZ Z2 Solve non-smooth discrete optimization problems with first-order methods, such as a primal-dual iteration [Chambolle/Pock ’11]

  • K. Bredies

38 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-117
SLIDE 117

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Numerical realization

Discretization in 2D and for Y = L2(Ω′): Discretize TGVosci

α,β,c with forward/backward differences

Maintain kernel, i.e., ker(TGVosci

α,β,c) = span{x → cos(ω · x), x → sin(ω · x)}

c =

  • 2 − 2 cos(ω1)

1 + cos(ω1 − ω2) − cos(ω1) − cos(ω2) 1 + cos(ω1 − ω2) − cos(ω1) − cos(ω2) 2 − 2 cos(ω2)

  • for ω = (ω1, ω2) ∈ I

R2 \ πZ Z2 Solve non-smooth discrete optimization problems with first-order methods, such as a primal-dual iteration [Chambolle/Pock ’11]

  • K. Bredies

38 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-118
SLIDE 118

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Numerical realization

Discretization in 2D and for Y = L2(Ω′): Discretize TGVosci

α,β,c with forward/backward differences

Maintain kernel, i.e., ker(TGVosci

α,β,c) = span{x → cos(ω · x), x → sin(ω · x)}

c =

  • 2 − 2 cos(ω1)

1 + cos(ω1 − ω2) − cos(ω1) − cos(ω2) 1 + cos(ω1 − ω2) − cos(ω1) − cos(ω2) 2 − 2 cos(ω2)

  • for ω = (ω1, ω2) ∈ I

R2 \ πZ Z2 Solve non-smooth discrete optimization problems with first-order methods, such as a primal-dual iteration [Chambolle/Pock ’11]

  • K. Bredies

38 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-119
SLIDE 119

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Optimization algorithm

Iteration:                                        λn+1 =

  • λn + σ(K

m

  • i=1

¯ un

i − f )

  • /(1 + σ)

for i = 1, . . . , m do pn+1

i

= Pαi

  • pn

i + σ(∇¯

un

i − ¯

w n

i )

  • qn+1

i

= Pβi

  • qn

i + σ(E ¯

w n

i + ci ¯

un

i )

  • ˜

un+1

i

= un

i − τ(K ∗λn+1 − div1 pn+1 i

+ ciqn+1

i

) un+1

i

= Shrinkτγi(˜ un+1

i

) w n+1

i

= w n

i + τ(pn+1 i

+ div2 qn+1

i

) ¯ un+1

i

= 2un+1

i

− un

i

¯ w n+1

i

= 2w n+1

i

− w n

i

end for

  • K. Bredies

39 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-120
SLIDE 120

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Denoising “barbara”

noisy image

  • K. Bredies

40 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-121
SLIDE 121

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Denoising “barbara”

TGV2 regularization

  • K. Bredies

40 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-122
SLIDE 122

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Denoising “barbara”

ICTGVosci regularization

  • K. Bredies

40 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-123
SLIDE 123

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Denoising “barbara”

ICTGVosci regularization Better recovery of oscillatory structures

  • K. Bredies

40 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-124
SLIDE 124

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Cartoon/texture decomposition

Tested algorithms: Total variation/G-norm decomposition [Aujol et al. ’05] Total variation/H−1-norm decomposition [Osher/Sole/Vese ’03] Framelet/Local discrete cosine transform model [Cai/Osher/Shen ’09] Infimal convolution of

  • scillation TGV
  • K. Bredies

41 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-125
SLIDE 125

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Cartoon/texture decomposition

TV-G-norm decomposition

  • K. Bredies

42 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-126
SLIDE 126

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Cartoon/texture decomposition

TV-H−1-norm decomposition

  • K. Bredies

42 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-127
SLIDE 127

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Cartoon/texture decomposition

Framelet+LDCT model

  • K. Bredies

42 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-128
SLIDE 128

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Cartoon/texture decomposition

ICTGVosci model

  • K. Bredies

42 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-129
SLIDE 129

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Image denoising

Tested models/algorithms: Total generalized variation of second order [B./Kunisch/Pock ’10] Nonlocal total variation [Gilboa/Osher ’08] Infimal convolution of total generalized variation [Holler/Kunisch ’14] Framelet + local discrete cosine transform [Cai/Osher/Shen ’09] Infimal convolution of oscillation TGV Block matching and 3D filtering [Dabov et al. ’07]

  • K. Bredies

43 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-130
SLIDE 130

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Denoising “barbara”

noisy image

  • K. Bredies

44 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-131
SLIDE 131

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Denoising “barbara”

TGV2 regularization (27.41 dB)

  • K. Bredies

44 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-132
SLIDE 132

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Denoising “barbara”

Nonlocal TV regularization (32.05 dB)

  • K. Bredies

44 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-133
SLIDE 133

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Denoising “barbara”

ICTGV regularization (31.15 dB)

  • K. Bredies

44 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-134
SLIDE 134

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Denoising “barbara”

Framelet + LDCT model (31.70 dB)

  • K. Bredies

44 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-135
SLIDE 135

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Denoising “barbara”

ICTGVosci regularization (32.21 dB)

  • K. Bredies

44 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-136
SLIDE 136

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Denoising “barbara”

BM3D (34.43 dB)

  • K. Bredies

44 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-137
SLIDE 137

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Image inpainting

Setup: Recover an image from 50% randomly deleted pixels Tested algorithms: Total generalized variation [B./Kunisch/Pock ’10] Framelet/Local discrete cosine transform model [Cai/Osher/Shen ’09] Infimal convolution of

  • scillation TGV
  • K. Bredies

45 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-138
SLIDE 138

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Inpainting “barbara”

corrupted image

  • K. Bredies

46 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-139
SLIDE 139

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Inpainting “barbara”

TGV2 regularization (27.49 dB)

  • K. Bredies

46 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-140
SLIDE 140

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Inpainting “barbara”

Framelet + LDCT model (32.75 dB)

  • K. Bredies

46 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-141
SLIDE 141

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Inpainting “barbara”

ICTGVosci regularization (34.03 dB)

  • K. Bredies

46 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-142
SLIDE 142

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: Undersampled MRI

Setup: Recover an image from a few radially sampled Fourier coefficients Tested algorithms: Total generalized variation + shearlet model [Guo/Qin/Yin ’14] Infimal convolution of

  • scillation TGV
  • K. Bredies

47 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-143
SLIDE 143

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: MRI reconstruction

TGV + shearlet model (70 radial lines)

  • K. Bredies

48 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-144
SLIDE 144

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: MRI reconstruction

TGV + shearlet model (60 radial lines)

  • K. Bredies

48 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-145
SLIDE 145

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: MRI reconstruction

TGV + shearlet model (50 radial lines)

  • K. Bredies

48 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-146
SLIDE 146

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: MRI reconstruction

ICTGVosci model (70 radial lines)

  • K. Bredies

48 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-147
SLIDE 147

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: MRI reconstruction

ICTGVosci model (60 radial lines)

  • K. Bredies

48 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-148
SLIDE 148

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: MRI reconstruction

ICTGVosci model (50 radial lines)

  • K. Bredies

48 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-149
SLIDE 149

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: MRI reconstruction

ground truth

  • K. Bredies

48 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-150
SLIDE 150

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Example: MRI reconstruction

Quantitative comparison: Model 50 lines 60 lines 70 lines TGV + shearlet 28.77 dB 29.59 dB 30.10 dB ICTGVosci 29.16 dB 29.98 dB 30.56 dB

  • K. Bredies

49 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-151
SLIDE 151

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Outline

1 Infimal convolution regularization 2 Total generalized variation

Definition and properties Applications

3 Infimal convolution TGV

Accelerated dynamic MRI

4 Infimal convolution of oscillation TGV

Oscillation TGV and infimal convolution Numerical realization Applications

5 Summary

  • K. Bredies

50 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary

slide-152
SLIDE 152

INSTITUTE FOR MATHEMATICS AND SCIENTIFIC COMPUTING

Summary

Infimal convolution is a flexible tool to combine variational models, covering several known approaches It provides a regularizer for linear inverse problems in many cases It allows to construct stabilizing and regularizing cartoon/texture decomposition models Numerical experiments show a promising efficiency

Yiming Gao and Kristian Bredies. Infimal convolution of oscillation total generalized variation for the recovery of images with structured texture. SIAM Journal on Imaging Sciences, 11(3):2021–2063, 2018. Supported by the Austrian Science Fund (FWF) Project P29192 “Regularization graphs for variational imaging”

  • K. Bredies

51 / 51 Infimal convolution TGV ICTGV ICTGVosci Summary