Convex Optimization and Inpainting: A Tutorial Thomas Pock - - PowerPoint PPT Presentation

Convex Optimization and Inpainting: A Tutorial

Thomas Pock

Institute of Computer Graphics and Vision, Graz University of Technology

Dagstuhl seminar: Inpainting-Based Image Compression

1 / 56

Shannon-Nyquist sampling theorem

◮ In the field of digital signal processing, the sampling theorem

is a fundamental bridge between continuous-time signals and discrete-time signals

◮ It establishes a sufficient condition for a sample rate that

avoids aliasing fs ≥ 2fmax, where fs is the sampling frequency and fmax is the maximal frequency of the signal to be sampled. Example: Aliasing in 8× undersampled image

2 / 56

Compressed sensing

◮ Compressed sensing (CS) is a signal processing technique for

efficiently acquiring and reconstructing a signal

◮ It is based on finding solutions to underdetermined linear

systems

◮ The underlying principle is that the sparsity of a signal can be

exploited to recover it from far fewer samples than required by the Shannon-Nyquist sampling theorem.

(a) Original image (b) Sampling (c) No CS (d) Using CS

3 / 56

Solution of underdeterminded systems

◮ Let us consider the following underdetermined system of

equations of the form Ax = b

◮ b is a m × 1 measurement vector ◮ x is the n × 1 unknown signal ◮ A is the m × n basis matrix (dictionary), with m < n, which is

• f the form

A =

• a1, . . . , an
• ,

where each ai defines a basis atom.

◮ How can we solve the underdetermined system of equations?

4 / 56

Regularization

◮ Let us consider the regularized problem

min

x f (x)

subject to Ax = b

◮ A first simple choice is the squared ℓ2 distance f (x) = x2 2 ◮ The unique solution ˆ

x of the problem is then given by ˆ x2 = AT(AAT)−1b, which is exactly the pseudo-inverse of A.

◮ The quadratic regularization tries to find a solution ˆ

x that has the smallest ℓ2 norm.

5 / 56

Sparsity

◮ Another form of regularization that received a lot of attention

during the last years is based on sparsity

◮ The idea is that the underlying ”dimension” of a signals’

complexity is small if represented in a suitable basis

◮ A simple and intuitive form of sparsity is given by the ℓ0

(pseudo) norm of a vector x x0 = #{i : xi = 0}, and hence x0 < n if x is sparse.

◮ Hence we consider the following problem

min

x x0

subject to Ax = b, which is known as ”Basis Pursuit” [Chen, Donoho ’94]

6 / 56

Convex relaxation

◮ The previous problem is NP-hard and hence very hard to solve

if the degree of sparsity is not very small

◮ A simple idea is to replace the ℓ0 pseudo norm by its closest

convex approximation, which is the ℓ1 norm: min

x x1

subject to Ax = b,

◮ This problem can actually be solved using convex optimization

algorithms

◮ Under certain circumstances, the solution of the convex ℓ1

problem yields the same sparse solution as the solution of the ℓ0 problem

7 / 56

Noise

◮ In case there is noise in the measurement, we can replace the

equality in the constraint by an inequality constraint, leading to min

x x1

subject to Ax − b2 ≤ σ2, where σ > 0 is an estimate of the noise level.

◮ This problem can equivalently be written as the unconstrained

• ptimization problem

min

x x1 + λ

2 Ax − b2, where λ > 0 is a suitable Lagrange multiplier.

◮ This model is known as the ”Lasso” (Least absolute shrinkage

and selection operator) [Tibshirani ’96]

◮ The model performs a least squares fit while ensuring that

• nly a few basis atoms are used.

8 / 56

The Lasso model

◮ In statistics, the Lasso model is used to perform linear

regression and regularization, order to improve the prediction accuracy of a statistical model

◮ Sparsity in the Lasso model has a nice geometric

interpretation why the ℓ1 norm leads to sparse solutions

x1

• 1
• 0.5

0.5 1 1.5 2 2.5 3

x2

• 1
• 0.5

0.5 1 1.5 2 2.5 3

x1

• 1
• 0.5

0.5 1 1.5 2 2.5 3

x2

• 1
• 0.5

0.5 1 1.5 2 2.5 3

f (x) = ·1 f (x) = ·2

2

9 / 56

Example

◮ The Lasso model can also be interpreted as a model that tries

to “synthesize” a given signal b using basis atoms from A. basis atoms of A given signal b synthesized signal Ax

10 / 56

Other sparsity inducing functions

Besides the ℓ1 norm, there are other interesting sparsity inducing

• functions. Assume x ∈ Rm×n

◮ Mixed ℓ1,2 norm: x2,1 = m i=1 |

n

j=1 |xi,j|2| can be used

to induce sparsity in groups of variables

◮ The nuclear norm x∗ = min{m,n} i=1

σi(x) can be used to induce sparsity in the singular values of x which in turn imposes a low rank prior on x

11 / 56

Synthesis vs. analysis

◮ A closely related (yet different) problem is obtained by moving

the linear operator A to the sparsity inducing function. min

x By1 + λ

2 y − b2

2 ◮ Here, the linear operator B can be interpreted as an operator

“analyzing” the signal

◮ The model performs a least squares fit while ensuring that the

inner product with a given set of basis atomes in B vanishes most of the time

◮ The most influential models in imaging based on such sparse

analysis operators are those based on total variation regularization

12 / 56

Convex optimization 1

In imaging, mainly two classes of optimization problems are dominating

◮ “Smooth plus nonsmooth”

min

x f (x) + g(x),

where f (x) is a smooth function with Lipschitz continuous gradient and g is a simple convex function, with efficient to compute proximal map

◮ Can be solved with proximal gradient methods [Goldstein ’64],

[Nesterov ’83], [Combettes, Wajs ’05)], [Beck, Teboulle ’09]

• yk = ...

xk+1 = proxτg(yk − τ∇f (yk))

13 / 56

Convex optimization 2

◮ “Non-smooth with linear operator”

min

x f (Kx) + g(x),

where f , g are prox-simple convex functions and K is a linear

• perator

◮ Perform splitting

min

x,z f (z) + g(x),

s.t. Kx = z

◮ Consider the augmented Lagrangian

min

x,z max y

Kx − z, y + f (z) + g(x) + 1 2δKx − z2

◮ Alternating direction of multipliers (ADMM) [Glowinski,

Marroco (1975)], [Boyd, Eckstein et al. ’11]

◮ Equivalent to Douglas Rachford splitting [Douglas, Rachford

’56], [Lions, Mercier ’79]

◮ Many variants exist ...

14 / 56

The ROF model

◮ Introduced in [Rudin, Osher, Fatemi ’92] and extended in

[Chambolle, Lions ’97] min

u λ

• Ω

|Du| + 1 2

• Ω

|u(x) − u⋄(x)|2 dx, where Ω is the image domain, u⋄ is a given (noisy) image and λ > 0 is a regularization parameter.

◮ The term

• Ω |Du| is the total variation (TV) of the image u

and the gradient operator D is understood in its distributional sense.

◮ A standard way to define the total variation is by duality:

• Ω

|Du| := sup −

• Ω

u(x) div ϕ(x) dx : ϕ ∈ C ∞

c (Ω; Rd), |ϕ(x)|∗ ≤ 1, ∀x ∈ Ω,

where Ω ⊂ Rd is a d-dimensional open set.

15 / 56

Functions with bounded variation

◮ The space

BV(Ω) =

• u ∈ L1(Ω) :
• Ω

|Du| < +∞

• ,
• f functions with bounded variations equipped with the norm

uBV = uL1 +

• Ω |Du|, is a Banach space.

◮ The function | · | could be any norm and the dual norm is

given by |φ|∗ := sup

|x|≤1

φ, x

◮ For smooth images, the TV measures the L1 norm of the

image gradient

◮ The TV is also well-defined for functions with sharp

discontinuities

◮ For characteristic functions of smooth sets, it measures

exactly the length or area of the surface of the set inside Ω.

16 / 56

Finite differences discretization

◮ In the discrete setting, we consider a scalar-valued digital

image u ∈ Rm×n of m × n pixels

◮ A simple and standard approach to define the discrete total

variation is to define a finite differences operator D : Rm×n → Rm×n×2 (Du)i,j,1 =

• ui+1,j − ui,j

if 1 ≤ i < m, else, (Du)i,j,2 =

• ui,j+1 − ui,j

if 1 ≤ j < n, else.

◮ We will also need the operator norm D which is estimated

as D ≤ √ 8

17 / 56

The discrete total variation

◮ The discrete total variation is defined as

Dup,1 =

m,n

• i=1,j=1

|(Du)i,j|p =

m,n

• i=1,j=1
• (Du)p

i,j,1 +(Du)p i,j,2

1/p, that is, the ℓ1-norm of the p-norm of the pixelwise image gradients.

◮ For p = 1 we obtain the anisotropic total variation and if

p = 2 we obtain the isotropic total variation

18 / 56

Some properties of the total variation

◮ From a sparsity point of view, the total variation induces

sparsity in the gradients of the image, hence, it favors piecewise constant images

◮ This property is known as staircasing effect, which is often

considered as a drawback for certain applications

◮ The case p = 1 allows for quite effective splitting techniques

but favors edges to be aligned with the grid

◮ The case p = 2 can also be considered as a simple form of

group sparsity, grouping together the spatial derivatives in each dimension

◮ The isotropic variant does not exhibit a grid bias and hence is

• ften preferred in practice

19 / 56

ℓ2 data fitting

◮ The classical ROF model is based on a quadratic data fitting

term min

u λDup,1 + 1

2u − u⋄2

2 ◮ Using duality, the model can also be written as

min

p δ{·2,∞≤λ}(p) + 1

2D∗p − u⋄2

2,

where p is the dual variable.

◮ The dual ROF model is easier to minimize. ◮ Note that the dual ROF model is an anti-sparse Lasso model!

20 / 56

Example

(a) Original image (b) Noisy image u⋄ (c) Denoised image u

21 / 56

ℓ1 data fitting

◮ The quadratic data fitting term of the ROF model i

specialized for zero-mean Gaussian noise.

◮ In case of impulsive noise containing gross outliers, a simple,

yet effective modification is to replace the quadratic fitting term by an ℓ1-data term

◮ The resulting model, called TV-ℓ1 model, is given by

min

u λDup,1 + u − u⋄1 ◮ The model additionally tries to induce sparsity in the data

fitting

◮ The model is contrast invariant ◮ The regularization parameter is linked to the area of

structures that are “removed” from the image

22 / 56

Example

(a) Original image (b) Noisy image (c) TV-ℓ1 (d) ROF

23 / 56

Scale selection

TV-ℓ1

(a) λ = 0 (b) λ = 4 (c) λ = 5 (d) λ = 8 (e) λ = 0 (f) λ = 1 (g) λ = 2 (h) λ = 4

ROF

24 / 56

Poisson noise

◮ In case of Poisson noise, a popular data-fitting term is given

by the generalized Kullback-Leibler divergence, see e.g. [Steidl, Teuber ’10] min

u λDup,1 +

• i,j

ui,j − u⋄

i,j log ui,j,

ui,j > 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

25 / 56

Example: Poisson noise

(i) original image (j) noisy image (k) ROF, λ = 1/10 (l) TV-entropy, λ = 1/7

26 / 56

Linear inverse problems

◮ In many imaging applications the “forward” model is of the

form u⋄ = Au + n, where A : Rm×n → Rk×l is a linear operator and n is some additive noise.

◮ The type of the linear operator depends on the application:

◮ Deblurring: A is a convolution operator ◮ CT reconstruction: A is a Radon transform ◮ MRI reconstruction: A is a Fourier transform

◮ A total variation model incorporating an additional linear

• perator is usually of the form:

min

u λDup,1 + 1

2Au − u⋄2

2,

and u⋄ ∈ Rk×l is the given data.

27 / 56

Example

(a) Zero filling (b) TV regularized result

28 / 56

Higher order smoothness

◮ There have been several attempts to generalize the total

variation to higher order smoothness

◮ One example is via infimal convolution [Chambolle, Lions ’97]

IC(u) = min

u=u1+u2

• Ω

|Du1| +

• Ω

|D2u2|

◮ Another variant, called total generalized variation (TGV) has

been proposed in [Bredies, Kunisch, P. ’10] TGV k

λ (u) = sup

• Ω

u(x) divk ϕ(x) dx : ϕ ∈ C k

c (Ω; Symk(Rd), divl ϕ(x)∞ ≤ λl, l = 0...k − 1, ◮ For k = 2 and in 1D, both variants are the same

29 / 56

Second order TGV

◮ In case, k = 2 one can give a more intuitive formulation of

TGV TGV 2

λ(u) = inf v λ1

• Ω

|Du − v| + λ0

• Ω

|Dv|, where u ∈ BV (Ω) and v ∈ BV (Ω; R2)

◮ This gets rid of the staircasing effect on affine parts of the

image

◮ We introduce the discrete scalar images u, u⋄ ∈ Rm×n and

vectorial image v = (v1, v2) ∈ Rm×n×2.

◮ The discrete (non-symmetric) counterpart of a TGV variant of

the ROF model is given by min

u,v λ1Du − v2,1 + λ0Dv2,1 + 1

2u − u⋄2, where D : Rm×n×2 → Rm×n×4 is again a finite differences

• perator which can be decomposed as Dv = (Dv1, Dv2),

where D

30 / 56

Example

(a) original image g (b) noisy image f (c) ROF, λ = 0.1 (d) TGV, λ0,1 = (1/4, 1/9)

31 / 56

Vectorial total variation

◮ So far we have considered only scalar valued images ◮ Assume, we have given vector valued images u = (u1, ..., uk),

e.g. RGB images.

◮ The question is, what is a good generalization of the total

variation to such vector valued images?

◮ In principle, we could use any matrix norm acting on the

Jacobian Du

◮ An interesting class of matrix norms is given by p-Schatten

norms |J(x)|Sp = min{d,k}

• n=1

σp

n(J(x))

1

p

, ∀p ∈ [1, ∞), |J(x)|S∞ = max

n∈{1,...,min{d,k}} σn(J(x)),

where the σn(J(x)) denote the singular values of the Jacobian J(x)

32 / 56

Vectorial ROF model

◮ In case p = 2, the Schatten norm corresponds to the

Frobenius norm

◮ In case p = 1, the Schatten norm is equal to the nuclear norm

and hence forces the Jacobian to be of low rank [Duran, M¨

• ller, Sbert, Cremers ’16]

◮ In case p = ∞, we obtain the operator norm, which penalizes

the larges singular value

◮ A discrete version of the vectorial ROF model (p = 1) is

min

u λDuS1,1 + 1

2u − u⋄2

33 / 56

Example

(a) original image (b) noisy image (c) Frobenius, λ = 0.1 (d) nuclear, λ = 0.1

34 / 56

Other analysis operators

◮ In the discrete setting, one can replace the finite differences

• perator D by a more general analysis operator

Φ : Rm×n → Ck1×···×kK

◮ This includes for example complex wavelets [Kingsbury ’01],

curvelets [Cand` es, Donoho ’05], or shearlets [Guo, Kutyniok, Labate ’06] which provide optimal sparse approximations of cartoon-like images

◮ A straight forward extension of the ROF model is given by

min

u λΦu1 + 1

2u − u⋄2

2. ◮ Can be combined with any reasonable data term, e.g.

inpainting

35 / 56

Inpainting?

Original image

36 / 56

Inpainting?

90% missing pixels

36 / 56

Inpainting?

TV reconstruction

36 / 56

Inpainting?

TGV reconstruction

36 / 56

Inpainting?

Shearlet reconstruction

36 / 56

The Mumford-Shah problem

◮ Piecewise smooth approximation of functions [Mumford, Shah

’89]

◮ Variational free boundary problem

min

u MS(u) =

• Ω\Su

|∇u|2 dx +νHd−1(Su)+λ

• Ω

(u −u⋄)2 dx

◮ Computing a minimizer of the Mumford-Shah functional poses

a difficult minimization problem due to its non-convexity

◮ Simulated annealing [Geman, Geman ’84] ◮ Graduated non-convexity (GNC) procedure [Blake, Zisserman

’87]

◮ Phase field approximation of [Ambrosio, Tortorelli, ’90] ◮ Curve evolution via level set methods [Vese, Chan ’02] 37 / 56

The approach of Alberti, Bouchitte and Dal Maso

◮ The Euler-Lagrange equations of the Mumford-Shah

functional provide only a necessary condition for minimality

◮ In [Alberti, Bouchitte, Dal Maso, ’03], the authors provide a

sufficient condition for (some) minimizers of the Mumford-Shah functional.

◮ The basic idea is to consider the graph Γu of u instead of the

function u

◮ Rewrite the Mumford-Shah functional by means of the flux of

a suitable vector field ϕ through the graph Γu

x t 1u(x, t) Ω Su νΓu u(x) Γu u− u+

1u(x, t) =

• 1

if t < u(x), else,

38 / 56

A lower bound

◮ Suppose, the maximum flux of a vector field ϕ = (ϕx, ϕt)

through the graph provides a lower bound to the Mumford-Shah energy MS(u) ≥ sup

ϕ∈K

• Γu

ϕ · νΓu dH2.

x t 1u(x, t) Ω Su νΓu u(x) Γu u− u+ ϕ

◮ It turns out that the above relation is true for

K =

• ϕ
• ϕt(x, t) ≥ ϕx(x, t)2

4 −µ(t−f (x))2, t2

t1

ϕx(x, s) ds ≤ ν

• 39 / 56

A sufficient condition

◮ The integral can be extended to Ω × R

MS(u) = sup

ϕ∈K

• Ω×R

ϕ · D1u,

◮ The key observation is now: If for a given u the supremum is

attained by a divergence-free vector field ϕu ∈ K, one has MS(v) = sup

ϕ∈K

• Ω×R

ϕ·D1v ≥

• Ω×R

ϕu·D1v =

• Ω×R

ϕu·D1u = MS(u), for any v which agrees with u on the boundary of Ω

◮ Hence u is a minimizer of the Mumford-Shah functional ◮ If the vector field is divergence-free, it is called a “calibration” ◮ It remains unclear if a calibration exists for each minimizer ...

40 / 56

Convex relaxation

◮ Relaxation of the binary function 1u : Ω → {0, 1} to functions

v : Ω → [0, 1]

◮ Results in the convex-concave saddle-point problem

min

v∈BV(Ω×R,[0,1])

• sup

ϕ∈K

• Ω×R

ϕ · Dv

• ◮ The Euler-Lagrange equations imply that the optimal ϕ is

divergence free

◮ If the minimal v is binary, the calibration argument can be

applied

◮ Can be solved via the PDHG algorithm [P., Cremers, Bischof,

Chambolle, ’09]

◮ Difficulty: Projection onto the convex set K

41 / 56

Example

(a) original image u⋄ (b) piecewise smooth image u

42 / 56

Visualization of the subgraph

(a) relaxed function v

43 / 56

The crack tip problem

Optimality shown in [Bonnet, David ’01] Phase-field approximation Convex relaxation

44 / 56

Subjective boundaries

45 / 56

Subjective boundaries

45 / 56

Subjective boundaries

45 / 56

The elastica functional

◮ Consider the so-called elastica functional

• Γ

(α + β|κ|2) dγ , α, β > 0 ,

◮ Long history, dating back at least to Bernoulli and Euler ◮ Psychophysical experiments suggest that subjective

boundaries “seen” by humans are linear or curvilinear

◮ Subjective boundaries can be well explained by minimizers of

the elastica functional [Kanizsa ’79]

◮ Defines a strong prior for image reconstruction problems

[Nitzberg, Mumford, Shiota ’93]

◮ Related functionals are also used for shape reconstruction

problems [Willmore ’93], [Franken, Rumpf, Wirth ’10], [Bretin, Masnou, Oudet ’13], ...

46 / 56

Visual Cortex

◮ Experiments suggest that the visual cortex is made of

• rientation sensitive layers, cells are connected to get a sense
• f curvature [Hubel, Wiesel ’59]

47 / 56

Roto-translation group

◮ Idea: Use the sub-Riemannian structure of the roto-translation

group to describe the geometry of the visual cortex

◮ First works by [Hoffman ’89], [Petitot ’98] to use the

sub-Riemannian structure as a mathematical model for the visual cortex

◮ Sub-Riemannian diffusion and mean curvature motion [Citti,

Sarti 03/06], [Duits, Franken 10], [Boscain et al. ’14], Citti et

• al. ’15] for inpainting.

◮ Vertext penalizing energies based on directional derivatives in

the RT space [Bredies, P. Wirth ’13]

◮ Relaxation of the elastica energy in a 4D space [Bredies, P.

Wirth ’15]

◮ Relations to LP formulation over pairs of line segments by

[Schoenemann, Masnou, Cremers ’11]

◮ New (simplified) ongoing work jointly with Antonin Chambolle

48 / 56

A parametric curve

◮ Assume we want to represent the boundary of a smooth set

E ∈ Ω by a parametrized curve (x1(t), x2(t)) with parameter t ∈ [0, 1]

◮ The curvature κE of ∂E is defined as the ratio between the

variation of the tangential angle θ and the variation of its arc length s, that is κE = dθ ds =

dθ dt ds dt dθ dt

• dx1

dt

2 +

• dx2

dt

2 .

◮ We now consider a parametrized 3D curve (x1(t), x2(t), θ(t))

which lifts the boundary ∂E to the roto-translational space.

49 / 56

Lifted curve

◮ We define the tangential vector p(t) = (px(t), pθ(t)) with

px(t) = dx1(t) dt , dx2(t) dt

• ,

pθ(t) = dθ(t) dt , and |px(t)| = dx1(t) dt 2 + dx2(t) dt 2 .

◮ The curvature is therefore given by

κE(t) = pθ(t) |px(t)|,

50 / 56

Curvature penalizing energies

◮ We consider f : IR → [0, +∞] a convex, lower-semicontinuous

function and want to define a lower-semicontinuous extension to energies of the type E →

• ∂E

f (κE) dH1 where E ⊂ Ω is a set with C 2 boundary, and κE is the curvature of the set.

◮ Using our tangential vector p(t), it is easy to see that the

energy can be (formally) written as

• ∂E

f (κE) dH1 = 1 f (pθ/|px|)|px| dt =

• Ω×S1

f (τ θ/|τ x|)|τ x| dH1 ΓE where τ(x, θ) is the normalized tangential vector.

51 / 56

Different energies

◮ f1(t) = 1 + k|t|. This energy penalizes the arclength plus the

absolute curvature:

• Ω×S1 f1(τ θ/|τ x|)|τ x| dH1

ΓE =

• Ω×S1 |τ x|+k|τ θ| dH1

ΓE.

52 / 56

Different energies

◮ f1(t) = 1 + k|t|. This energy penalizes the arclength plus the

absolute curvature:

• Ω×S1 f1(τ θ/|τ x|)|τ x| dH1

ΓE =

• Ω×S1 |τ x|+k|τ θ| dH1

ΓE.

◮ f2(t) =

• 1 + k|t|2. This energy penalizes the arclength of

the lifted curve in the RT space.

• Ω×S1 f2(τ θ/|τ x|)|τ x| dH1

ΓE =

• Ω×S1
• |τ x|2 + k|τ θ|2 dH1

ΓE

52 / 56

Different energies

◮ f1(t) = 1 + k|t|. This energy penalizes the arclength plus the

absolute curvature:

• Ω×S1 f1(τ θ/|τ x|)|τ x| dH1

ΓE =

• Ω×S1 |τ x|+k|τ θ| dH1

ΓE.

◮ f2(t) =

• 1 + k|t|2. This energy penalizes the arclength of

the lifted curve in the RT space.

• Ω×S1 f2(τ θ/|τ x|)|τ x| dH1

ΓE =

• Ω×S1
• |τ x|2 + k|τ θ|2 dH1

ΓE

◮ f3(t) = 1 + k|t|2. This is the classical Eulers’s elastica energy.

• Ω×S1 f3(τ θ/|τ x|)|τ x| dH1

ΓE =

• Ω×S1 |τ x|+k |τ θ|2

|τ x| dH1 ΓE. Convex energies in the measure σ = τ dH1 ΓE

52 / 56

The measure σ

The measure σ is not arbitrary: It satisfies two important constraints: i By construction, it is a circulation and has zero divergence in Ω × S1 since we want to represent closed curves (or ending at the boundary) ii Its marginals in Ω × S1, denoted by

• S1 σx (and which are also

divergence free), coincide with a 90◦-rotation of the measure DχE

53 / 56

Convex representation

◮ We define the following convex function

h(θ, p) =      |px|f (pθ/|px|) if px ∈ IR+θ, px = 0, f ∞(pθ) if px = 0, +∞ else.

◮ Based on that, we consider the following (convex) energy

F(u) = inf

• Ω×S1

h(θ, σ) : div σ = 0,

• Ω×S1

ϕ⊥ · σx =

• Ω

ϕ · Du

• .

◮ Can be combined with different data terms ...

54 / 56

Inpainting?

Original image

55 / 56

Inpainting?

90% missing pixels

55 / 56

Inpainting?

TV reconstruction

55 / 56

Inpainting?

Curvature reconstruction using 1 + k|κ|2.

55 / 56

Shape completion

(a) Original shape (b) Input (c) Inpainted shape

Weickert’s cat: Shape completion using the function

• 1 + k|κ|2.

56 / 56