23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 M - PDF document

M I Variational Methods and Optimization in Imaging Institut Henri Poincar´ e, Paris, February 4–8, 2019 A 1 2 3 4 5 6 Stable Models and Algorithms for 7 8 Backward Diffusion Evolutions 9 10 11 12 13 14 Joachim Weickert 15 16 Mathematical Image Analysis Group 17 18 Saarland University, Saarbr¨ ucken, Germany 19 20 21 22 23 24 joint work with 25 26 Martin Welk (UMIT, Hall, Austria) 27 28 Leif Bergerhoff (Saarland University) 29 30 Marcelo C´ ardenas (Saarland University) 31 32 Guy Gilboa (Technion, Haifa, Israel) 33 34 35 36 37 38 partial funding: DFG Leibniz Award and ERC Advanced Grant INCOVID 39 40 41 M I Introduction (1) A 1 2 Introduction 3 4 5 6 � Forward diffusion equations blur or smooth images. 7 8 = ⇒ attempts to invert these evolutions for deblurring or sharpening images 9 10 11 12 13 14 15 16 17 18 19 20 21 22 ↔ 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41

M I Introduction (2) A 1 2 Problems 3 4 5 6 � Backward diffusion is typically regarded as ill-posed: 7 8 • Solution does not exist for non-smooth initial data. 9 10 • If it exists, it is highly sensitive w.r.t. perturbations. 11 12 13 14 � Thus, many researchers refrain from using backward diffusion. 15 16 17 18 19 20 Goals 21 22 23 24 � show how these problems can be 25 26 • handled by sophisticated numerics 27 28 29 30 • or circumvented by smart modelling 31 32 � demonstate these principles with two prototypical applications: 33 34 35 36 • advanced numerics for the FAB diffusion of Gilboa et al. 2002 37 38 • novel convex model for backward diffusion (Bergerhoff et al. 2018) 39 40 41 M I Outline A 1 2 Outline 3 4 5 6 � FAB Diffusion 7 8 • Continuous Model 9 10 11 12 • Explicit Scheme 13 14 • Efficient Numerics 15 16 • Experiments 17 18 19 20 � Backward Diffusion with Convex Energy 21 22 • Model and Theory 23 24 25 26 • Numerical Algorithm 27 28 • Experiment 29 30 31 32 � Conclusions 33 34 35 36 37 38 39 40 41

M I Outline A 1 2 Outline 3 4 5 6 � FAB Diffusion 7 8 9 10 • Continuous Model 11 12 • Explicit Scheme 13 14 • Efficient Numerics 15 16 • Experiments 17 18 19 20 � Backward Diffusion with Convex Energy 21 22 • Model and Theory 23 24 25 26 • Numerical Algorithm 27 28 • Experiment 29 30 31 32 � Conclusions 33 34 35 36 37 38 39 40 41 M I FAB Diffusion: Continuous Model (1) A 1 2 FAB Diffusion: Continuous Model 3 4 5 6 The Perona–Malik Filter (1990) 7 8 Consider open image domain Ω ⊂ R 2 and some bounded image f : Ω → R . 9 10 � 11 12 � Create family of filtered versions u ( x , t ) of f ( x ) as solution of 13 14 � � 15 16 g ( | ∇ u | 2 ) ∇ u ∂ t u = div on Ω × (0 , ∞ ) , 17 18 u ( x , 0) = f ( x ) on Ω , 19 20 n ⊤ ∇ u = 0 on ∂ Ω × (0 , ∞ ) , 21 22 23 24 where n denotes the outer normal vector to the image boundary ∂ Ω . 25 26 diffusivity g is monotonically decreasing positive function of | ∇ u | 2 � 27 28 � smoothes within flat regions and enhances edges between them 29 30 31 32 � gradient descent of a possibly nonconvex but monotone energy 33 34 � 35 36 Ψ( | ∇ u | 2 ) d x E ( u ) = 37 38 Ω 39 40 where the penaliser (potential) Ψ( | ∇ u | 2 ) satisfies Ψ ′ ( | ∇ u | 2 ) = g ( | ∇ u | 2 ) . 41

M I FAB Diffusion: Continuous Model (2) A 1 2 Forward-and-Backward (FAB) Diffusion 3 4 5 6 (Gilboa / Sochen / Zeevi 2002) 7 8 9 10 � goal: stronger sharpening than classical Perona-Malik filters 11 12 13 14 � equip Perona-Malik diffusion 15 16 � � g ( | ∇ u | 2 ) ∇ u 17 18 ∂ t u = div 19 20 21 22 with a diffusivity that takes positive and negative values. 23 24 � fairly mild assumptions in this talk: 25 26 27 28 g ∈ C 1 [0 , ∞ ) , g (0) = c 1 > 0 , g ( . ) ≥ − c 2 with c 1 > c 2 ≥ 0 . 29 30 31 32 33 34 corresponds to nonconvex and nonmonotone potential Ψ( | ∇ u | 2 ) � 35 36 37 38 39 40 41 M I FAB Diffusion: Continuous Model (3) A 1 2 How Unpleasant can this Become ? 3 4 5 6 Diffusivity g ( s 2 ) Diffusivity g ( s 2 ) Diffusivity g ( s 2 ) 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 Potential Ψ( s 2 ) Potential Ψ( s 2 ) Potential Ψ( s 2 ) 35 36 37 38 39 40 41

M I FAB Diffusion: Continuous Model (4) A 1 2 Theoretical Results so Far 3 4 5 6 � cannot be covered by standard theory for diffusion filters (W. 1998) 7 8 � no continuous well-posedness theory 9 10 11 12 � Gilboa / Sochen / Zeevi (IEEE TIP 2002): 13 14 standard implementations violate extremum principle 15 16 � Gilboa / Sochen / Zeevi (JMIV 2004): 17 18 experimental stabilisation with a fidelity term and biharmonic regularisation 19 20 21 22 Can we establish a fully discrete theory ? 23 24 Does this lead to practical algorithms for images? 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 M I Outline A 1 2 Outline 3 4 5 6 � FAB Diffusion 7 8 • Continuous Model 9 10 11 12 • Explicit Scheme 13 14 • Efficient Numerics 15 16 • Experiments 17 18 19 20 � Backward Diffusion with Convex Energy 21 22 • Model and Theory 23 24 25 26 • Numerical Algorithm 27 28 • Experiment 29 30 31 32 � Conclusions 33 34 35 36 37 38 39 40 41

M I FAB Diffusion: Explicit Scheme (1) A 1 2 FAB Diffusion: Explicit Scheme 3 4 5 6 Goal 7 8 � establish comprehensive theory for an explicit discretisation of FAB diffusion 9 10 11 12 Explicit Scheme 13 14 15 16 Explicit finite difference discretisation of diffusion equation 17 18 19 20 � � � � g ( | ∇ u | 2 ) ∂ x u g ( | ∇ u | 2 ) ∂ y u ∂ t u = ∂ x + ∂ y 21 22 23 24 in some inner pixel ( i, j ) at time level k yields the scheme 25 26 27 28 � � u k +1 − u k g k i +1 ,j + g k u k i +1 ,j − u k − g k i,j + g k u k i,j − u k 1 i,j i,j i,j i,j i − 1 ,j i − 1 ,j 29 30 = τ h 1 2 h 1 2 h 1 31 32 � � g k i,j +1 + g k u k i,j +1 − u k − g k i,j + g k u k i,j − u k 33 34 1 i,j i,j i,j − 1 i,j − 1 + 35 36 h 2 2 h 2 2 h 2 37 38 with grid sizes h 1 , h 2 and time step size τ . 39 40 41 M I FAB Diffusion: Explicit Scheme (2) A 1 2 Where Do Problems Arise ? 3 4 The standard discretisation of g ( | ∇ u | 2 ) is given by � 5 6 7 8 � � � 2 � � 2 � u k i +1 ,j − u k u k i,j +1 − u k 9 10 i − 1 ,j i,j − 1 g k i,j := g + . 2 h 1 2 h 2 11 12 � �� � 13 14 can be positive at extrema 15 16 17 18 � It can create negative diffusivities in extrema, which give rise to instabilities. 19 20 21 22 Is There a Remedy ? 23 24 � 25 26 A nonstandard discretisation produces a vanishing gradient in extrema: 27 28 � � � u k i +1 ,j − u k · u k i,j − u k 29 30 i,j i − 1 ,j g k i,j := max , 0 g h 1 h 1 31 32 � �� 33 34 u k i,j +1 − u k · u k i,j − u k i,j i,j − 1 + max , 0 . 35 36 h 2 h 2 37 38 39 40 � same quadratic consistency order as standard discretisation 41

M I FAB Diffusion: Explicit Scheme (3) A 1 2 Two Technical Definitions 3 4 5 6 The grey values of f = ( f i ) ∈ R N are restricted to a finite interval of length � 7 8 9 10 f i − min R := max f i . i i 11 12 13 14 15 16 � Since g is continuous and c 1 > c 2 ≥ 0 , there exists a constant ω > 0 such that 17 18 g ( s 2 ) > c 2 19 20 ∀ s ∈ (0 , ωR ) . 21 22 23 24 25 26 c 1 27 28 c 2 29 30 31 32 0 33 34 | ∇ u | ωR 35 36 − c 2 37 38 39 40 41 M I FAB Diffusion: Explicit Scheme (4) A 1 2 Theorem [Theory for the Explicit FAB Scheme] 3 4 With the preceding assumptions and definitions, consider the explicit scheme for FAB 5 6 diffusion with nonstandard discretisation. 7 8 If the time step size τ satisfies 9 10 11 12 ω 2 h 4 1 h 4 13 14 2 τ ≤ � , � � � ω 2 h 2 h 2 1 + h 2 1 h 2 2 + h 2 1 + h 2 2 c 1 · · 15 16 2 2 17 18 then this scheme has the following properties: 19 20 21 22 � Well-Posedness 23 24 For every k ∈ N 0 , the solution u k +1 depends in a continuous way on perturbations 25 26 of the initial image f . 27 28 29 30 � Average Grey Value Invariance 31 32 33 34 N N 1 1 � � 35 36 u k j = f j =: µ ∀ k ∈ N 0 . N N 37 38 j =1 j =1 39 40 41

M I FAB Diffusion: Explicit Scheme (5) A 1 2 � Maximum-Minimum Principle 3 4 5 6 f j ≤ u k min i ≤ max f j ∀ i, ∀ k ∈ N 0 . 7 8 j j 9 10 11 12 � Lyapunov Sequence 13 14 V k := max 15 16 u k u k j − min j j j 17 18 is a Lyapunov sequence: decreasing in k and bounded from below. 19 20 21 22 � Convergence to a Constant Steady State 23 24 25 26 k →∞ u k ∀ i. lim i = µ 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 M I Outline A 1 2 Outline 3 4 5 6 � FAB Diffusion 7 8 • Continuous Model 9 10 11 12 • Explicit Scheme 13 14 • Efficient Numerics 15 16 • Experiments 17 18 19 20 � Backward Diffusion with Convex Energy 21 22 • Model and Theory 23 24 25 26 • Numerical Algorithm 27 28 • Experiment 29 30 31 32 � Conclusions 33 34 35 36 37 38 39 40 41