Scale-space theory and PDE-based image filtering Nicolas Rougon - PowerPoint PPT Presentation

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Primary visual cortex ◮ The 6 LGN layers are mapped via the optic radiation to the primary visual cortex (V1) V7 V3a ◮ V1 maintains a retinotopic map The central 10° of the visual field occupies V3 roughly half of V1 V2 V1 This distorsion, called cortical magnification, V4 echoes the increased acuity of the fovea 5 6 7 8 9 4 The L(R) retinal hemifield is 1 2 3 7 89 1 2 4 3 5 6 6 mapped onto the R(L) 8 7 9 5 hemisphere V1 (lateralization) Scene RIGHT eye V1 Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography V1 cells ◮ V1 neurons organize into a layered architecture I I Pyramidal cell II + III II III IVa IVa,b IVb IVc IVc V Axons V VI VI Stellate cell Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography V1 physiology ◮ V1 physiology and functional organization have been elucidated by David Hubel and Torsten Wiesel (Nobel Prize in Medecine, 1981) ◮ V1 neurons divide into simple / complex / hypercomplex cells. These 3 types differ by the way they respond to visual stimuli Neuron activity is monitored via electrophysiology Most V1 neurons are orientation selective i.e. respond strongly to lines/bars/edges with a specific orientation Some V1 neurons are direction selective i.e. respond strongly to oriented lines/bars/edges moving in a specific direction Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Receptive fields ◮ The LGN / V1 retinopic mapping enables correlating cell / neuron activity with retinal stimuli Receptive field Region of the retina influencing the activity of a cell / neuron assembly when exposed to a light stimulus Receptive fields of V1 simple cells best stimulus A B C D A elongated light bar B elongated dark bar C elongated dark bar D edge X excitatory zone inbibitory zone Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Receptive fields selectivity best cell type geometry binocular orientation direction disparity stimuli X-Y ganglion light no no no no M-P-K blob no no no no elongated simple yes some some some bar / edge elongated complex yes yes some yes bar / edge short edge hypercomplex yes yes some yes corner Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography V1 functional architecture ◮ V1 cells organize functionally into a columnar architecture Neurons with activity mainly Orientatjon influenced by one eye organize columns into ocular dominance columns Complex cells III IVa IVb Neurons with a given orientation Simple cells IVc selectivity organize into orientation Ocular columns dominance columns Contraletral eye Hypercolumns gather neurons Ipsilateral eye having the same receptive field location, but all different 6 5 4 orientation/direction selectivities 3 LGN 2 1 and both eye dominances Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography V1 functional architecture ◮ V1 columns can be investigated in vitro via histology in vivo via intrinsic optical imaging or high-field fMRI orientations columns ocular dominance columns Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Biological visual information processing ◮ Governing principles Retinopic mapping Retinal maps are preserved / registered along visual pathways Functional simplicity Visual cells / neurons are divided into a few specialized types with preferred response to a given class of stimuli Architectural efficiency Visual cells / neurons with the same dominance / selectivity organize into layers Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Biological visual information processing The human visual system ≡ a geometric engine ◮ Along the pathways to the cortex, visual information is processed in a massively parallel way locally at multiple integration scales hierarchically with no feedback in its early steps selectively with an increasing degree of nonlinearity ◮ These biological features are reflected by scale-space theory which can be seen as a mathematical theory of early vision Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Scale-space theory ◮ A deterministic framework for deriving hierarchical image / shape representations according to level of detail (LOD) based on continuous data modeling images as smooth functions over Ω ⊂ R n – shapes as smooth submanifolds of R n – applicable to nD , scalar/vector, still/animated data delivering scene decomposition into LOD – scene description at given LOD → filtering / restoration – relations between scene components at varying LODs – object assignment to distinctive scale range enabling multiscale image analysis – feature extraction – motion estimation / tracking – segmentation / grouping – scene / shape reconstruction Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Scale-space theory ◮ Though developed in image science, scale-space theory is deeply connected to calculus ◮ differential geometry | PDE theory | variational calculus theoretical physics ◮ quantum field theory behavioral / cognitive neurosciences ◮ neuropsychology | psychobiology | psychophysics Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Scale operators scale ◮ Scale-spaces are built / explored t using local image operators t 2 parameterized by a scale variable T t 1 , t 2 t ∈ [ t , t ] ⊂ R + t 1 T T t inner scale (pixel size) t 1 t 2 t t outer scale (image size) 0 image space scale operators T t produce images at scale t from original ones scale transition operators T t , t ′ generate images at scale t ′ from images at lower scale t the image family ( T t f ) t is the scale-space representation of the image f Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Structural axioms ◮ Causality axioms ensure that scale operators do not create details Semigroup Scale-spaces have a Associativity T t 1 T t 2 = T t 1 + t 2 hierarchical structure ≡ image pyramids Identity T 0 = I d Local comparison ∀ y ∈ N ( x ) f ( y ) > g ( y ) ⇒ ( T t f ) ( x ) > ( T t g ) ( x ) for any scale t less than an extinction scale t e Scale operators preserve ordering between image level sets during their lifetime t e in scale-space Scale operators do not enhance any image structures Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Structural axioms ◮ The smoothness axiom ensures that image simplification by scale operators is a C 1 -continuous process Smoothness Scale operators have a continuous derivative ∂ T at t = 0 T t f − f ∂ T f = lim t t → 0 ∂ T is known as the infinitesimal generator of the scale operator semigroup (or, in short, as the scale-space generator) Scale operators preserve image smoothness Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Structural axioms ◮ Smoothness axiom Let Q f , x be the quadratic approximation of the (locally smooth) image f over the neighborhood N ( x ) of pixel x ∈ R d Q f , x ( y ) = ( x − y ) T A ( x − y ) + p T ( x − y ) + c ( A symmetric ( d × d ) matrix, p ∈ R d , = c ∈ R The smoothness axiom states that T t Q f , x − Q f , x ∂ T Q f , x = lim t t → 0 is a function F ( A , p , c , x , t ) continuous w.r.t. the highest frequency component A Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Multiscale vs multiresolution representations ◮ Whereas scale-space methods operate on the original pixel grid, multiresolution techniques combine image simplication with grid decimation to yield image pyramids level subsampling (usually dyadic) u ( k − 1) = ↓ 2 u ( k ) n -2 smoothing n -1 N � u ( k − 1) ( x ) = c n u ( k ) (2 x − n ) n = − N n Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Multiscale vs multiresolution representations ◮ Low-pass pyramid c n ≥ 0 positivity unimodularity c | n | ≥ c | n | +1 symmetry c n = c − n � c n = 1 normalization � c 2 n = � c 2 n +1 equidistribution ◮ Example: Gaussian pyramids N = 1 (binomial filter) N = 2 ( a ≈ 0 . 4) � � � � 1 1 1 2 1 1 − 2 a 1 4 a 1 1 − 2 a 4 4 ◮ Differences of low-pass pyramids yield band-pass pyramids Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Multiscale vs multiresolution representations ◮ Gaussian scale-space t 0 1 2 3 4 ◮ Gaussian pyramid (rescaled to full resolution) 0 1 2 3 4 k Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Generating PDE Generating PDE The scale-space representation of the image f is the solution u ( x , t ) = ( T t f )( x ) of the PDE u t = ∂ T u with u ( · , 0) = f and Neumann (mirroring) boundary conditions ◮ Photometric interpretation: The generating PDE describes how luminance varies during a scale transition t 0 10 25 75 150 300 600 Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Generating PDE ◮ Geometric interpretation: The generating PDE determines how image level sets evolve during a scale transition Level set evolution is described by a speed law x t = V ( x ) n along its external normal n = − ∇ u |∇ u | V ( x ) is obtained by substitution from differenciating the level set equation u ( x , t ) = c u t + ∇ u · x t = 0 Level set scale-space flow V ( x ) = ∂ T u |∇ u | Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Scale-space generators Representation theorem (Alvarez-Morel-Lions) Scale-space generators are differential operators of order n ≤ 2 ∂ T u = F ( D 2 u , Du , u ) ◮ The generating PDE combines 3 processes diffusion responsible for smoothing is driven by order-2 terms related to image curvature/convexity ◮ parabolic reaction responsible for transport is driven by order-1 terms related to local contrast/orientation ◮ hyperbolic addition responsible for luminance transformation is driven by order-0 terms ◮ The dominant process is dictated by the highest-order term Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Morphological axioms ◮ Morphological axioms state how scale operators depend / act on image content (involved features / targeted structures) ◮ Added to structural axioms, they allow for completely elucidating scale-space generators under closed-form ◮ They divide into 2 groups Linearity T t ( α u + β v ) = α T t u + β T t v Invariance ≡ commutation between scale operators and a specific group G of transforms strong t ′ = t – ∀ g ∈ G g T t ′ = T t g weak t ′ = ϕ ( t ) – (resynchronization) Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Morphological axioms ◮ Linearity Universality: scale-space is built w/o any prior assumption Blind processing: scale-space operators act similarly on image content whatever its (local) characteristics ◮ Invariance Introducing priors implies nonlinearity Globally: G models some variability on sensor The scale-space representation of an image does not depend on the related sensor calibration Locally: G models some variability on image content Scale operators act similarly on image structures whatever their related appearance Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Morphological axioms ◮ Photometric invariance G consists of transforms over the luminance interval [0 , 2 b − 1] group transforms invariance general invertible strong ( T t f ) t does not depend on sensor photometric calibration T t acts similarly on image content whatever its contrast Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Morphological axioms ◮ Spatial invariance G consists of transforms over the image domain R d group transforms invariance Euclidean isometry strong � t ′ = affine affine weak | det( g ) | t projective perspective weak ( T t f ) t does not depend on sensor geometric calibration Rotation-invariance ≡ T t has no preferred orientation Image processing is isotropic Zoom-invariance ≡ T t has no preferred extension Image structures are processed similarly whatever their size Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Motivation Geometric scale-spaces Biological foundations Conservative scale-spaces Mathematical foundations Bibliography Scale-space families ◮ Scale-spaces can be classified into 3 families depending on the set of selected morphological axioms Structural Morphological axioms axioms linearity invariance contrast Geometric non linear spatial Conservative linear Gaussian Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Outline Foundations 1 Linear scale-space 2 Geometric scale-spaces 3 Conservative scale-spaces 4 Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Linear scale-space ◮ Assuming linearity leads to a single scale-space model Generating PDE (Marr-Koenderink-Witkin) There is a unique linear, isometry-invariant and zoom-invariant scale-space. Its generating PDE is the isotropic heat equation u t = ∆ u This model is referred to as the linear scale-space Its generator is the Laplacian operator ∂ T = ∆ Image simplification is performed via luminance diffusion Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Linear scale-space ◮ Solutions of linear PDEs are obtained by convolving their initial condition against a particular solution (kernel) The heat equation kernel is the isotropic Gaussian � � −| x | 2 1 G σ ( x ) = exp d 2 σ 2 (2 πσ 2 ) 2 Scale operators The scale-operators for the linear scale-space are Gaussian convolutions with variance proportional to scale T t = G √ 2 t ⋆ Equivalent terminology: Gaussian scale-space Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Extensions of linear scale-space ◮ Spatial frequency tuning: Gabor filters � � −| x | 2 1 G σ, k ( x ) = exp exp ( − i k · x ) 2 σ 2 d (2 πσ 2 ) 2 The wave vector k defines spatial orientation/frequency frequency Sampling the ( t , k )-space yields the Gabor space Widely used for texture modeling and discrimination in computer vision / pattern recognition ( e.g. character, fingerprint, iris) Relevant model for simple cells in mammals visual cortex orientatjon Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Extensions of linear scale-space ◮ Spatial tuning � � −| x − ξ | 2 1 G σ, ξ ( x ) = exp d 2 σ 2 (2 πσ 2 ) 2 Modeling receptive field assemblies in bio-inspired vision ◮ Spatio-temporal linear scale-space � � −| x | 2 − τ 2 1 1 G σ s , σ τ ( x ) = exp d 1 2 σ 2 2 σ 2 (2 πσ 2 (2 πσ 2 s ) τ ) 2 2 s τ σ 2 s = 2 t (spatial scale) | σ 2 τ = 2 τ (temporal scale) Multiscale modeling / analysis of video sequences Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Properties of linear scale-space ◮ Causality: the generating PDE verifies a maximum principle Ω × R + u ( x , t ) = max max u ( x , 0) Ω d = 1: non-creation of local extrema ( T t f ) t (∆ T t f = 0) t t e d > 1: non-enhancement of local extrema survival time t e is a measure of feature saliency Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Properties of linear scale-space ◮ Smoothness: the linear scale-space yields C ∞ image families ◮ Convolution theorem: image derivatives belong to scale-space ∂ x i y j u = G √ 2 t ⋆ ∂ x i y j f Image local geometry is extensively available in the linear scale-space ≡ universal front-end for image understanding Gaussian derivatives ≡ precomputable geometric kernels ∂ x i y j u = ∂ x i y j G √ 2 t ⋆ f Natural Gaussian derivatives w.r.t. normalized coordinates � x = x /σ are multiscale extensions of standard derivatives y j G √ t → 0 ∂ � lim 2 t = ∂ x i y j x i � Images are processed as distributions Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Properties of linear scale-space ◮ Gaussian differential kernels ( d = 2) G σ ( G σ ) x ( G σ ) y ( G σ ) xx ( G σ ) xy ( G σ ) yy ( G σ ) xxx ( G σ ) xxy ( G σ ) xyy ( G σ ) yyy D 2 n G σ are wavelets Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Properties of linear scale-space ◮ Multiscale feature extraction u |∇ u | ∆ u Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Properties of linear scale-space ◮ Isotropy: simplication is performed by diffusing out image blobs ◮ Separability: 1D computations In the spatial domain using order-4 IRR filters [Deriche] or Hermite polynomials H n � x � x G σ = ( − 1) n ∂ n H n G σ σ n σ x e − x 2 = ( − 1) n H n ( x ) e − x 2 ∂ n In the spectral domain using FFT G σ ⋆ f = FFT − 1 � � FFT( G σ ) · FFT( f ) Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Discrete linear scale-space ◮ Transposing the linear scale-space framework from R d to digital grids Ω ⊂ Z d is a hard problem Directly discretizing the Gaussian scale operators results in violating causality axioms ◮ An intrinsically discrete derivation of scale operators, ensuring that scale-space structural axioms hold over Z d , is mandatory The key point lies in satisfying the non-enhancement of local extrema property in a discrete setting Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Discrete linear scale-space 1D discrete linear scale-space (Lindeberg) The scale operators for the 1D discrete linear scale-space are convolutions against a discrete kernel T t T t = T t ⋆ which is related to Bessel functions J n T t ( n ) = e − α t I n ( α t ) I ± n ( t ) = ( − i ) n J n ( it ) T t is known as the discrete analog of the Gaussian kernel G σ Sampling the continuous Gaussian scale operators does not lead to discrete linear scale operators Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Discrete linear scale-space Generating PDE ( d = 1) (Lindeberg) The 1D discrete linear scale-space is generated by the semi-discrete heat equation � � u t ( x , t ) = 1 u ( x + 1 , t ) − 2 u ( x , t ) + u ( x − 1 , t ) 2 Its generator is the discrete Laplacian kernel � 1 � ∆ 3 = − 2 1 This PDE holds exactly Recursion properties of Bessel functions The discrete linear scale-space is properly derived by discretizing the continuous linear scale-space generator Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Discrete linear scale-space Generating PDE ( d = 2) (Lindeberg) A 2D discrete linear scale-space is generated by a semi-discrete heat equation ( α ∈ [0 , 1]) u t = α ∆ 4 u + (1 − α ) ∆ 8 u ∆ 4 , ∆ 8 are discrete Laplacian kernels     1 1 1 ∆ 8 = 1     ∆ 4 = 1 − 4 1 − 4     2 1 1 1 For d > 1, the discrete linear scale-space is not unique Each generator relates to a local topology on the image grid e.g. separability: α = 1 | isotropy: α = 2 3 Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Discrete linear scale-space Generating PDE ( d = 3) (Lindeberg) A 3D discrete linear scale-space is generated by a semi-discrete heat equation ( α, β ∈ [0 , 1] 2 ) u t = α ∆ 6 u + β ∆ 18 u + (1 − α − β ) ∆ 26 u ∆ 6 , ∆ 18 , ∆ 26 are discrete Laplacian kernels � ∆ 6 u ijk = u lmn − 6 u ijk N ∗ 6 N ∗ 6 ( ijk ) � � � ∆ 18 u ijk = 1 u lmn − 12 u ijk N ∗ 4 18 \ 6 N ∗ 18 \ 6 ( ijk ) � � � ∆ 26 u ijk = 1 u lmn − 8 u ijk N ∗ 4 26 \ 18 N ∗ 26 \ 18 ( ijk ) Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Discrete linear scale-space ◮ Implementation Let ( ∂ T u ) ij be a discrete linear scale-space generator at pixel ij Finite difference discretization of u t ( u t ) ij = u ij ( t + δ t ) − u ij ( t ) δ t Explicit scheme � ∂ T u ( t ) � u ij ( t + δ t ) = u ij ( t ) + δ t ij Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Limitations of linear scale-space ◮ Oversmoothing artifacts due to strong regularization properties of Gaussian scale operators T t contrast loss of salient image structures ◮ Non-preservation of image geometry due to linearity and isotropy of Laplacian scale-space generators ∂ T delocation of image structures orientation smoothing ◮ complex multiscale image analysis schemes Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE & Scale operators Linear scale-space Properties Geometric scale-spaces Discrete linear scale-space Conservative scale-spaces Limitations Bibliography Limitations of linear scale-space t 0 4 16 64 256 ◮ Overcoming these limitations requires nonlinear and anisotropic scale-space models Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Outline Foundations 1 Linear scale-space 2 Geometric scale-spaces 3 Conservative scale-spaces 4 Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Euclidean geometric scale-spaces Generating PDE (Alvarez-Morel-Lions) Contrast- and isometry-invariant scale-spaces are generated by reaction-diffusion PDEs of the form � curv( u ) � u t = |∇ u | F where F is an increasing function of image level set mean curvature � � ∇ u curv( u ) = ∇ · |∇ u | ◮ Geometric interpretation: Image simplification is performed by evolving level sets according to a purely geometric speed law � curv � u ( x ) �� V ( x ) = F These models are known as Euclidean geometric scale-spaces Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Euclidean geometric scale-spaces ◮ Classical models F ( x ) PDE type model reaction differential c (hyperbolic) mathematical morphology diffusion Euclidean intrinsic x (parabolic) scale-space reaction-diffusion entropic c + α x (parabolic) scale-space Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Morphological scale-spaces ◮ Multiscale structuring elements � x � Functions: g t ( x ) = tg Sets: tB = { tb : b ∈ B } t g 0 ( 0 ) = 0 0 B = { 0 } Multiscale morphological operators E g t f = f ⊖ ˇ E g g t Multiscale erosion with 0 = I d t f = f ⊕ ˇ D g D g Multiscale dilation g t with 0 = I d E g and D g t are dual operators t Multiscale erosions/dilations w.r.t. structuring sets are derived by using flat structuring functions g : B → { 0 } Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Morphological scale-spaces Scale-space structure (Brockett-Maragos) Multiscale dilations D g t (erosions E g t ) are scale operators D g t 1 D g t 2 = D g E g t 1 E g t 2 = E g ◮ Semigroup: and t 1 + t 2 t 1 + t 2 for nonnegative concave functions g ( i.e. with convex subgraph) ◮ Local comparison: D g t ( E g t ) is an increasing operator ◮ Smoothness: Modeling images as Lipschitz functions, the semi- � D g � � E g � t ) has an infinitesimal generator ∂ D g ( ∂ E g ) group t ( t t ∂ E g = − ∂ D g Duality: Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Morphological scale-spaces Generating PDE (Brockett-Maragos-Boomgaard) Morphological scale-spaces are generated by eikonal equations i.e. (hyperbolic) Hamilton-Jacobi PDEs dilation PDE V ( x ) g u t = |∇ u | 1 unit ball 1 u t = max | u x i | |∇ u | max | u x i | unit diamond � � 1 u t = | u x i | | u x i | unit cube |∇ u | � � 1 + g 2 ( 0 ) |∇ u | 2 + g 2 ( 0 ) u t = unit disc |∇ u | 2 u t = |∇ u | 2 |∇ u | parabola Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Morphological scale-spaces ◮ Implementation In order to handle shocks along image level sets concavities, the generators ∂ T u are discretized using entropic schemes based on lateral finite difference approximations of ∇ u D + x u ij = u i +1 , j − u ij D − = u ij − u i − 1 , j x u ij D + = u i , j +1 − u ij y u ij D − = u ij − u i , j − 1 y u ij Explicit scheme � ∂ T u ( t ) � u ij ( t + δ t ) = u ij ( t ) + δ t ij Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Morphological scale-spaces ◮ Implementation Spherical dilation ∂ T u = |∇ u | � � � � 2 � � D + � 2 + min � D − |∇ u ij | = max ξ u ij , 0 ξ u ij , 0 ξ = x , y ∂ T u = −|∇ u | Spherical erosion � � � � 2 � � D + � 2 + max � D − |∇ u ij | = min ξ u ij , 0 ξ u ij , 0 ξ = x , y Note: the same schemes are used for discretizing pressure forces in level set-based active contour models Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Euclidean intrinsic scale-space Generating PDE (Morel-Osher-Sethian-Gage-Hamilton) The Euclidean intrinsic scale-space is generated by the (parabolic) Euclidean intrinsic heat equation � ∇ u � u t = |∇ u | ∇ · |∇ u | ◮ Geometric interpretation: Image simplification is performed by evolving level sets according to the mean curvature motion x t = curv( u ) n This is the speed law of a membrane Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Euclidean intrinsic scale-space ◮ Mean curvature motion decreases | curv( u ) | 1 First, level sets are convexified Discontinuities are instantly smoothed out 2 Once convex, level sets are then contracted to a point which finally vanishes Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Gauge coordinates ◮ In global Cartesian coordinates, generating PDEs can be complex and does not highlight how image geometry is simplified ◮ Hence the idea of finding simpler expressions in a local coordinate system related to image geometry Relevant local frames should share the contrast- and isometry- invariance properties of scale operators These properties are verified by image level lines Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Gauge coordinates level n = − ∇ u lines |∇ u | t = ∇ ⊥ u stream y lines n |∇ u | u = cte O x t ◮ The local frame ( t , n ) induces a local coordinate system ( ξ, η ) on the image grid ξ ( η ) is an arclength along level (stream) lines ∂ ξ , ∂ η are Lie derivatives � = � ∂ ξ , ∂ η � t · ∇ , n · ∇ � Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Gauge coordinates ◮ 1 st -order local image derivatives u ξ = 0 u η = −|∇ u | Gauge property: Choosing a reference for u yields a simplified representation of ∇ u ◮ 2 nd -order local image derivatives u ξξ = |∇ u | − 2 � u xx u 2 � y − 2 u xy u x u y + u yy u 2 x u ξη = |∇ u | − 2 � u xx u x u y − u yy u x u y + u xy u 2 � y − u xy u 2 x u ηη = |∇ u | − 2 � u xx u 2 � x + 2 u xy u x u y + u yy u 2 y Level line curvature Stream line curvature curv( u ) = − u ξξ curv ⊥ ( u ) = − u ξη u η u η Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Euclidean intrinsic vs linear scale-space ◮ Expressing generating PDEs in gauge coordinates highlights how scale-space acts on image local geometry scale-space Cartesian coordinates gauge coordinates u t = ∆ u u t = u ξξ + u ηη linear � ∇ u � intrinsic u t = |∇ u | ∇ · u t = u ξξ |∇ u | Euclidean The isotropic heat equation has a diffusion component u ηη across image level sets, inducing inter-region smoothing (biais) The Euclidean intrinsic heat equation performs anisotropic diffusion solely along image level sets Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Affine intrinsic scale-space Generating PDE (Alvarez-Morel-Lions-Sapiro-Tannenbaum) There is a unique contrast- and affine-invariant scale-space, known as the affine intrinsic scale-space. Its generating PDE is the (parabolic) affine intrinsic heat equation � ∇ u � �� 1 3 u t = |∇ u | ∇ · |∇ u | ◮ Geometric interpretation: Image simplification is performed by evolving level sets according to the affine mean curvature motion � curv( u ) � 1 3 n x t = Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Euclidean/affine intrinsic scale-spaces ◮ Implementation The scale-space generator ∂ T u is discretized by substituting in its expression finite difference approximations of (D u , D 2 u ) ( u x ) ij = 1 2 ( u i +1 , j − u i − 1 , j ) ( u xx ) ij = u i +1 , j − 2 u ij + u i − 1 , j ( u xy ) ij = u i +1 , j +1 − u i − 1 , j +1 − u i +1 , j − 1 + u i − 1 , j − 1 To avoid singularities, mean curvature motion is replaced by (discrete) linear diffusion ( ∂ T u = ∆ u ) when |∇ u | ≪ 1 Explicit scheme � ∂ T u ( t ) � u ij ( t + δ t ) = u ij ( t ) + δ t ij Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Limitations of geometric scale-spaces ◮ Though better suited to content-adapted simplification than the linear models, geometric scale-spaces suffer from artifacts Level set curvature-driven PDEs delocate / convexify image structures and smooth out corners / junctions Image level sets (and more generally, all image structures) are processed similarly whatever their contrast ◮ Scale operators capable of processing image content selectively, by preserving well-contrasted level sets while filtering the others, would allow for improved representations Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Limitations of geometric scale-spaces Linear separable Gaussian Geometric Euclidean intrinsic Conservative Perona-Malik t 0 1 4 7 10 Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Limitations of geometric scale-spaces Linear separable Gaussian Geometric Euclidean intrinsic Conservative Perona-Malik t 0 2 5 8 11 Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Generating PDE Linear scale-space Morphological scale-spaces Geometric scale-spaces Intrinsic scale-spaces Conservative scale-spaces Limitations Bibliography Limitations of geometric scale-spaces Linear separable Gaussian Geometric Euclidean intrinsic Conservative Perona-Malik t 0 2 5 8 11 Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Outline Foundations 1 Linear scale-space 2 Geometric scale-spaces 3 Conservative scale-spaces 4 Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Anisotropic diffusion Generating PDE (Perona-Malik) The (parabolic) variable-conduction heat equation � g ( |∇ u | ) ∇ u � u t = ∇ · generates an isometry-invariant scale-space for any given C 1 -continuous, positive, decreasing function g such that g (0) = 1 x → + ∞ g ( x ) = 0 lim 1 2 This PDE is known as anisotropic diffusion (it is in fact an isotropic inhomogeneous diffusion equation) Diffusion is governed by the conduction function g xg ( x ) is known as the flux function Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Anisotropic diffusion ◮ Generating PDE in gauge coordinates � u η g ( u η ) � ′ u ηη u t = g ( u η ) u ξξ + Weak diffusion across image level sets occurs for high-contrast x → + ∞ [ xg ( x ) ] ′ = 0 values, even though lim Edge blurring is avoided if diffusion along level sets dominates. This is ensured if the conduction function g verifies g ( x ) lim [ xg ( x ) ] ′ = 0 x → + ∞ Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Anisotropic diffusion ◮ Convex flux result in smoothing name g ( x ) xg ( x ) tanh x Green x 1 L 1 - L 2 √ 1+ x 2 1 Fair 1+ x 1 Total variation x 0 x Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Anisotropic diffusion ◮ Nonconvex flux result in joint smoothing / contrast enhancement xg (x) name g ( x ) 2 e − ( x K ) Perona-Malik 1 Lorentzian 2 1+ ( x K ) 1 � 2 � 2 Geman-McClure 1+ ( x K ) K x 0 � 1 − � x � 2 � 2 x ≤ K Tuckey K x > K 0 Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Anisotropic diffusion ◮ Nonconvex flux result in joint smoothing / contrast enhancement The hyperparameter K acts as a contrast xg (x) threshold |∇ u | ≤ K Image is viewed as low-texture and smoothed K x 0 |∇ u | > K Image is viewed as salient edge and enhanced (the PDE behaves locally as an inverse heat equation) Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Anisotropic diffusion ◮ Influence of hyperparameter K K = 100 K = 400 t 2 5 8 11 Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Anisotropic diffusion ◮ Influence of hyperparameter K K = 100 K = 400 t 2 5 8 11 Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Properties of anisotropic diffusion ◮ Simplification: Anisotropic diffusion decreases image L p -norms, centered statistical moments and Shannon entropy ◮ Well-posedness: The variable-conduction heat equation is well- posed iff. the flux function is convex For nonconvex flux functions, inverse diffusion occurs when |∇ u | > K , generating local instability Joint image smoothing / contrast enhancement is ill-posed Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Well-posed anisotropic diffusion Generated PDE (Catté-Dibos-Whitaker-Pizer) The (parabolic) variable-conduction heat equation � g ( |∇ G σ ⋆ u | ) ∇ u � u t = ∇ · generates an isometry-invariant scale-space for any given σ > 0 and C 1 -continuous, positive, decreasing function g such that g (0) = 1 x → + ∞ g ( x ) = 0 lim 1 2 This PDE is well-posed Unconditional well-posedness is obtained by computing conduction from smooth contrast estimates This model is known as image selective smoothing Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Properties of anisotropic diffusion ◮ Conservation of mean luminance � � 1 1 u ( x , t ) d x = u ( x , 0) d x | Ω | | Ω | Ω Ω Variable-conduction diffusion scale-spaces are also known as conservative scale-spaces ◮ Trivial asymptotics: Anisotropic diffusion yields uniform images at the large scale limit � 1 t → + ∞ u ( x , t ) = lim u ( x , 0) d x | Ω | Ω Diffusion must be stopped before excessive loss of detail No optimal stopping criterion is currently available Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Biased anisotropic diffusion Generating PDE (Nordström) The (parabolic) variable-conduction heat equation � + λ ( u 0 − u ) � g ( |∇ u | ) ∇ u u t = ∇ · generates an isometry-invariant scale-space for any given λ > 0 and C 1 -continuous, positive, decreasing function g such that g (0) = 1 x → + ∞ g ( x ) = 0 lim 1 2 λ ( u 0 − u ) is a data link term, constraining filtered images u to remain similar to the original image u 0 Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Biased anisotropic diffusion Variational formulation (Barlaud-Aubert) Biased anisotropic diffusion corresponds to solving a regularized denoising problem by iteratively minimizing the energy � � ϕ ( |∇ u | ) d x + λ ( u 0 − u ) 2 d x E ( u ) = 2 Ω Ω where ϕ is a 1 st -order discontinuity-preserving stabilizer Variational derivative � ϕ ′ ( |∇ u | ) � ∂ u E ( u ) = −∇ · ∇ u − λ ( u 0 − u ) |∇ u | Biased anisotropic diffusion arises as a gradient descent u t = − ∂ u E ( u ) with conduction defined as g ( x ) = ϕ ′ ( x ) x Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Biased anisotropic diffusion ◮ Implementation Finite difference discretization of the diffusion term |∇ u | is approximated by absolute lateral differences � � � � g ( |∇ u | ) ∇ u � � | D + � D + D − ∇ · ij ≈ g ξ u ij | ξ u ij ξ ξ = x , y Denoting ∆ n ∆ w ij u = u i − 1 , j − u ij ij u = u i , j − 1 − u ij ∆ s ∆ e ij u = u i +1 , j − u ij ij u = u i , j +1 − u ij this rewrites as � � g ( |∇ u | ) ∇ u � g ( | ∆ ξ ij u | ) ∆ ξ ∇ · ij ≈ ij u ξ = n , s , w , e Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Biased anisotropic diffusion ◮ Implementation (cont’d) Finite difference discretization of the diffusion term This corresponds to a convolution � g ( |∇ u | ) ∇ u � ∇ · ij ≈ K ( u ij ) ⋆ u ij with an image-based diffusion kernel K ( u )  � | ∆ n �  ij u | g  � | ∆ w � − � g ( | ∆ ξ � | ∆ e �    ij u | ij u | ) ij u | K ( u ij ) = g g   � | ∆ s � ij u | g Note: In the linear diffusion limit ( ϕ ( x ) = x 2 i.e. g ( x ) = 1), K ( u ) reduces to the 4-connected Laplacian kernel ∆ 4 Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Biased anisotropic diffusion ◮ Implementation (cont’d) Finite difference discretization of the biaised diffusion term � ∂ T u � ij ≈ K ( u ij ) ⋆ u ij + λ ( u 0 − u ) ij Finite difference discretization of u t ( u t ) ij = u ij ( t + δ t ) − u ij ( t ) δ t Explicit scheme � ∂ T u ( t ) � u ij ( t + δ t ) = u ij ( t ) + δ t ij 1 ◮ Under the CFL condition δ t ≤ 4 , this scheme is stable and verifies a maximum principle Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Shock filters Generating PDE (Rudin-Osher) For any 2 nd -order diffusive (elliptic) operator L , the (hyperbolic) PDE � L ( u ) � u t = −|∇ u | sign is well-posed and yields piecewise C 0 images at the large scale limit L behaves as a 2 nd -order edge detector. Classical choices are L ( u ) = ( ∇ u ) T D 2 u ∇ u = u ηη L ( u ) = G σ ⋆ ∆ u Image level lines are pushed towards edges with unit speed � L ( u )( x ) � V ( x ) = sign Discontinuities are created as shocks along edges Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Shock filters ◮ Generalization A broad family of shock filters is derived by replacing the sign function by an arbitrary Lipschitz function F s.t. xF ( x ) > 0 � L ( u ) � u t = −|∇ u | F Given 0 < α ≤ 1, the well-posed (hyperbolic) PDE � � � � L ( u ) � 1 + |∇ u | 2 + (1 − α ) |∇ u | u t = − α F yields piecewise C 1 images at the large scale limit In both cases, the backward PDEs are also well-posed Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Anisotropic diffusion Geometric scale-spaces Biased anisotropic diffusion Conservative scale-spaces Shock filters Bibliography Conclusion ◮ Scale-space theory provides a well-established framework for describing images/shapes w.r.t. level of detail The linear scale-space is a universal visual front-end Nonlinear scale-spaces provide geometry-preserving models tailored to specific invariance constraints ◮ Scale-spaces enable hierarchical implementations of a variety of image/shape understanding problems Motion analysis Restoration/enhancement Feature extraction Matching Shape from X Segmentation Grouping/decomposition into parts . . . Nicolas Rougon IMA4509 | Scale-space & PDE filtering

Foundations Linear scale-space Geometric scale-spaces Conservative scale-spaces Bibliography Bibliography T. Lindeberg Scale space theory in computer vision Kluwer Academic Publishers, 1994 L.M.J. Florack Image structure Kluwer Academic Publishers, 1997 B.M. ter Haar Romeny Geometry-driven diffusion in computer vision Kluwer Academic Publishers, 1994 Nicolas Rougon IMA4509 | Scale-space & PDE filtering