Frequency and ridge estimation using structure tensor Anna - PowerPoint PPT Presentation

Frequency and ridge estimation using structure tensor Anna Mikaelyan and Josef Bigun October 15, 2013

Estimation of image orientation In some image processing scenarios it is necessary to use orientation image: ◮ motion estimation ◮ noise reduction ◮ detection of corner points ◮ estimation of line/ridge flow

Estimating orientation in fingerprint/fingermark Initial motivation for calculating orientation ◮ noise reduction ◮ separate dominant orientation from non-relevant one (especially in case of fingermark)

Structure tensor (ST) - tool for orientation estimation Linear symmetry tensor (version of ST) � I 20 (( D x + i · D y ) f ) 2 � � � � L = = I 11 � | ( D x + i · D y ) f | 2 Why use LST? ◮ continuous (dense) way to represent information ◮ intuitive image representation: ◮ I 20 = ( λ max − λ min ) e 2 ∠ u max , where u max - eigenvector associated with the largest eigenvalue λ max . ◮ I 11 defines contrast range of the orientation estimation

Research problem We want to estimate orientation of the image based on ST: ST How to tune ST to create ’best’ orientation image?

Implementation of Structure tensor (ST) 1. Convolve image f with the (complex) filter h: f cgrad = h ∗ f with 2 πσ 2 e − x 2+ y 2 1 2 σ 2 , h ( x, y, σ 2 in ) = ( D x + iD y ) g = ( D x + iD y ) where σ 2 in fixates the inner-scale (the frequency-contents) resulting in an image with complex valued pixels, f cgrad

Implementation of Structure tensor (ST) 1. Convolve image f with the (complex) filter h: f cgrad = h ∗ f with 2 πσ 2 e − x 2+ y 2 1 2 σ 2 , h ( x, y, σ 2 in ) = ( D x + iD y ) g = ( D x + iD y ) where σ 2 in fixates the inner-scale (the frequency-contents) resulting in an image with complex valued pixels, f cgrad 2. Apply pointwise (complex) squaring to obtain infinitesimal linear symmetry f ils ( x, y ) = f 2 cgrad

Implementation of Structure tensor (ST) 1. Convolve image f with the (complex) filter h: f cgrad = h ∗ f with 2 πσ 2 e − x 2+ y 2 1 2 σ 2 , h ( x, y, σ 2 in ) = ( D x + iD y ) g = ( D x + iD y ) where σ 2 in fixates the inner-scale (the frequency-contents) resulting in an image with complex valued pixels, f cgrad 2. Apply pointwise (complex) squaring to obtain infinitesimal linear symmetry f ils ( x, y ) = f 2 cgrad 3. Convolve f ils and | f ils | with a Gaussian filter g ( x, y, σ 2 out ) defining the neighbourhood (the outer-scale)

Implementation of Structure tensor (ST) 1. Convolve image f with the (complex) filter h: f cgrad = h ∗ f with 2 πσ 2 e − x 2+ y 2 1 2 σ 2 , h ( x, y, σ 2 in ) = ( D x + iD y ) g = ( D x + iD y ) where σ 2 in fixates the inner-scale (the frequency-contents) resulting in an image with complex valued pixels, f cgrad 2. Apply pointwise (complex) squaring to obtain infinitesimal linear symmetry f ils ( x, y ) = f 2 cgrad 3. Convolve f ils and | f ils | with a Gaussian filter g ( x, y, σ 2 out ) defining the neighbourhood (the outer-scale)

Implementation of Structure tensor (ST) 1. Convolve image f with the (complex) filter h: f cgrad = h ∗ f with 2 πσ 2 e − x 2+ y 2 1 2 σ 2 , h ( x, y, σ 2 in ) = ( D x + iD y ) g = ( D x + iD y ) where σ 2 in fixates the inner-scale (the frequency-contents) resulting in an image with complex valued pixels, f cgrad 2. Apply pointwise (complex) squaring to obtain infinitesimal linear symmetry f ils ( x, y ) = f 2 cgrad 3. Convolve f ils and | f ils | with a Gaussian filter g ( x, y, σ 2 out ) defining the neighbourhood (the outer-scale) ( σ in , σ out ) are important parameters of ST defining its scale

Finding the Frequency Content of a Signal I. σ in : y = Asin ( ω 0 r ) ω 0 = 2 π T 0

Finding the Frequency Content of a Signal I. σ in : y = Asin ( ω 0 r ) period T 0 ∈ [ T min , T max ] ω 0 = 2 π T 0

Finding the Frequency Content of a Signal I. σ in : y = Asin ( ω 0 r ) amplitude A period T 0 ∈ [ T min , T max ] ω 0 = 2 π T 0

Finding the Frequency Content of a Signal I. σ in : Linear Shift-Invariant System y = Asin ( ω 0 r ) amplitude A period T 0 ∈ [ T min , T max ] ω 0 = 2 π T 0

Finding the Frequency Content of a Signal I. σ in : Linear Shift-Invariant System y = Asin ( ω 0 r ) amplitude A y ′ = Bsin ( ω 0 r + φ 0 ) period T 0 ∈ [ T min , T max ] ω 0 = 2 π T 0

Finding the Frequency Content of a Signal I. σ in : Linear Shift-Invariant System y = Asin ( ω 0 r ) amplitude A y ′ = Bsin ( ω 0 r + φ 0 ) period T 0 ∈ [ T min , T max ] ω 0 = 2 π T 0

Finding the Frequency Content of a Signal I. σ in : Linear Shift-Invariant System y = Asin ( ω 0 r ) amplitude A y ′ = Bsin ( ω 0 r + φ 0 ) period T 0 ∈ [ T min , T max ] ω 0 = 2 π T 0 II. ( I. ) 2

Finding the Frequency Content of a Signal I. σ in : Linear Shift-Invariant System y = Asin ( ω 0 r ) amplitude A y ′ = Bsin ( ω 0 r + φ 0 ) period T 0 ∈ [ T min , T max ] ω 0 = 2 π T 0 II. ( I. ) 2

Finding the Frequency Content of a Signal I. σ in : Linear Shift-Invariant System y = Asin ( ω 0 r ) amplitude A y ′ = Bsin ( ω 0 r + φ 0 ) period T 0 ∈ [ T min , T max ] ω 0 = 2 π T 0 Convolution II. ( I. ) 2 ∗ Gaussian, σ out

Frequency in Fourier Domain σ − 1 o ω o

Frequency in Fourier Domain σ − 1 o ω 2 o

Estimation of parameters σ in , σ out σ in - fixates inner-scale, i.e. the frequency contents σ in = 0 . 9 σ in = 2 . 5

Estimation of parameters σ in , σ out , cont. σ out = 1 . 57 σ in - fixates outer-scale, i.e. integration σ out = 1 . 5 σ out = 3

Updates on Structure tensor (ST) ◮ Estimation of the image information from ST ◮ log( I 11 ) = log | 2 − 1 Aω 2 0 | − ω 2 0 σ 2 in ω 0 is estimated by fitting a line to observations of log( I 11) versus σ 2 in (by which log( I 11) is computed). ◮ Estimate parameters of the ST automatically from the image ◮ σ in = 1 ω 0 , where ω 0 - local frequency of the ridge pattern ◮ σ out = 1 . 57 σ in

Updates on Structure tensor (ST) ◮ Estimation of the image information from ST ◮ log( I 11 ) = log | 2 − 1 Aω 2 0 | − ω 2 0 σ 2 in ω 0 is estimated by fitting a line to observations of log( I 11) versus σ 2 in (by which log( I 11) is computed). ◮ Estimate parameters of the ST automatically from the image ◮ σ in = 1 ω 0 , where ω 0 - local frequency of the ridge pattern ◮ σ out = 1 . 57 σ in

Orientation Estimation Procedure Dense frequency estimation ⇒ Dense orientation estimation ⇒ Image enhancement ⇒ Dense frequency estimation ⇒ · · · Repeat until convergence in (global) mean frequency is achieved. Retain the dense images in the last cycle, i.e. enhanced image, orientation map and frequency map and use them for ridge counting.

Orientation Estimation Procedure Dense frequency estimation ⇒ Dense orientation estimation ⇒ Image enhancement ⇒ Dense frequency estimation ⇒ · · · Repeat until convergence in (global) mean frequency is achieved. Retain the dense images in the last cycle, i.e. enhanced image, orientation map and frequency map and use them for ridge counting. Figure: Ground truth image with frequency ω = 2 ∗ π/ 13 = 0 . 4833

Frequency estimation ⇒ Orientation estimation ⇒ Image enhancement Dense frequency estimation (no-orientation is necessary to know) ◮ Frequency estimation is done by fitting orientation to scale space. In turn this requires at least computation of a discrete scale space for log ( I 11) . ◮ This is achieved by computing log ( I 11) at 3 or 5 different in chosen around a σ 2 determined by the inner scales σ 2 global mean omega (via σ 2 = 1 /ω 2 ). ◮ For fingermarks 3 inner scales were sufficient. For high resolution frequency estimation of synthetic images with heavy noise we used 3 scales.

Frequency estimation ⇒ Orientation estimation ⇒ Image enhancement ◮ Compute the average frequency mean om and variance var om of the frequency over the entire image (or ROI). ◮ Around mean om define 5 frequencies to sample by help of var om. ◮ The mean om and its variance estimated in the synthetic test image were (left and right parts): MeanL VarL MeanR VarR 0.4839 0.0000 0.4849 0.0001, Groundtruth is 0.4833

Frequency estimation ⇒ Orientation estimation ⇒ Image enhancement ◮ Use the sigma that corresponds to these frequencies as inner scales for linear symmetry algorithm and generate 5 different orientation maps ◮ Merge the different orientation maps to a master orientation map by picking orientation+certainty according to the closest frequency in the omega image.

Frequency estimation ⇒ Orientation estimation ⇒ Image enhancement For each pixel of the original image we know (or ROI), we generate dynamically a Gabor filter, matching exactly the latest values in corresponding pixels in the frequency map and orientation map of the latest iteration.

Iteration Frequency estimation ⇒ Orientation estimation ⇒ Image enhancement Master orientation and master frequency maps are obtained as those in the final cycle, when mean global frequency has converged. Typically it requires 2-4 cycles for convergence to occur for SD27 fingermarks.

Orientation Estimation Frequency ( ω ) Estima- tion ω -converge Output: I20 ( ∈ [0 , 1])

Automatic extraction of ridge distance between minutia Goal: pressure invariant/resilient (minutia) constellation features, allowing to replace/complement distance measurements between minutia in matchers.