Frequency and ridge estimation using structure tensor Anna - - PowerPoint PPT Presentation

Frequency and ridge estimation using structure tensor

Anna Mikaelyan and Josef Bigun October 15, 2013

Estimation of image orientation

In some image processing scenarios it is necessary to use

• rientation image:

◮ motion estimation ◮ noise reduction ◮ detection of corner points ◮ estimation of line/ridge flow

Estimating orientation in fingerprint/fingermark

Initial motivation for calculating orientation

◮ noise reduction ◮ separate dominant orientation from non-relevant one

(especially in case of fingermark)

Structure tensor (ST) - tool for orientation estimation

Linear symmetry tensor (version of ST)

L = I20 I11

• =

((Dx + i · Dy) f)2

• | (Dx + i · Dy) f|2
• Why use LST?

◮ continuous (dense) way to represent information ◮ intuitive image representation:

◮ I20 = (λmax − λmin)e2∠umax,

where umax - eigenvector associated with the largest eigenvalue λmax.

◮ I11 defines contrast range of the orientation estimation

Research problem

We want to estimate orientation of the image based on ST: ST How to tune ST to create ’best’ orientation image?

Implementation of Structure tensor (ST)

• 1. Convolve image f with the (complex) filter h: fcgrad = h ∗ f

with h(x, y, σ2

in) = (Dx + iDy)g = (Dx + iDy)

1 2πσ2 e− x2+y2

2σ2 ,

where σ2

in fixates the inner-scale (the frequency-contents)

resulting in an image with complex valued pixels, fcgrad

Implementation of Structure tensor (ST)

• 1. Convolve image f with the (complex) filter h: fcgrad = h ∗ f

with h(x, y, σ2

in) = (Dx + iDy)g = (Dx + iDy)

1 2πσ2 e− x2+y2

2σ2 ,

where σ2

in fixates the inner-scale (the frequency-contents)

resulting in an image with complex valued pixels, fcgrad

• 2. Apply pointwise (complex) squaring to obtain infinitesimal

linear symmetry fils(x, y) = f2

cgrad

Implementation of Structure tensor (ST)

• 1. Convolve image f with the (complex) filter h: fcgrad = h ∗ f

with h(x, y, σ2

in) = (Dx + iDy)g = (Dx + iDy)

1 2πσ2 e− x2+y2

2σ2 ,

where σ2

in fixates the inner-scale (the frequency-contents)

resulting in an image with complex valued pixels, fcgrad

• 2. Apply pointwise (complex) squaring to obtain infinitesimal

linear symmetry fils(x, y) = f2

cgrad

• 3. Convolve fils and |fils| with a Gaussian filter g(x, y, σ2
• ut)

defining the neighbourhood (the outer-scale)

Implementation of Structure tensor (ST)

• 1. Convolve image f with the (complex) filter h: fcgrad = h ∗ f

with h(x, y, σ2

in) = (Dx + iDy)g = (Dx + iDy)

1 2πσ2 e− x2+y2

2σ2 ,

where σ2

in fixates the inner-scale (the frequency-contents)

resulting in an image with complex valued pixels, fcgrad

• 2. Apply pointwise (complex) squaring to obtain infinitesimal

linear symmetry fils(x, y) = f2

cgrad

• 3. Convolve fils and |fils| with a Gaussian filter g(x, y, σ2
• ut)

defining the neighbourhood (the outer-scale)

Implementation of Structure tensor (ST)

• 1. Convolve image f with the (complex) filter h: fcgrad = h ∗ f

with h(x, y, σ2

in) = (Dx + iDy)g = (Dx + iDy)

1 2πσ2 e− x2+y2

2σ2 ,

where σ2

in fixates the inner-scale (the frequency-contents)

resulting in an image with complex valued pixels, fcgrad

• 2. Apply pointwise (complex) squaring to obtain infinitesimal

linear symmetry fils(x, y) = f2

cgrad

• 3. Convolve fils and |fils| with a Gaussian filter g(x, y, σ2
• ut)

defining the neighbourhood (the outer-scale) (σin, σout) are important parameters of ST defining its scale

Finding the Frequency Content of a Signal

• I. σin :

ω0 = 2π

T0

y = Asin(ω0r)

Finding the Frequency Content of a Signal

• I. σin :

ω0 = 2π

T0

y = Asin(ω0r) period T0 ∈ [Tmin, Tmax]

Finding the Frequency Content of a Signal

• I. σin :

ω0 = 2π

T0

y = Asin(ω0r) period T0 ∈ [Tmin, Tmax] amplitude A

Finding the Frequency Content of a Signal

• I. σin :

ω0 = 2π

T0

y = Asin(ω0r) period T0 ∈ [Tmin, Tmax] amplitude A Linear Shift-Invariant System

Finding the Frequency Content of a Signal

• I. σin :

ω0 = 2π

T0

y = Asin(ω0r) period T0 ∈ [Tmin, Tmax] amplitude A Linear Shift-Invariant System y′ = Bsin(ω0r + φ0)

Finding the Frequency Content of a Signal

• I. σin :

ω0 = 2π

T0

y = Asin(ω0r) period T0 ∈ [Tmin, Tmax] amplitude A Linear Shift-Invariant System y′ = Bsin(ω0r + φ0)

Finding the Frequency Content of a Signal

• I. σin :

ω0 = 2π

T0

y = Asin(ω0r) period T0 ∈ [Tmin, Tmax] amplitude A Linear Shift-Invariant System y′ = Bsin(ω0r + φ0) II. (I.)2

Finding the Frequency Content of a Signal

• I. σin :

ω0 = 2π

T0

y = Asin(ω0r) period T0 ∈ [Tmin, Tmax] amplitude A Linear Shift-Invariant System y′ = Bsin(ω0r + φ0) II. (I.)2

Finding the Frequency Content of a Signal

• I. σin :

ω0 = 2π

T0

y = Asin(ω0r) period T0 ∈ [Tmin, Tmax] amplitude A Linear Shift-Invariant System y′ = Bsin(ω0r + φ0) II. (I.)2 ∗ Convolution Gaussian, σout

Frequency in Fourier Domain

ωo σ−1

Frequency in Fourier Domain

ω2

• σ−1

Estimation of parameters σin, σout

σin - fixates inner-scale, i.e. the frequency contents

σin = 0.9 σin = 2.5

Estimation of parameters σin, σout, cont.

σout = 1.57σin - fixates outer-scale, i.e. integration

σout = 1.5 σout = 3

Updates on Structure tensor (ST)

◮ Estimation of the image information from ST

◮ log(I11) = log |2−1Aω2

0| − ω2 0σ2 in

ω0 is estimated by fitting a line to observations of log(I11) versus σ2

in (by which log(I11) is computed).

◮ Estimate parameters of the ST automatically from the image

◮ σin =

1 ω0 , where ω0 - local frequency of the ridge pattern

◮ σout = 1.57σin

Updates on Structure tensor (ST)

◮ Estimation of the image information from ST

◮ log(I11) = log |2−1Aω2

0| − ω2 0σ2 in

ω0 is estimated by fitting a line to observations of log(I11) versus σ2

in (by which log(I11) is computed).

◮ Estimate parameters of the ST automatically from the image

◮ σin =

1 ω0 , where ω0 - local frequency of the ridge pattern

◮ σout = 1.57σin

Orientation Estimation Procedure

Dense frequency estimation ⇒ Dense orientation estimation ⇒ Image enhancement ⇒ Dense frequency estimation ⇒ · · ·

Repeat until convergence in (global) mean frequency is

• achieved. Retain the dense images in the last cycle, i.e.

enhanced image, orientation map and frequency map and use them for ridge counting.

Orientation Estimation Procedure

Dense frequency estimation ⇒ Dense orientation estimation ⇒ Image enhancement ⇒ Dense frequency estimation ⇒ · · ·

Repeat until convergence in (global) mean frequency is

• achieved. Retain the dense images in the last cycle, i.e.

enhanced image, orientation map and frequency map and use them for ridge counting.

Figure: Ground truth image with frequency ω = 2 ∗ π/13 = 0.4833

Frequency estimation ⇒ Orientation estimation ⇒ Image enhancement

Dense frequency estimation (no-orientation is necessary to know)

◮ Frequency estimation is done by fitting orientation to scale

• space. In turn this requires at least computation of a

discrete scale space for log(I11).

◮ This is achieved by computing log(I11) at 3 or 5 different

inner scales σ2

in chosen around a σ2 determined by the

global mean omega (via σ2 = 1/ω2).

◮ For fingermarks 3 inner scales were sufficient. For high

resolution frequency estimation of synthetic images with heavy noise we used 3 scales.

Frequency estimation ⇒ Orientation estimation ⇒ Image enhancement

◮ Compute the average frequency mean om and variance

var om of the frequency over the entire image (or ROI).

◮ Around mean om define 5 frequencies to sample by help of

var om.

◮ The mean om and its variance estimated in the synthetic

test image were (left and right parts): MeanL VarL MeanR VarR 0.4839 0.0000 0.4849 0.0001, Groundtruth is 0.4833

Frequency estimation ⇒ Orientation estimation ⇒ Image enhancement

◮ Use the sigma that corresponds to these frequencies as

inner scales for linear symmetry algorithm and generate 5 different orientation maps

◮ Merge the different orientation maps to a master orientation

map by picking orientation+certainty according to the closest frequency in the omega image.

Frequency estimation ⇒ Orientation estimation ⇒ Image enhancement

For each pixel of the original image we know (or ROI), we generate dynamically a Gabor filter, matching exactly the latest values in corresponding pixels in the frequency map and

• rientation map of the latest iteration.

Iteration

Frequency estimation ⇒ Orientation estimation ⇒ Image enhancement

Master orientation and master frequency maps are obtained as those in the final cycle, when mean global frequency has

• converged. Typically it requires 2-4 cycles for convergence to
• ccur for SD27 fingermarks.

Orientation Estimation

Frequency (ω) Estima- tion ω-converge Output: I20 (∈ [0, 1])

Automatic extraction of ridge distance between minutia

Goal: pressure invariant/resilient (minutia) constellation features, allowing to replace/complement distance measurements between minutia in matchers.

Automatic extraction of ridge distance between minutia

Goal: pressure invariant/resilient (minutia) constellation features, allowing to replace/complement distance measurements between minutia in matchers.

ω k

α

ˆ ω =

ω cos α,

α is the slope angle of the ridges

Automatic extraction of ridge distance between minutia

Goal: pressure invariant/resilient (minutia) constellation features, allowing to replace/complement distance measurements between minutia in matchers.

ˆ ωˆ

• k

ω k

α

NumOfRidges= L

ˆ T

where the average period is ˆ T = 2π

ˆ ω ,

and the distance between minutia is L. ˆ ω =

ω cos α,

α is the slope angle of the ridges

NumOfRidges =

1 2π

• L ˆ

ω(x, y)dL

Currently, we are incorporating local orientation estimation quality y into dL.

Ridge count

For the proposed method(Ra) and comparing method Rb1 |ǫrc| 1 2 3 4 5 RA(|ǫrc|) 0.43 0.78 0.90 0.95 0.97 0.98 RB(|ǫrc|) 0.37 0.71 0.84 0.91 0.94 0.96 RA′(|ǫrc|) 0.51 0.83 0.92 0.96 0.97 0.98 RB′(|ǫrc|) 0.43 0.77 0.88 0.93 0.94 0.96

Figure: Comparative estimation in ridge count (within 0,1,2 ridge error) between fingermark and fingerprint, on paths between minutia pairs

1Lindeberg T., Feature detection with automatic scale selection, 1998

Examples of ridge count results

Correct and erroneous ridge count estimation examples:

1 2 5 7 8

Results

◮ Auto-enhancement of the orientation map (by scale

adaptive linear symmetry algorithm)

◮ Semi-automatic matching software. This will allow to

extend/correct the automatically obtained orientation map by human experts using dense colour based maps.

Results

◮ Auto-enhancement of the orientation map (by scale

adaptive linear symmetry algorithm)

◮ Semi-automatic matching software. This will allow to

extend/correct the automatically obtained orientation map by human experts using dense colour based maps.

◮ The feature vector containing the extended features

◮ 78% ridge count estimation agreement (within 1 ridge error

• n +70’000 minutia pairs) between fingermark and tenprint

using all possible minutia pairs

◮ 83% ridge count estimation agreement (within 1 ridge error

• n +6’000 minutia pairs) between fingermark and tenprint

WHEN pairs with edges passing by other minutia are removed from statistics