Histogram-based methods Nicolas ROUGON ARTEMIS Department - - PowerPoint PPT Presentation

Institut Mines-Télécom

Histogram-based methods

Nicolas ROUGON

ARTEMIS Department

Nicolas.Rougon@telecom-sudparis.eu

IMA 4103

Extraction d’Information Multimédia

Institut Mines-Télécom

Overview

IMA 4103 - Nicolas ROUGON

■ Thresholding & Noise ■ Histogram-based methods

• Histogram-based segmentation
• Histogram-based image enhancement

Institut Mines-Télécom

Problem statement

■ For simplicity, we deal with gray level still images L

• Defined over the digital grid Ω  Zn

denoting |Ω| the # of pixels

• Encoded on b bits

L(x)  Λ = {0..2b-1} (x  Ω)

IMA 4103 - Nicolas ROUGON

■ The presented notions readily generalize to

• Multichannel / multi-view images
• Image sequences

■ We hereafter investigate the possibility of segmenting images

based on color features only

Institut Mines-Télécom

Thresholding & Noise

IMA 4103 - Nicolas ROUGON

Institut Mines-Télécom

IMA 4103 - Nicolas ROUGON

Thresholding

Tλ : upper-threshold operator at level λ

    

• therwise

λ

λ

T

) ( ) ( ) )( ( x x x L L L if

Tλ : lower-threshold operator at level λ

    

• therwise

λ if Tλ ) ( ) ( ) )( ( x x x L L L

■ Image thresholding

T100(L) T100(L) L

• Thresholding is the basic operator for retaining

pixels in a luminance range

Institut Mines-Télécom

IMA 4103 - Nicolas ROUGON

Binarization

B λ : upper-binarization operator at level λ

    

• therwise

λ 1 Bλ ) ( ) )( ( x x L L if

B λ : lower-binarization operator at level λ

    

• therwise

λ if 1 Bλ ) ( ) )( ( x x L L

■ Image binarization

B100(L) B100(L) L

• Whereas threshold operators output images,

image binarization yields sets

Institut Mines-Télécom

IMA 4103 - Nicolas ROUGON

Level sets

■ Image level sets

L λ : image level set at level λ

 

λ Ω

λ

   ) (x x L L

• The collection (Lλ) λΛ of image level sets

defines the image topographic map

• It provides an algebraic image decomposition



Λ λ λ 

 L L

• This allows for region-based geometric image

processing ► mathematical morphology

Institut Mines-Télécom

IMA 4103 - Nicolas ROUGON

Level lines

■ Image level lines

L λ : image level lines at level λ

 

λ Ω

λ

   ) (x x L L

• Ω  Rn : level lines comprise

 constant luminance regions (plateau) ► non generic (noise)  non-intersecting closed (except along ∂Ω) codimension-1 surfaces L 47

• Ω  Zn : connectivity does not hold

 sub-pixel image interpolation is required

Institut Mines-Télécom

IMA 4103 - Nicolas ROUGON

Level lines

■ Image level lines

L λ : image level lines at level λ

 

λ Ω

λ

   ) (x x L L

• Level lines provide a geometric image

decomposition

 Description of local image geometry  Basic representation for contour-based geometric image processing

Institut Mines-Télécom

Intensity-based image segmentation

IMA 4103 - Nicolas ROUGON

■ Basic definition

Intensity-based image segmentation consists of estimating a family

• f thresholds λ1 < µ1  ... λi < µi  ...  λN < µN allowing to partition

the luminance space Λ into non-overlapping intervals [λi, µi] defining the segmentation regions Ri

 

1 ) )( T B Ω

i i

μ λ

   x x L Ri (



N i i

R R

 



1

/ Ω

• More generally, segmentation regions can be defined as unions
• f intensity ranges [λi, µi]

Institut Mines-Télécom

IMA 4103 - Nicolas ROUGON

Intensity-based image segmentation

■ In practice

• Natural images are always noisy
• Natural objects are generally textured

Accurately segmenting natural images based on intensity / color features only is rarely possible

• Sensor and scene (object & source) properties induce variability

in object appearance

 low contrast  non-uniform lighting  cast shadows  multiple reflections  transparency  …

Institut Mines-Télécom

IMA 4103 - Nicolas ROUGON

Intensity-based image segmentation

■ In practice

 Restoration (denoising, deblurring…)

• Preprocessing is often necessary to reduce appearance variability

and improve signal quality

• Statistics and machine learning provide tools for consistently

modeling/dealing with intensityvariability

 Hard assignment to disjoint intervals is replaced by probabilistic /statistical assignment to overlapping clusters

► This framework allows for intensity mixtures

 Photometric calibration  Contrast-enhancement

► This statement applies to any segmentation approach

Institut Mines-Télécom

IMA 4103 - Nicolas ROUGON

Intensity-based image segmentation

■ Performance and limitations

• Integrating spatial / spatiotemporal context information is usually

necessary to disambiguate local intensity information ► Improved accuracy / robustness

• Intensity-based segmentation

approaches are mostly applicable to simple images

► Cartoon model

• Nonetheless, intensity-based approaches are often relevant for

efficiently deriving a coarse segmentation (pre-segmentation) ► Meaningful for real-time applications e.g. object tracking in live video sequences

Institut Mines-Télécom

Image noise

IMA 4103 - Nicolas ROUGON

• Speckle noise

► US ► OCT (interferometric imaging) ► SAR ► PC microscopy ► PC X-Ray

• Shot noise

► X-Ray (photon counting imaging) ► TEP / SPECT

• Impulse noise

(salt-and-pepper)

Multiplicative L = L0 n Complex models Additive

L = L0 + n

• Optical imaging
• Thermal imaging

► IR

• Quantization noise

■ Noise models

Institut Mines-Télécom

Image noise

IMA 4103 - Nicolas ROUGON

• Speckle noise

► US ► OCT (interferometric imaging) ► SAR ► PC microscopy ► PC X-Ray

• Shot noise

► X-Ray (photon counting imaging) ► TEP / SPECT

Heavy-tailed

Laplace | negative- exponential | α-stable

Poisson Gaussian

• Optical imaging
• Thermal imaging

► IR

■ Noise distributions

• Salt-and-pepper noise

Impulse

• Quantization noise

Uniform

• Thermal noise

► MRI

Rician | Rayleigh

Institut Mines-Télécom

Image noise

IMA 4103 - Nicolas ROUGON

■ Noise distributions

Optical Infrared

• Gaussian
• Rician |Rayleigh

MRI

Institut Mines-Télécom

Image noise

IMA 4103 - Nicolas ROUGON

■ Noise distributions

SAR US OCT LIDAR PC microscopy

• Negative exponential

Sonar PC X-Ray

Institut Mines-Télécom

Image noise

IMA 4103 - Nicolas ROUGON

■ Noise distributions

PET industrial X-Ray SPECT

• Poisson

medical X-Ray Astronomical Hyperspectral

Institut Mines-Télécom

Image noise

IMA 4103 - Nicolas ROUGON

■ Noise distributions

• Impulse

Optical

• Uniform

Color quantization

Institut Mines-Télécom

Image noise

IMA 4103 - Nicolas ROUGON

■ Modeling issues

A consistent noise model for a sensor/ imaging chain is a key point

• For methodology design

Integrating realistic noise priors leads to optimal denoising schemes and robust image analysis estimators

• For performance assessment

Adequate noise generators allows for synthesizing simulated test images with controlled SNR

Institut Mines-Télécom

Generic image segmentation framework

IMA 4103 - Nicolas ROUGON

Preprocessing Segmentation Postprocessing

Filtering & Restoration

Contour-based approaches Region-based approaches

• Denoising
• Deblurring
• Enhancement
• …
• Hole filling
• Split & Merge
• …
• Linking
• Thinning
• …
• Artifacts removal

► Preprocessing can be integrated into segmentation, yielding robust segmentation schemes

Institut Mines-Télécom

Histogram techniques

IMA 4103 - Nicolas ROUGON

Institut Mines-Télécom

Image histogram

IMA 4103 - Nicolas ROUGON

■ Image histogram

The histogram HL of an image L is the array counting the number of gray level occurrences in L k  Λ HL (k ) = # { x  Ω | L(x) = k }

intensity range modes

Institut Mines-Télécom

Image histogram

IMA 4103 - Nicolas ROUGON

■ Image histogram

The histogram HL of an image L is the array counting the number of gray level occurrences in L

• Histograms can be quantified

► Histogram binning

k  Λ HL (k) = # { x  Ω | L(x)  Bk }

• Optimal (uniform) bin size

selection rules

Sturges | Scott | Freedman-Diaconis Knuth | Wand

Institut Mines-Télécom

Image histogram

IMA 4103 - Nicolas ROUGON

• Arbitrary domains Ω

e.g. image region / pixel neighborhood

► Regional / local histogram

• Multichannel / multi-view images

► Multidimensional histogram / co-occurrence matrix

■ Generalization

The previous definition generalizes to

• Arbitrary (quantified) image features

e.g. local orientation / motion

► Feature histogram

Institut Mines-Télécom

Image histogram

IMA 4103 - Nicolas ROUGON

■ Properties

• HL accounts for luminance distribution over Ω

► HL summarizes luminance global / regional / local statistical properties

 

) ( H Ω 1 p k k

L L



 Empirical density estimator

Institut Mines-Télécom

Image histogram

IMA 4103 - Nicolas ROUGON

■ Properties

• Luminance statistics can be efficiently computed from HL



 ) ( p k k L

L



  ) (k L k

L

L

p ) ( σ

2 2



  ) ( p ) ( σ

3 3

• S

k L k L

L

L

3 ) ( p ) ( σ

4 3

• K

  



k L k L

L L



  ) ( p log ) ( p ) ( H

2

k k L

L L





2

) ( p ) ( E k L

L

central moments

 Mean  Variance  Skewness  Kurtosis  Entropy  Energy

texture indices

Institut Mines-Télécom

Image histogram

IMA 4103 - Nicolas ROUGON

■ Qualitative properties

• Skewed / narrow histogram means low contrast

 Skewed right

• verexposed image

 Skewed left underexposed image  Narrow low contrast image

Institut Mines-Télécom

Image histogram

IMA 4103 - Nicolas ROUGON

■ Properties

• HL contains no spatial information ► No geometric information

 Radically different images can have the same histogram See example  Random spatial permutations

• f pixels yield images with the

same histogram

Institut Mines-Télécom

Histogram-based image segmentation

IMA 4103 - Nicolas ROUGON

■ Principle

• Bimodal histograms showing well-separated modes often
• riginate from images containing object(s) / background

with clearly distinct average intensities

 Object / background segregation can be achieved by thresholding

► Thresholds are estimated from image histogram

Institut Mines-Télécom

Histogram-based image segmentation

IMA 4103 - Nicolas ROUGON

■ A statistical approach is often adopted

• Let background (object) pixels be distributed according to a RV Xb (Xo)

with density pb (po)

   

• p

p X Pr X Pr

b b

• k

k 

True ► k  background False ► k  object  Likelihood test  Conditional probabilities are estimated using a prior model fitted on the histogram

• This approach readily extends to multiple object segmentation

 A classical model for a mode is Gaussian density ► Gaussian Mixture Model

Institut Mines-Télécom

IMA 4103 - Nicolas ROUGON

Histogram-based image segmentation

• Probability density estimation: a hard problem

 kernel estimator ► smoothed normalized histogram

kernel

− positive − symmetric − decreasing − unit mass

σ

density

► Fixed bandwidth σ > Parzen estimator

luminance

► Standard kernel : Gaussian

■ Robust density estimation

    x

x d ) (

• Ω

1 p

Ω σ



 L k K k

L

Institut Mines-Télécom

Histogram-based image segmentation

IMA 4103 - Nicolas ROUGON

■ Otsu’s method

• Let background (object) pixels –obtained by lower (upper)- threshold

at some level λ– be distributed according to RVs Xi (λ) (i ϵ {o,b}) with mean values μi (λ) and standard deviations σi (λ)

 Class probabilities are estimated from the histogram as







λ

) ( p λ p

k L b

k ) (







λ

) ( p λ p

k L

• k

) (

 The optimal threshold λ* is defined as the minimizer of the intra-class variance

) (

2

λ

intra

σ ) ( ) ( ) ( ) ( ) (

2 2 2

λ λ p λ λ p λ

intra b b

• σ

σ σ  

Institut Mines-Télécom

Histogram-based image segmentation

IMA 4103 - Nicolas ROUGON

■ Otsu’s method

• Total variance theorem
• This approach can be extended to multi-level thresholding

) (

2

λ

inter

σ ) ( ) (

2 2 2

λ λ

inter intra

σ σ σ  

 

2 2

) ( ) ( ) ( ) ( ) ( λ λ λ p λ p λ

inter b

• b
• μ

μ σ  

 Otsu’s algorithm performs exhaustive search for λ*  λ* is a maximizer of the inter-class variance

Institut Mines-Télécom

Histogram-based image segmentation

IMA 4103 - Nicolas ROUGON

■ Limitations

• Histogram segmentation fails for

 Multimodal histograms with poorly separated modes  Flat histograms

• Histogram modes do not necessarily correspond to objects

(e.g. strong texture) ► Histogram-based segmentation is limited to single object / background separation problems in cartoon-like images

• Nonetheless, due to its statistical relevance, histogram is a

powerful tool for global image enhancement

Institut Mines-Télécom

Point-based image transforms

IMA 4103 - Nicolas ROUGON

■ Point-based image transform

Transform T acting on image L with value at pixel x  Ω depending only on luminance L(x)

x  Ω T(L)(x) = f(L(x))

• Quantification

T(L)(x) = (int) [ f(L(x)) + 0.5]

• Clipping

T(L)(x) = max[ min[ f(L(x)), 2b-1 ], 0 ]

• Point-based image transforms are image histogram transforms

 Linear transforms  Nonlinear transforms

Institut Mines-Télécom

Linear point-based image transforms

IMA 4103 - Nicolas ROUGON

■ T(L)(x) = αL(x) + β

• α < 1 : histogram contraction

 Widely used in medical imaging for visualization purpose ► Windowing

►

12-bit range 16-bit data 8-bit display

window level

Institut Mines-Télécom

Linear point-based image transforms

IMA 4103 - Nicolas ROUGON

 Global contrast enhancement

►

• α > 1 : histogram expansion

■ T(L)(x) = αL(x) + β

Institut Mines-Télécom

Linear point-based image transforms

IMA 4103 - Nicolas ROUGON

►

 Global contrast enhancement The magnitude of enhancement depends on the initial intensity range

■ Histogram stretching

Histogram expansion into full intensity range [0,2b-1]

 

) min( ) ( ) min( ) max( 1 2 ) )( ( TS L L L L L

b

    x x

Institut Mines-Télécom

Nonlinear point-based image transforms

IMA 4103 - Nicolas ROUGON

►

 Contrast enhancement of dark structures

■ Logarithmic histogram stretching

Histogram stretching of the logarithm of L ) )( ) (1 Log ( T ) )( ( T

S LS

x x L L  

• Alternative to histogram stretching for wide-histogram images

 Saturation artifacts can occur

Institut Mines-Télécom

Nonlinear point-based image transforms

■ Cumulated normalized histogram

k  Λ



 



k

) ( H Ω 1 ) ( H

i L L

i k

 

) ( H ) ( Pr k k L

L

  x

 Empirical cumulative distribution function estimator

■ Information-theoretic point-based image transform

• Uniform distributions maximize entropy
• We seek for a point-based transform yielding flat histogram-images

IMA 4103 - Nicolas ROUGON

L

H

L

H

Institut Mines-Télécom

Nonlinear point-based image transforms

■ Cumulated normalized histogram

k  Λ



 



k

) ( H Ω 1 ) ( H

i L L

i k

 

) ( H ) ( Pr k k L

L

  x

 Empirical cumulative distribution function estimator

L

H

 

) ( H ) )( ( T

L U

x x L L 

• induces a point-based transform TU
• TU (L) is uniform over Λ

 

 

 

 

k k k L k L k

L

      ) ( H H ) ( H ) ( Pr ) ( H Pr ) ( H

1

• L
• 1

L ) ( T

L L U

x x

■ Information-theoretic point-based image transform

• Uniform distributions maximize entropy
• We seek for a point-based transform yielding flat histogram-images

Proof

IMA 4103 - Nicolas ROUGON

Institut Mines-Télécom

Nonlinear point-based image transforms

■ Histogram equalization

Stretching of the cumulative normalized histogram ) )( ) ( T ( T ) )( ( T

U S EQ

x x L L 

►

 Global contrast enhancement  Saturation artifacts can occur

IMA 4103 - Nicolas ROUGON

Institut Mines-Télécom

Nonlinear point-based image transforms

■ Histogram equalization

►

• HL is not perfectly flat for quantified images

►

Stretching of the cumulative normalized histogram

) )( ) ( T ( T ) )( ( T

U S EQ

x x L L 

IMA 4103 - Nicolas ROUGON

Institut Mines-Télécom

Nonlinear point-based image transforms

■ Generalization

• We seek for a point-based transform yielding images with some

arbitrary-shaped histogram

• Let Href be a reference histogram built from an image Lref

Assuming the cumulated normalized histogram is invertible, we define a point-based image transform Tref as

ref

H

 

 

) ( H H ) )( ( T

L 1

• ref

ref

x x L L 

• Then:

 

ref T

H H

ref



L

 

           

) ( H ) ( H H H ) ( H H ) ( Pr ) ( H H Pr ) ( H

ref ref 1

• L

ref 1

• L

1

• ref

) ( T

L L ref

k k k L k L k

L

      x x

Proof

IMA 4103 - Nicolas ROUGON

Institut Mines-Télécom

Nonlinear point-based image transforms

■ Histogram specification

) ( ) ( T ) )( ( T

ref Sp

x x L L 

► ► ► ►

• HL is not perfectly equal to Href for quantified images

IMA 4103 - Nicolas ROUGON

Institut Mines-Télécom

Nonlinear point-based image transforms

■ In practice

• is not necessarily strictly increasing

Hence its inverse is not guaranteed to exist

ref

H

• A standard choice for ensuring injectivity is to use the following

definition

 

 

k p p k

p

  ) ( H min H

ref 1

• ref

IMA 4103 - Nicolas ROUGON

Institut Mines-Télécom

Histogram-based methods

Nicolas ROUGON

ARTEMIS Department

Nicolas.Rougon@telecom-sudparis.eu

IMA 4103

Extraction d’Information Multimédia