[PPT] - Statistical Models of Images, with Application to Denoising and PowerPoint Presentation

SLIDE 1

Statistical Models of Images, with Application to Denoising and Texture Synthesis

Eero Simoncelli Center for Neural Science, and Courant Institute of Mathematical Sciences New York University http://www.cns.nyu.edu/∼eero

SLIDE 2

Statistical Image Models

Applications in Image Processing / Graphics:

Compression
Restoration
Enhancement
Synthesis

Theoretical Neurobiology:

Ecological optimality principle for early visual processing
Adaptation / plasticity / learning

SIAM, 3/02 1

SLIDE 3

Model I: Gaussian

10 10

1

10

2

10

3

10 10

1

10

2

10

3

10

4

10

5

10

6

Spatial−frequency (cycles/image) Power

Power spectra of natural images fall as 1/f α, α ∼ 2.

[Field ’87, Ruderman & Bialek ’94, etc]

SIAM, 3/02 2

SLIDE 4

Bandpass Filters Reveal non-Gaussian Behaviors

500 500 10

4

10

2

10

Filter Response Probability

Response histogram Gaussian density

Marginal densities of bandpass filtered images are non-Gaussian.

[Field87, Mallat89].

SIAM, 3/02 3

SLIDE 5

Optimizing non-Gaussianity

Linear operators with maximally independent (or maximally non-Gaussian) re- sponses are oriented bandpass filters

[Bell/Sejnowski ’97; Olshausen/Field ’96]

SIAM, 3/02 4

SLIDE 6

Sample Kurtosis vs. Filter Bandwidth

0.5 1 1.5 2 2.5 3 4 6 8 10 12 14 16 Filter Bandwidth (octaves) Sample Kurtosis

For most images, maximum is near one octave [after Field, 1987].

SIAM, 3/02 5

SLIDE 7

Separable Wavelets

Basis functions are bandpass filters, related by translation, dilation, modu-

lation.

Orthogonal.
Lacking translation- and rotation-invariance.

SIAM, 3/02 6

SLIDE 8

Steerable Pyramid

Basis functions are oriented bandpass filters, related by translation, dilation,

rotation (directional derivatives, order K−1).

Tight frame, 4K/3 overcompleteness for K orientations.
Translation-invariant, rotation-invariant.

[Freeman & Adelson, ’90; Simoncelli et.al., ’91; Freeman & Simoncelli ’95]

SIAM, 3/02 7

SLIDE 9

Steerable Pyramid: Block Diagram

L(-ω) L0(-ω) H0(-ω) BK-1(-ω) 2↑ 2↓ L0(ω) H0(ω) B1(-ω) B0(-ω) B1(ω) B0(ω) L(ω) BK-1(ω)

Lowpass band is recursively split using central diagram (gray box).

SIAM, 3/02 8

SLIDE 10

Model II: Wavelet Marginals

Boats Lena Toys Goldhill

−100 −50 50 100 −100 −50 50 100 −100 −50 50 100 −100 −50 50 100

p = 0.62 p = 0.56 p = 0.52 p = 0.60 ∆H = 0.014 ∆H = 0.013 ∆H = 0.021 ∆H = 0.0019

Coefficient densities well fit by generalized Gaussian

[Mallat ’89; Simoncelli/Adelson ’96]:

f(c) ∝ e−|c/s|p, p ∈ [0.5,0.8].

Non-Gaussianity due to both image content and choice of basis.

SIAM, 3/02 9

SLIDE 11

Coefficient Dependency

Large-magnitude subband coefficients are found at neighboring positions, ori- entations, and scales.

SIAM, 3/02 10

SLIDE 12

Wavelet Conditional Histogram

40

40

50

0.2 0.6 1

40

40 0.2 0.6 1

40

40

40

40

Conditional mean is zero
But, conditional variance grows with amplitude of L2

SIAM, 3/02 11

SLIDE 13

Conditional Histograms

Strength of dependency is different for each pair of filters: But the form of dependency is highly consistent across a wide range of images.

SIAM, 3/02 12

SLIDE 14

Model III: Local GSM model

Model generalized neighborhood of coefficients as a Gaussian Scale Mixture (GSM) [Andrews & Mallows ’74]:

x = √z

u, where

z and

u are independent

x|z is Gaussian, with covariance zCu
marginals are always leptokurtotic
we choose a flat (non-informative) prior on

log(z)

[Wainwright & Simoncelli, ’99]

SIAM, 3/02 13

SLIDE 15

GSM Simulations

image data

−500 500 −500 500 −500 500 −500 500 −500 500 −500 500 −500 500 −500 500

model sim

−500 500 −500 500 −500 500 −500 500 −500 500 −500 500 −500 500 −500 500

Conditional Histograms of pairs of coefficients with different spatial separa- tions.

SIAM, 3/02 14

SLIDE 16

Denoising I (Gaussian model)

y = x+w, where w is Gaussian, white. y is an observed transform coefficient. Bayes least squares solution: IE(x|y) = σ2

x

σ2

x +σ2 w

y

SIAM, 3/02 15

SLIDE 17

Denoising II (marginal model)

p = 0.5 p = 1.0 p = 2.0 Bayes least squares solution: IE(x|y) =

dx P(y|x) P(x) x dx P(y|x) P(x)

No closed-form expression with generalized Gaussian prior, but numerical com- putation is reasonably efficient.

[Simoncelli & Adelson, ’96]

SIAM, 3/02 16

SLIDE 18

Denoising III (GSM model)

IE(x| y) =

dz P(z|

y) IE(x| y,z) =

dz P(z|

y)

zCu(zCu +Cw)−1

y

ctr

where

P(z|

y) =

P(

y|z) P(z)

dz P(

y|z) P(z),

P(

y|z) = exp(− yT(zCu +Cw)−1 y/2)

(2π)N|zCu +Cw|

Numerical computation of solution is reasonably efficient if one jointly diago- nalizes Cu and Cw ...

[Portilla et.al., ’01]

SIAM, 3/02 17

SLIDE 19

Denoising Simulation: Face

noisy (4.8) I-linear (10.61) II-marginal (11.98) III-GSM nbd: 5×5+ p (13.60)

Semi-blind (all parameters estimated except for σw).
All methods use same steerable pyramid decomposition.
SNR (in dB) shown in parentheses.

SIAM, 3/02 18

SLIDE 20

Denoising Simulation: Fingerprint

riginal

soft thresholding (17.5) noisy (8.1) GSM (21.2)

PSNR shown in parentheses.
Both methods use same steerable pyramid decomposition.
Joint statistics capture oriented structures.

SIAM, 3/02 19

SLIDE 21

Denoising Comparison

5 10 15 20 25 30 35 40 45 50 2 4 6 8 10 12 14 16

PSNR improvement as a function of noise level, averaged over three images:

squares: GSM
triangles: MatLab wiener2, optimized neighborhood [Lee, ’80]
circles: soft thresholding, optimized threshold [Donoho, ’95]

SIAM, 3/02 20

SLIDE 22

Example Texture Types

structured random periodic 2nd-order Can we derive a statistical model (and sampling technique) to represent all of these?

SIAM, 3/02 21

SLIDE 23

Synthesis: Gaussian model

Captures periodicity.

SIAM, 3/02 22

SLIDE 24

Synthesis: Wavelet marginal model

Captures some local structure.

[Heeger & Bergen, ’95]

SIAM, 3/02 23

SLIDE 25

Synthesis: GSM model

[Portilla & Simoncelli, ’00]

SIAM, 3/02 24

SLIDE 26

Credits

Local Gaussian Scale mixtures: Martin Wainwright (MIT) Global Tree Model: Martin Wainwright & Allan Willsky (MIT) Denoising: Javier Portilla (U. Granada), Vasily Strela (Drexel U.), Martin Wainwright (MIT) Texture Analysis/Synthesis: Javier Portilla (U. Granada) Compression: Robert Buccigrossi (U Pennsylvania) Funding provided by the National Science Foundation, the Alfred P. Sloan Foundation, and the Howard Hughes Medical Institute.

SIAM, 3/02 25