Tone Reproduction Tone Reproduction Tone Reproduction Erik - - PowerPoint PPT Presentation

Tone Reproduction Tone Reproduction Tone Reproduction

Erik Reinhard

University of Central Florida School of Computer Science reinhard@cs.ucf.edu

Preparing Images for Display Preparing Images for Display

• Need to map floating point numbers to

Need to map floating point numbers to bytes bytes

• Need to preserve some aspect of the image:

Need to preserve some aspect of the image:

– – Contrast Contrast – – Visibility Visibility – – Brightness Brightness – – … …

Global vs. Local Global vs. Local

• Global

Global

– – Scale each pixel according to a fixed curve Scale each pixel according to a fixed curve – – Key issue: shape of curve Key issue: shape of curve

• Local

Local

– – Scale each pixel by a curve that is modulated by Scale each pixel by a curve that is modulated by a local average a local average – – Key issue: size of local neighborhood Key issue: size of local neighborhood

Global Operators Global Operators

Ward Tumblin Ferwerda

Global Operators Global Operators

Tumblin Ward Ferwerda

Local Operator Local Operator

Pattanaik

Spatial Processing Spatial Processing

• Circularly symmetric

Circularly symmetric receptive fields receptive fields

• Center

Center-

• surround mechanisms

surround mechanisms

– – Laplacian Laplacian of Gaussian

• f Gaussian

– – Difference of Gaussians Difference of Gaussians – – Blommaert Blommaert

• Scale space model

Scale space model

Scale Space Scale Space (Histogram Equalized Images) (Histogram Equalized Images)

Tone Reproduction Idea Tone Reproduction Idea

• Modify existing global

Modify existing global

• perator to be a local
• perator to be a local
• perator, e.g. Greg Ward’s
• perator, e.g. Greg Ward’s
• Use spatial processing to

Use spatial processing to determine a local determine a local adaptation level for each adaptation level for each pixel pixel

5 . 2 4 . 4 . max max 5 . 2 4 . 4 . max max

) , ( 219 . 1 2 219 . 1 ) , ( 219 . 1 2 219 . 1 ) , ( ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ + ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ + = ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ + ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ + = y x L L L y x L L L L L y x L L

a d d

• utput

w d d

• utput

Blommaert Blommaert Brightness Model Brightness Model

2 2 2

2 2

1

s k r i i

i

e s k R

−

= π

i i

R v u L s y x V ⊗ = ) , ( ) , , (

) , , ( 2 ) , , ( ) , , ( ) ( ) , , (

1 2 2 1

s y x V s s y x V s y x V s W s y x V + − =

ϕ

∑

=

n

s s

s y x V y x B ) , , ( ) , (

Gaussian filter Neural response Center/ surround Brightness

Brightness Brightness

∑

=

n

s s

s y x V y x B ) , , ( ) , (

Scale Selection Alternatives Scale Selection Alternatives

How large should a local neighborhood be?

)) , ( , , ( ) , ( s s y x s y x V y x B

n m

− =

Mean value

ε < )) , ( , , ( : y x s y x V s

m m

Thresholded

Mean Value Mean Value

)) , ( , , ( ) , ( s s y x s y x V y x B

n m

− =

Thresholded Thresholded

ε < ) , , ( :

m m

s y x V s

Tone Tone-

• Mapping

Mapping

Local adaptation

5 . 2 4 . 4 . max max 1

) , ( 219 . 1 2 219 . 1 ) , ( )) , ( , , ( ) , ( ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ + ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ + = = y x L L L y x L L y x s y x V y x L

a d d

• utput

m a

Greg Ward’s tone- mapping with local adaptation

Results Results

• Good results, but something odd about scale

Good results, but something odd about scale selection: selection:

• For most pixels, a large scale was selected

For most pixels, a large scale was selected

• Implication: a simpler algorithm should be

Implication: a simpler algorithm should be possible possible

Simplify Algorithm Simplify Algorithm

) , ( ) , ( ) , ( 1 ) , ( ) , ( 219 . 1 2 219 . 1 ) , (

5 . 2 4 . 4 . max max

y x L L a y x L y x L y x L L y x L L L y x L L

w w

• utput

a d d

• utput

= + ≈ ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ + ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ + =

Greg Ward’s tone- mapping with local adaptation Simplify Fix overall lightness

• f image

Global Operator Results Global Operator Results

Our method Ward

Global Operator Results Global Operator Results

Ward Our method

Global Global Local Local

)) , ( , , ( 1 ) , ( ) , ( 1 ) , (

1

y x s y x V y x L L y x L y x L L

m

• utput
• utput

+ = + =

Global operator Local operator

Local Operator Results Local Operator Results

Global Local

Local Operator Results Local Operator Results

Global Local Pattanaik

Lesson Learned Lesson Learned

• A good local average is obtained by using a

A good local average is obtained by using a (Gaussian) blurred pixel. (Gaussian) blurred pixel.

• The amount of blur to apply varies per pixel

The amount of blur to apply varies per pixel

• The filter kernel should as large as possible,

The filter kernel should as large as possible, without crossing sharp contrast boundaries without crossing sharp contrast boundaries

• This is why the scale selection mechanism

This is why the scale selection mechanism works! works!

Global Operators Global Operators

• Usually a sigmoid:

Usually a sigmoid:

• Sigma is the semi

Sigma is the semi-

• saturation constant

saturation constant

• Most global operators can be rewritten into

Most global operators can be rewritten into a form similar to this a form similar to this

n a n w n w d

L L L L ) ( σ + =

Global Operators Global Operators -

• Sigmoid

Sigmoid

Drago

La La

• Adaptation luminance computed as

Adaptation luminance computed as

– – Global average Global average

• Log average luminance

Log average luminance

• Interpolation scheme

Interpolation scheme

– – Local average Local average

• Use four neighbors (good enough in practice)

Use four neighbors (good enough in practice)

• Scale selection mechanism

Scale selection mechanism

• Bilateral filter

Bilateral filter

• Typically small filter kernels

Typically small filter kernels

Photoreceptor Photoreceptor-

• based

based

n a a pixel w a a w w d a w w d a w w d

fL L L a aL L L B B B L G G G L R R R = − + = + = + = + = ) ( ) 1 ( ) ( ) ( ) ( σ σ σ σ

n a n w n w d

L L L L ) ( σ + =

Also interpolate between R,G,B values and pixel luminance

Interpolation Scheme for La Interpolation Scheme for La

Luminance interpolation

Interpolation Scheme for La Interpolation Scheme for La

Between R,G,B and luminance

Bilateral filter Bilateral filter

Local La Local La

Local bilateral Global Local SUSAN

Conclusions Conclusions

• Most tone reproduction operators are

Most tone reproduction operators are sigmoidal sigmoidal

• Can be either local or global

Can be either local or global

• Filter kernels (local) should be small, but in

Filter kernels (local) should be small, but in any case should not cross sharp contrast any case should not cross sharp contrast boundaries boundaries

Conclusions Conclusions

• Sigmoids

Sigmoids also part of most color appearance also part of most color appearance models models

• Color appearance and tone reproduction will

Color appearance and tone reproduction will converge converge