HDR Image Compression based on HDR Image Compression based on Local - - PowerPoint PPT Presentation

School of Electrical Engineering and Computer Science Kyungpook National Univ.

HDR Image Compression based on HDR Image Compression based on Local Adaptation for Scene and Local Adaptation for Scene and Display Using Retinal Model Display Using Retinal Model

14 14th

th Color Imaging Conference

Color Imaging Conference,

, Lijie Lijie Wang, Takahiko Wang, Takahiko Horiuchi Horiuchi, and , and Hiroacki Hiroacki Kotera Kotera

Presented by Tae Presented by Tae Hyoung Hyoung Lee Lee

2 / 20

Simple static local adaptation method for HDR

image compression

– Recreation of the same sensations between the real scene and the compressed images on displays at steady state local adaptation, respectively – Scene adaptation based of retinal model – Use of bilateral filter for preserving details without banding artifacts

Abstract Abstract

3 / 20

Introduction Introduction

Dynamic range

– HVS : 14-order magnitude – HDR images : 6-order magnitude – LDR display : 2 or 3-order magnitude

Tone mapping

– Recreation of HDR image to LDR images

Perception of HDR of scene luminance by HVS

– Adaptation

4 / 20

Tone mapping

– Global model

• Simple and efficient by using single spatially-invariant

curve

• Problem in local contrast

– Local model

• Preservation of local visual contrast by using spatially

variant operation

• Artifacts around high contrast edges

5 / 20

Proposed method

– HDR image compression based on a retinal model

• Prediction of the response of eyes at any given

adaptation level

– Aim

• Recreation of the same sensations between the real

scene and its range compressed image on the displays at steady state local adaptation, respectively

– Computation of adaptations for both the scene and displays

– Use of bilateral filter

• Suppression of the banding artifacts

6 / 20

Retinex

– Preservation of local contrast and details – Banding artifacts around high contrast edges

MSR

– Removal of banding artifacts with good color appearance by using 3-7 SSR images

Related work Related work

7 / 20

Model of retinal cell

– S-shaped response – Role of

• Determination by adaptation

to the overall scene intensity

• Small value with dark scene

– Perception of glares at high light environment

• Large value with light scene

Retinal Model Retinal Model

n n n

I I I R σ + = ) (

Where, is the light intensity, is I value that causes the half-maximum response, and is a sensitivity control parameter as 0.73(0.7-2.0). σ I n

σ

• Fig. 1. The response of retina at adaptation

level for overall luminance. Dramatic compression at high and dark shadow.

8 / 20

Objective

– Match between viewed scene appearance and display luminance

Tone reproduction framework Tone reproduction framework

• Fig. 2. Tone reproduction framework of Pattanaik et al.

n d n d n d d display n w n w n w w scene

y x y x I y x I y x R R y x y x I y x I y x R R ) , ( ) , ( ) , ( ) , ( ) , ( ) , ( ) , ( ) , ( σ σ + = = + = =

blk

R I R Q − = ) (

Where, is the reference black response

blk

R (Scene luminance) (Display luminance) Appearance vector

w

I

d

I

9 / 20

– Assumption (difference between Pattanaik’ model)

[1]Display appearance is considered to be equivalent to scene appearance as shown in Figure 3 [2]Both scene and display adaptations are local [3]Display adaptation is formulated in relation to scene adaptation by retinal model [4]Bilateral filter is applied to mimic the edge preserving of HVS to reduce the banding artifacts

• Fig. 3. Tone reproduction framework of proposed model.

10 / 20

– Objective

• Recreation of the same sensations between the world

scene and the display, respectively

• Proposed model

– Preservation of contrast and omission of n ) , ( ) , ( y x R y x R

d w

= ) , ( ) , ( ) , ( ) , ( y x y x I y x y x I

w w d d

σ σ =

Final value Final value Unknown values Unknown values

11 / 20

Computation of display adaptation

– Reason of use for display

• The same sensation of HVS after own local adaptation

– Use of local adaptation process

• Characteristic of HVS

– Different adaptation process depending on the real world and images on displays

• Relation with surround background

Display adaptation Display adaptation

• Fig. 4. The simultaneous contrast.

(a) (b)

12 / 20

– Determination of

• Use of the retinal response model

– Input as the scene adaptation – Output as the display adaptation

• Match of the display adaptation level with similar narrow

range

– Expression of the very dark and light intensities in real world

• Monotonic curve

– Corresponding of Light scene adaptation to light display adaptation – Corresponding of dark scene adaptation to dark display adaptation α σ σ σ + = ) , ( ) , ( ) , ( y x y x y x

w w d

) , ( y x

d

σ

) , ( ) , ( ) , ( ) , ( y x y x I y x y x I

w w d d

σ σ =

Known value by using Gaussian filter Known value by using Gaussian filter

Scene adaptation half point level

13 / 20

– Determination of display luminance

α σ + = ) , ( ) , ( ) , ( y x y x I y x I

w w d

Decision of these values are key point. Decision of these values are key point.

14 / 20

Computation of display adaptation

– Determination of

• Surround image in Retinex with luminance channel

– Kotera et al. based on Center/Surround method » Banding artifact by using single Gaussian filter

Scene adaptation Scene adaptation

) , ( y x

w

σ

{ }∫ ∫

= + − = ⊗ = 1 ) , ( . / ) ( exp ) , ( ) , ( ) , ( ) , , (

2 2

dxdy y x G y x K y x G y x Y y x G y x S

m m m m m m

σ σ

15 / 20

– Substitution of Gaussian filter by bilateral filter

2

2 ) ( ) ( ) (

) ( ) ( ) ( ) ( ) ( ) ( ) ( 1 ) (

g s p

I I s p s heigh p s p s heigh p p s p w

e I I g I I g s p f s k I I I g s p f s k s

σ

σ

− − ∈ ∈

= − − − = − − =

∑ ∑

Standard Gaussian filter Standard Gaussian filter Decrease of weight of pixels with large luminance differences

• ver center pixel Is

=>avoiding banding or haloring Decrease of weight of pixels with large luminance differences

• ver center pixel Is

=>avoiding banding or haloring

16 / 20

Reproduction of RGB image

– Resulting images with different

Experiment Experiment

) , ( ) , ( ) , ( ) , ( y x I y x I y x RGB y x RGB

d w w d

⋅ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ =

γ

Where, is a gamma parameter (0.5 to 1.0) γ

γ

• Fig. 4. The results of Memorial Church for

. 1 , 8 . , 5 . = γ

17 / 20

• Fig. 7. The result of cathedralby proposed method

Set of

– with larger mean value than 0.5 – with smaller mean value than 0.5

α

1 = α 1 . = α

• Fig. 6. The result of groveC by proposed method

Problem with dark area

18 / 20

Improvement of details in dark shadows

– Gain parameter

• Improvement of details and preservation of the light

regions

– As increase of , c(x,y) tends to be 1

w

σ

• Fig. 8. The result of rosette proposed method. Better performance in dark area for (b).

) , ( ) , ( ) , ( ) , ( y x c y x y x I y x I

w w d

⋅ + = α σ ) ) ) , ( ( exp( 1 ) , (

2

b y x a y x c

w

σ − ⋅ + =

Where, is an interactive constant by (0,0.1), and is set to 1 for HDR image, otherwise to 0. b a

19 / 20

Comparison

(a) (b) (c) (d)

• Fig. 9. (a) proposed method by Gaussian filter, (b) proposed method by bilateral filter,

(c) Larson et al., and (d) Durand et al.

• Fig. 10. Comparison of small office.

20 / 20

Simple static local adaptation method for HDR

image compression

– Basis on retinal model – Recreation of the same visual sensations between the real scene and the image on display – Use of bilateral filter to remove banding and halo effects

Conclusion Conclusion