Angular Embedding: from Jarring Intensity Differences to Perceived - - PowerPoint PPT Presentation

Angular Embedding: from Jarring Intensity Differences to Perceived Luminance

Stella X. Yu Computer Science Boston College

Acknowledgements: Edward H. Adelson Clare Boothe Luce Professorship NSF CAREER IIS-0644204 IEEE Conference on Computer Vision and Pattern Recognition, 2009 1 / 15

Distinction: Intensity, Brightness, and Lightness

1 2 3 4 5 6 .7 .4 .4 .3 .3 .2

intensity = measured luminance: I1 > I2 = I3 > I4 = I5 > I6 brightness = perceived luminance: B1 > B2 > B3 > B4 > B6 > B5 lightness = perceived reflectance: L1 = L2 > L3 = L4 = L6 > L5

2 / 15

Helmholtz and Hering Debate

• 1. Helmholtz: byproduct of high-level cognitive cause

– recover reflectance from luminance with unknown illumination – Land & McCann, Retinex, 1971 – Barrow & Tenenbaum, intrinsic images, 1978

• 2. in-between

– Ross & Pessoa, selective integration model, 2000 – Kelly & Grossberg, Form-And-Color-And-DEpth, 2000

• 3. Hering: manifestation of low-level physiological cause

– lateral inhibition, center-surround filtering – Blakeslee et al, multiscale filtering, 2005

3 / 15

Basic Brightness Illusions

L D

Simultaneous Contrast

D L S S L D

White Anti-snake Snake

4 / 15

Textbook Explanation: Center-Surround Filtering

+ D L

L D

+ L D + S S scale too large scale just right scale too small center-surround filter = difference of Gaussians

5 / 15

Selective Enhancement is a Must but not by Size

L D

−0.4 −0.2 0.2 0.4 0.1 0.3 0.5 0.7 0.9

L ✔ D increment-decrement derivative modified and integrated double-decrement

L D

−0.4 −0.2 0.2 0.4 0.1 0.3 0.5 0.7 0.9

D ✘ L Enhancing small edges only explains one of the two illusions!

6 / 15

Insight: Selective Enhancement by Edge Geometry

1 2

difference intensified around a corner!

b b

pixel a pixel b edge Coarse-scale differences provide the right selective enhancement. Brightness differences across an edge increase with its curvature.

7 / 15

Brightness is Analogous to Motion Perception

b b b b b b b b

short-range cue long-range cue fine-scale interior

measured intensity perceived brightness

cue integration

• 1. Feature →

enable brightness with short-range cues

fine-scale for interiors, and coarser-scale across edges

• 2. Aperture →

reinforce brightness with long-range cues

paths of higher confidence, originating from corners, dominate

• 3. Integration → realize

brightness from pairwise local cues

maximally fulfill local orderings in accordance with confidence levels

8 / 15

Brightness Modeling is Global Brightness Ordering

intensity I pairwise edges E pairwise differences (O,C) brightness B difference B − I

• 1. edge detection
• 2. brightness ordering
• 3. angular embedding

9 / 15

New Integration Method: Angular Embedding

input: local ordering O = pairwise differences C = confidence in O

• utput: global ordering

x = positions on a line, or z = positions on the unit circle

b

x(a)

b

x(b)

b

x(c)

• ld: linear space

x(a) − x(b) = O(a,b) ?

b

1 j

b z(b) b z(a) = ejθ(a) b

z(c) θ(b) θ(a) θ(c) new: angular space θ(a) − θ(b) = O(a,b) ?

10 / 15

Criterion: Minimize Distance to Local Average

b

1 j

b z(b) b

z(a)

b

z(c)

b z(b)ejO(a,b) b

z(c)ejO(a,c) C(a,b) C ( a , c )

b˜

z(a) O(a,b) O(a,c) θ(b) minimize: ǫ(z;O,C) =

• a

D(a,a) · |z(a) − ˜ z(a)|2 local average: ˜ z(a) =

• b

C(a,b) D(a,a) z(b)ejO(a,b) total confidence: D(a,a) =

• b

C(a,b)

11 / 15

Optimum: Angles of the Smallest Eigenvector

angular embedding minimize: ǫ(z;O,C) = z′Wz representation: z = ejθ error: W = (I − D−1M)′ D(I − D−1M) measurement: M = C • ejO degree: D = Diag(C1)

• ptimum:

θ ∗ = ∡z∗ = ∡ smallest-eigenvector-of (W,D) least squares minimize: ǫ(x;O,C) =

• C(a,b)(x(a) − x(b) − O(a,b))2

measurement: M = C • O + (C • O)′ degree: D = Diag((C + C′)1) transition: P = D−1(C + C′)

• ptimum:

x∗ = (I − P)−1 · (D−1M1)

12 / 15

An Efficient and More Robust Integration Method

• riginal image

3 × 6 measurement outliers neighbourhood radius = 2 LS optimum x∗ AE optimum z∗ AE optimum θ ∗

13 / 15

Brightness as Intensity Deviating along Gradient

deviation by scene interpretation deviation by intensity context itself Adelson, 1999: X junctions & atmospheres transparency haze clear paint n-rev-T rev-T n-rev-X s-rev-X d-rev-X intensity brightness difference

14 / 15

Brightness as Gestalt from Scale-Mixed Differences

input: objective intensity ⇓

• utput: subjective brightness

brightness − intensity Simultaneous Contrast Anti-Snake Snake Koffka Ring Benary Cross

15 / 15