Computational Photography Si Lu Spring 2018 - - PowerPoint PPT Presentation

▶
computational photography
SMART_READER_LITE
LIVE PREVIEW

Computational Photography Si Lu Spring 2018 - - PowerPoint PPT Presentation

May 01, 2023 116 likes •793 views

Computational Photography Si Lu Spring 2018 http://web.cecs.pdx.edu/~lusi/CS510/CS510_Computati onal_Photography.htm 04/12/2018 Last Time o Filters o De-noise 2 Today o Color o Color to Gray 3 Light and Color o The frequency, f , of light

slide-1
SLIDE 1

Computational Photography

Si Lu

Spring 2018

http://web.cecs.pdx.edu/~lusi/CS510/CS510_Computati

  • nal_Photography.htm

04/12/2018

slide-2
SLIDE 2

Last Time

  • Filters
  • De-noise

2

slide-3
SLIDE 3

Today

  • Color
  • Color to Gray

3

slide-4
SLIDE 4

Light and Color

  • The frequency, f, of light determines its “color”

n Wavelength, λ, is related: n Energy also related

  • Describe incoming light by a spectrum

n Intensity of light at each frequency n A graph of intensity vs. frequency

  • We care about wavelengths in the visible

spectrum: between the infra-red (700nm) and the ultra-violet (400nm)

4

slide-5
SLIDE 5

Normal Daylight

5

# Photons Wavelength (nm) 400 500 600 700

  • Note the hump at short wavelengths - the sky is blue
slide-6
SLIDE 6

Color and Wavelength

6

slide-7
SLIDE 7

Color Spaces

  • The principle of trichromacy means that the colors

displayable are all the linear combination of primaries

  • Taking linear combinations of R, G and B defines the

RGB color space

n the range of perceptible colors generated by adding some part each of R, G and B

  • If R, G and B correspond to a monitor’s phosphors

(monitor RGB), then the space is the range of colors displayable on the monitor

7

slide-8
SLIDE 8

RGB Color Space

8

slide-9
SLIDE 9

L*a*b* Color Space

  • RGB

n Perceptually non-uniform

  • L*a*b*

n More perceptually uniform n Look into Opencv or Matlab http://en.wikipedia.org/wiki/Lab_color_space

9

slide-10
SLIDE 10
  • The eye contains rods and cones

n Rods work at low light levels and do not see color

  • That is, their response depends only on how many photons, not their

wavelength

n Cones come in three types (experimentally and genetically proven), each responds in a different way to frequency distributions

Seeing in Color

10

slide-11
SLIDE 11

Color receptors

11

  • Each cone type has a

different sensitivity curve

n Experimentally determined in a variety of ways

  • For instance, the L-cone

responds most strongly to red light

  • “Response” in your eye

means nerve cell firings

  • How you interpret those

firings is not so simple …

slide-12
SLIDE 12

Color Perception

12

  • How your brain interprets nerve impulses from your

cones is an open area of study, and deeply mysterious

  • Colors may be perceived differently:

n Affected by other nearby colors n Affected by adaptation to previous views n Affected by “state of mind”

  • Experiment:

n Subject views a colored surface through a hole in a sheet, so that the color looks like a film in space n Investigator controls for nearby colors, and state of mind

slide-13
SLIDE 13

The Same Color?

13

slide-14
SLIDE 14

The Same Color?

14

slide-15
SLIDE 15

Color Deficiency

15

  • Some people are missing one type of receptor

n Most common is red-green color blindness in men n Red and green receptor genes are carried on the X chromosome

  • most red-green color blind men have two red genes or two

green genes

  • Other color deficiencies

n Anomalous trichromacy, Achromatopsia, Macular degeneration n Deficiency can be caused by the central nervous system, by

  • ptical problems in the eye, injury, or by absent receptors
slide-16
SLIDE 16

Color Deficiency

16

slide-17
SLIDE 17

Color Transformation

  • Re-coloring
  • Color to Gray

17

slide-18
SLIDE 18

Color2Gray: Salience-Preserving Color Removal

Amy Gooch Sven Olsen Jack Tumblin Bruce Gooch

slide-19
SLIDE 19

Color Grayscale New Algorithm

slide-20
SLIDE 20

Isoluminant Colors

Color Grayscale

slide-21
SLIDE 21

CIE CAM 97 Photoshop LAB CIE XYZ YCrCb

Traditional Methods:

Luminance Channels

slide-22
SLIDE 22

CIE CAM 97 Photoshop LAB CIE XYZ YCrCb

Traditional Methods:

Luminance Channels

Problem can not be solved by simply switching to a different space

slide-23
SLIDE 23

Traditional Methods

  • Contrast enhancement &

Gamma Correction

– Doesn’t help with isoluminant values

Photoshop Grayscale PSGray + Auto Contrast New Algorithm

slide-24
SLIDE 24

Goals

  • Dimensionality Reduction

– From tristimulus values to single channel

Loss of information

  • Maintain salient features in color image

– Human perception

slide-25
SLIDE 25

Relative differences

Color Illusion by Lotto and Purves http://www.lottolab.org

slide-26
SLIDE 26

Relative differences

Color Illusion by Lotto and Purves http://www.lottolab.org

slide-27
SLIDE 27

Relative differences

Color Illusion by Lotto and Purves http://www.lottolab.org

slide-28
SLIDE 28

Challenge 1: Influence of neighboring pixels

slide-29
SLIDE 29

Challenge 2: Dimension and Size Reduction

  • 120, -120

120, 120 100

slide-30
SLIDE 30

Original

Challenge 3: Many Color2Gray Solutions

. . .

slide-31
SLIDE 31

Algorithm Overview

  • Adjust g to incorporate both luminance and

chrominance differences

dij

  • For every pair of pixel i and j

– Compute Luminance distance – Compute Chrominance distance

  • Initialize image, g, with L channel
  • Convert to Perceptually Uniform Space

– CIE L*a*b*

slide-32
SLIDE 32

Color2Grey Algorithm

Optimization:

min S S ( (gi - gj) - di,j )2

i

j=i-m i+m

slide-33
SLIDE 33

Parameters

Map chromatic difference to increases or decreases in luminance values

q : Max chrominance offset a : Radius of neighboring pixels m :

slide-34
SLIDE 34

m = 2 m = 16 m= entire image

q= 300o a = 10 q= 49o a = 10

m : Neighborhood Size

slide-35
SLIDE 35

m : Neighborhood Size

m = 16 m= entire image

slide-36
SLIDE 36
  • Luminance Distance:

Problem: ||DCij|| is unsigned DLij = Li - Lj

Perceptual Distance

  • Chrominance Distance: ||DCij||
slide-37
SLIDE 37

C2 C1

Map chromatic difference to increases or decreases in luminance values

Color Space

+b* +a*

  • a*
slide-38
SLIDE 38

+DC1,2 q vq

sign(||DCi,j||) = sign(DCi,j . vq )

Color Difference Space

vq = (cos q, sin q)

+Db* +Da*

  • Da*
  • +

+

  • Db*
slide-39
SLIDE 39

Photoshop Grayscale

q = 225 q = 45

slide-40
SLIDE 40

Grayscale

q = 45 q = 135 q = 0

slide-41
SLIDE 41

How to Combine Chrominance and Luminance

dij = (Luminance) DLij

slide-42
SLIDE 42

How to Combine Chrominance and Luminance

dij = DLij ||DCij|| (Luminance) if |DLij| > ||DCij|| (Chrominance)

slide-43
SLIDE 43

. . .

How to Combine Chrominance and Luminance

d(a,q)ij = DLij ||DCij|| if DCij . nq ≥ 0 if |DLij| > ||DCij||

  • ||DCij||
  • therwise

Grayscale

slide-44
SLIDE 44

. . .

How to Combine Chrominance and Luminance

d(a,q)ij = DLij crunch(||DCij||) if DCij . nq ≥ 0 if |DLij| > crunch(||DCij||) crunch(-||DCij||)

  • therwise

Grayscale

slide-45
SLIDE 45

a: Chromatic variation maps to luminance variation

a = 5 a = 10 a = 25 a

  • a

crunch(x) = a * tanh(x/a)

slide-46
SLIDE 46

Color2Grey Algorithm

Optimization:

min S S ( (gi - gj) - di,j )2

j=i-m i i+m

slide-47
SLIDE 47

Original Color2Grey Photoshop Grey

Results

Color2Grey + Color

slide-48
SLIDE 48

Original PhotoshopGrey Color2Grey

slide-49
SLIDE 49

Original PhotoshopGrey Color2Grey+Color

slide-50
SLIDE 50

Original PhotoshopGrey Color2Grey

slide-51
SLIDE 51

Original PhotoshopGrey Color2Grey

slide-52
SLIDE 52

Implementation Performance

  • Image of size S x S

– O(m2 S2) or O(S4) for full neighborhood case

  • 12.7s 100x100 image
  • 65.6s 150x150 image
  • 204.0s 200x200 image

– GPU implementation

  • O(S2) ideal, really O(S3)

– 2.8s 100x100 – 9.7s 150x150 – 25.7s 200x200

Athlon 64 3200 CPU NVIDIA GeForce GT6800

slide-53
SLIDE 53

Future Work

  • Animations/Video
  • Faster

– Multiscale

  • Smarter

– Remove need to specify q

  • New optimization function designed to match both signed

and unsigned difference terms

– Image complexity measures

slide-54
SLIDE 54

Validate "Salience Preserving"

Apply Contrast Attention model by Ma and Zhang 2003

Original PhotoshopGrey Color2Grey

slide-55
SLIDE 55

Validate "Salience Preserving"

Original PhotoshopGrey Color2Grey

slide-56
SLIDE 56

Thank you

  • SIGGRAPH Reviewers
  • NSF
  • Helen and Robert J. Piros Fellowship
  • Northwestern Graphics Group
  • MidGraph2004 Participants

– especially Feng Liu

  • (sorry I spelled your name wrong in the acknowledgements)
slide-57
SLIDE 57

Next Time

  • Re-lighting

57

slide-58
SLIDE 58
slide-59
SLIDE 59

Original Color2Grey Color2Grey+Color

slide-60
SLIDE 60

Original Color2Grey Color2Grey+Color

slide-61
SLIDE 61

Original Color2Grey Color2Grey+Color

slide-62
SLIDE 62

Original PhotoshopGrey Color2Grey

slide-63
SLIDE 63

Original Color2Grey Color2Grey+Color

slide-64
SLIDE 64

Original PhotoshopGrey Color2Grey

slide-65
SLIDE 65

Original Color2Grey Color2Grey+Color

slide-66
SLIDE 66

Next Time

  • Re-lighting

66