Computational Photography Si Lu Spring 2018 http://web.cecs.pdx.edu/~lusi/CS510/CS510_Computati onal_Photography.htm 04/12/2018
Last Time o Filters o De-noise 2
Today o Color o Color to Gray 3
Light and Color o The frequency, f , of light determines its “color” n Wavelength, λ , is related: n Energy also related o Describe incoming light by a spectrum n Intensity of light at each frequency n A graph of intensity vs. frequency o We care about wavelengths in the visible spectrum: between the infra-red (700nm) and the ultra-violet (400nm) 4
Normal Daylight # Photons Wavelength (nm) 400 500 600 700 o Note the hump at short wavelengths - the sky is blue 5
Color and Wavelength 6
Color Spaces o The principle of trichromacy means that the colors displayable are all the linear combination of primaries o Taking linear combinations of R, G and B defines the RGB color space n the range of perceptible colors generated by adding some part each of R, G and B o If R, G and B correspond to a monitor’s phosphors (monitor RGB), then the space is the range of colors displayable on the monitor 7
RGB Color Space 8
L*a*b* Color Space o RGB n Perceptually non-uniform o L*a*b* n More perceptually uniform n Look into Opencv or Matlab http://en.wikipedia.org/wiki/Lab_color_space 9
Seeing in Color o The eye contains rods and cones n Rods work at low light levels and do not see color That is, their response depends only on how many photons, not their o wavelength Cones come in three types (experimentally and genetically proven), n each responds in a different way to frequency distributions 10
Color receptors o Each cone type has a different sensitivity curve Experimentally determined n in a variety of ways o For instance, the L-cone responds most strongly to red light o “Response” in your eye means nerve cell firings o How you interpret those firings is not so simple … 11
Color Perception o How your brain interprets nerve impulses from your cones is an open area of study, and deeply mysterious o Colors may be perceived differently: n Affected by other nearby colors n Affected by adaptation to previous views Affected by “state of mind” n o Experiment: n Subject views a colored surface through a hole in a sheet, so that the color looks like a film in space n Investigator controls for nearby colors, and state of mind 12
The Same Color? 13
The Same Color? 14
Color Deficiency o Some people are missing one type of receptor Most common is red-green color blindness in men n Red and green receptor genes are carried on the X chromosome n - most red-green color blind men have two red genes or two green genes o Other color deficiencies n Anomalous trichromacy, Achromatopsia, Macular degeneration Deficiency can be caused by the central nervous system, by n optical problems in the eye, injury, or by absent receptors 15
Color Deficiency 16
Color Transformation o Re-coloring o Color to Gray 17
Color2Gray: Salience-Preserving Color Removal Amy Gooch Sven Olsen Jack Tumblin Bruce Gooch
New Algorithm Color Grayscale
Isoluminant Colors Color Grayscale
Traditional Methods: Luminance Channels CIE CAM 97 Photoshop LAB CIE XYZ YCrCb
Traditional Methods: Luminance Channels Problem can not be solved by CIE CAM 97 Photoshop LAB simply switching to a different space CIE XYZ YCrCb
Traditional Methods • Contrast enhancement & Gamma Correction – Doesn’t help with isoluminant values Photoshop Grayscale PSGray + Auto Contrast New Algorithm
Goals • Dimensionality Reduction – From tristimulus values to single channel Loss of information • Maintain salient features in color image – Human perception
Relative differences Color Illusion by Lotto and Purves http://www.lottolab.org
Relative differences Color Illusion by Lotto and Purves http://www.lottolab.org
Relative differences Color Illusion by Lotto and Purves http://www.lottolab.org
Challenge 1: Influence of neighboring pixels
Challenge 2: Dimension and Size Reduction 120, 120 100 0 -120, -120
Challenge 3: Many Color2Gray Solutions Original . . .
Algorithm Overview • Convert to Perceptually Uniform Space – CIE L*a*b* • Initialize image, g , with L channel • For every pair of pixel i and j – Compute Luminance distance d ij – Compute Chrominance distance • Adjust g to incorporate both luminance and chrominance differences
Color2Grey Algorithm Optimization: i+ m min S S ( (g i - g j ) - d i,j ) 2 j=i- m i
Parameters m : Radius of neighboring pixels a : Max chrominance offset q : Map chromatic difference to increases or decreases in luminance values
m : Neighborhood Size q = 300 o a = 10 m = entire image m = 2 m = 16 q = 49 o a = 10
m : Neighborhood Size m = entire image m = 16
Perceptual Distance D L ij = L i - L j • Luminance Distance: • Chrominance Distance: || D C ij || Problem: || D C ij || is unsigned
Map chromatic difference to increases or decreases in luminance values +b* C 2 -a* +a* Color C 1 Space
+ D b* Color +D C 1,2 v q = (cos q , sin q ) Difference Space + - v q q - D a* + D a* + - sign ( || D C i,j || ) = sign ( D C i,j . v q ) - D b*
q = 45 q = 225 Photoshop Grayscale
q = 135 q = 45 q = 0 Grayscale
How to Combine Chrominance and Luminance d ij = D L ij (Luminance)
How to Combine Chrominance and Luminance (Luminance) if | D L ij | > || D C ij || D L ij d ij = || D C ij || (Chrominance)
How to Combine Chrominance and Luminance D L ij if | D L ij | > || D C ij || d(a,q) ij = if D C ij . n q ≥ 0 || D C ij || -|| D C ij || otherwise . . . Grayscale
How to Combine Chrominance and Luminance D L ij if | D L ij | > crunch(|| D C ij ||) d(a,q) ij = if D C ij . n q ≥ 0 crunch(|| D C ij ||) crunch(-|| D C ij ||) otherwise . . . Grayscale
a : Chromatic variation maps to luminance variation a -a crunch(x) = a * tanh(x/ a ) a = 5 a = 10 a = 25
Color2Grey Algorithm Optimization: i+ m min S S ( (g i - g j ) - d i,j ) 2 j=i- m i
Results Photoshop Color2Grey Original Color2Grey Grey + Color
Original PhotoshopGrey Color2Grey
Original PhotoshopGrey Color2Grey+Color
Original PhotoshopGrey Color2Grey
Original PhotoshopGrey Color2Grey
Implementation Performance • Image of size S x S – O( m 2 S 2 ) or O(S 4 ) for full neighborhood case • 12.7s 100x100 image • 65.6s 150x150 image Athlon 64 3200 CPU • 204.0s 200x200 image – GPU implementation • O(S 2 ) ideal, really O(S 3 ) – 2.8s 100x100 NVIDIA GeForce GT6800 – 9.7s 150x150 – 25.7s 200x200
Future Work • Faster – Multiscale • Smarter – Remove need to specify q • New optimization function designed to match both signed and unsigned difference terms – Image complexity measures • Animations/Video
Validate "Salience Preserving" Original PhotoshopGrey Color2Grey Apply Contrast Attention model by Ma and Zhang 2003
Validate "Salience Preserving" Original PhotoshopGrey Color2Grey
Thank you • SIGGRAPH Reviewers • NSF • Helen and Robert J. Piros Fellowship • Northwestern Graphics Group • MidGraph2004 Participants – especially Feng Liu • (sorry I spelled your name wrong in the acknowledgements)
Next Time o Re-lighting 57
Original Color2Grey Color2Grey+Color
Original Color2Grey Color2Grey+Color
Color2Grey Original Color2Grey+Color
Original PhotoshopGrey Color2Grey
Original Color2Grey Color2Grey+Color
Original PhotoshopGrey Color2Grey
Original Color2Grey Color2Grey+Color
Next Time o Re-lighting 66
Recommend
More recommend
