Color Science Steve Marschner CS 4620 Cornell University Cornell - - PowerPoint PPT Presentation

Steve Marschner CS 4620 Cornell University

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color Science

1

1: Color matching

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color Science

2

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23 [source unknown] 3

• Light is electromagnetic radiation

– exists as oscillations of different frequency (or, wavelength)

[Lawrence Berkeley Lab / MicroWorlds] Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

What light is

4

• Salient property is the spectral power distribution (SPD)

– the amount of light present at each wavelength – units: Watts per nanometer (tells you how much power you’ll find in a narrow range of wavelengths) – for color, often use “relative units” when overall intensity is not important

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Measuring light

wavelength band (width dλ) amount of light = 180 dλ (relative units) wavelength (nm)

5

• Colors are the sensations that arise from light energy
• f different wavelengths

– we are sensitive from about 380 to 760 nm—one “octave”

• Color is a phenomenon of human perception; it is not

a universal property of light

• Roughly speaking, things appear “colored” when they

depend on wavelength and “gray” when they do not.

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

What color is

6

• Build a model for human color perception
• That is, map a Physical light description to a

Perceptual color sensation

?

Physical Perceptual

[Stone 2003] Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

The problem of color science

7

• We can model the low-level

behavior of the eye by thinking

• f it as a light-measuring machine

– its optics are much like a camera – its detection mechanism is also much like a camera

• Light is measured by the

photoreceptors in the retina

– they respond to visible light – different types respond to different wavelengths

[Greger et al. 1995] Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

The eye as a measurement device

8

• Produces a scalar value (a number) when photons land
• n it

– this value depends strictly on the number of photons detected – each photon has a probability of being detected that depends

• n the wavelength

– there is no way to tell the difference between signals caused by light of different wavelengths: there is just a number

• This model works for many detectors:

– based on semiconductors (such as in a digital camera) – based on visual photopigments (such as in human eyes)

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

A simple light detector

9

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

A simple light detector

10

• Same math carries over to power distributions

– spectrum entering the detector has its spectral power distribution (SPD), s(λ) – detector has its spectral sensitivity or spectral response, r(λ)

measured signal input spectrum detector’s sensitivity

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Light detection math

11

• If we think of s and r as vectors, this operation is a dot

product (aka inner product)

– in fact, the computation is done exactly this way if we are using sampled representations of the spectra.

• let λi be regularly spaced sample points Δλ apart; then:
• this sum is very clearly a dot product

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Light detection math

12

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

64

Human eye: retina

Light passes through blood vessels & retinal layers before reaching the light-sensitive cells (“rods” & “cones”)

slide courtesy Pieter Peers

• S,M,L cones have

broadband spectral sensitivity

• S,M,L neural response is

integrated w.r.t. λ

– we’ll call the response functions rS, rM, rL

• Results in a trichromatic

visual system

• S, M, and L are tristimulus

values

[source unknown] Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Cone Responses

14

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Cone responses to a spectrum s

15

• Wanted to map a Physical light description to a

Perceptual color sensation

• Basic solution was known and standardized by 1930

– Though not quite in this form Physical Perceptual

[Stone 2003]

s

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Colorimetry: an answer to the problem

16

• Take a spectrum (which is a function)
• Eye produces three numbers
• This throws away a lot of information!

– Quite possible to have two different spectra that have the same S, M, L tristimulus values – Two such spectra are metamers

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Basic fact of colorimetry

17

• A dot product is a projection
• We are projecting a high dimensional vector (a

spectrum) onto three vectors

– differences that are perpendicular to all 3 vectors are not detectable

• For intuition, we can imagine a 3D analog

– 3D stands in for high-D vectors – 2D stands in for 3D – Then vision is just projection onto a plane

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Pseudo-geometric interpretation

18

• The information available to the visual system about a

spectrum is three values

– this amounts to a loss of information analogous to projection on a plane

• Two spectra that

produce the same response are metamers

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Pseudo-geometric interpretation

19

• Luminance

– the overall magnitude of the the visual response to a spectrum (independent of its color)

• corresponds to the everyday concept “brightness”

– determined by product of SPD with the luminous efficiency function Vλ that describes the eye’s overall ability to detect light at each wavelength – e.g. lamps are optimized to improve their luminous efficiency (tungsten vs. fluorescent vs. sodium vapor)

[Stone 2003] Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Basic colorimetric concepts

20

• Y just has another response curve (like S, M, and L)

– rY is really called “Vλ”

• Vλ is a linear combination of S, M, and L

– Has to be, since it’s derived from cone outputs

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Luminance, mathematically

21

• Chromaticity

– what’s left after luminance is factored out (the color without regard for overall brightness) – scaling a spectrum up or down leaves chromaticity alone

• Dominant wavelength

– many colors can be matched by white plus a spectral color – correlates to everyday concept “hue”

• Purity

– ratio of pure color to white in matching mixture – correlates to everyday concept “colorfulness” or “saturation”

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

More basic colorimetric concepts

22

• Have a spectrum s; want to match on RGB monitor

– “match” means it looks the same – any spectrum that projects to the same point in the visual color space is a good reproduction

• Must find a spectrum that the monitor can produce

that is a metamer of s

R, G, B?

[cs417—Greenberg] Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color reproduction

23

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Additive Color

[source unknown] 24

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

LCD display primaries

25

Curves determined by (fluorescent or LED) backlight and filters

– Native emission curves of 3 LED types

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

LED display primaries

26 [wikipedia user Deglr6328]

• Say we have a spectrum s we want to match on an

RGB monitor

– “match” means it looks the same – any spectrum that projects to the same point in the visual color space is a good reproduction

• So, we want to find a spectrum that the monitor can

produce that matches s

– that is, we want to display a metamer of s on the screen

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color reproduction

27

• We want to compute

the combination of r, g, b that will project to the same visual response as s.

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color reproduction

28

• The projection onto the three response functions can

be written in matrix form:

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color reproduction as linear algebra

29

• The spectrum that is produced by the monitor for the

color signals R, G, and B is:

• Again the discrete form can be written as a matrix:

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color reproduction as linear algebra

30

• What color do we see when we look at the display?

– Feed C to display – Display produces sa – Eye looks at sa and produces V

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color reproduction as linear algebra

31

• Goal of reproduction: visual response to s and sa is the

same:

• Substituting in the expression for sa,

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color reproduction as linear algebra

color matching matrix for RGB

32

• For given primaries, how much are needed to match

each spectral color?

– can be determined experimentally without knowing S, M, L – experiment:

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color matching functions

33 400 450 500 550 600 650 700 −0.1 0.0 0.1 0.2 0.3 0.4 0.5 wavelength (nm) sensitivity r (λ) g (λ) b (λ)

[Marschner & Shirley ch. 19 by Reinhard & Johnson]

Rsr(λ) + Gsg(λ) + Bsb(λ) δ(λ)

single wavelength on right
 mixture of 3 primaries on left (negative weight?
 mix in on right)

¯ r(λ) ¯ g(λ) ¯ b(λ)

• RGB displays have different primary spectra

– and therefore different color matching matrices

• Any display can match any color, though

– up to limits on max. brightness and saturation

• To translate RGB from display 1 to display 2:

– ask what color produces the same visual response

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Translating between displays

34

V = MSMLMRGB1C1 V = MSMLMRGB2C2 C2 = (MSMLMRGB2)−1MSMLMRGB1C1

color transformation matrix = Cmatch2MRGB1

• Camera RGBs are not display primary coefficients!

– there’s no reason mapping them directly would work – they project spectra onto a different 3D subspace

• Cameras have different metamers than humans

– the problem of mapping camera RGB to display RGB is an approximation game – this is fundamentally different from mapping between displays

• Nonetheless the same machinery is used

– map camera RGB to display RGB with a 3x3 matrix – matrix is empirically derived in a calibration process

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Translating camera RGB

35

• The eye makes 3 linear measurements of spectra

– linear algebra: projects onto 3D subspace

• Colors can be matched with 3 primaries

– linear algebra: displayable spectra form 3D subspace – eye’s response and RGB coefficients are related by a linear transformation—a 3x3 matrix – can compute RGB for any spectrum this way

• To translate RGB from one set of primaries to another

– each related to SML by a 3x3 matrix – translate RGB from one display to another by 3x3 matrix

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color matching summary

36

2: Color constancy

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color Science

37

• Observed spectra depend on object and light source
• Reflectance spectrum: what fraction a surface reflects

– as a function of wavelength

• Illuminant spectrum: spectral power of incident light

– as a function of wavelength – normally as a relative measurement (ignore overall brightness)

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Object colors

38

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23 slide courtesy Pieter Peers

[Stone 2003]

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23 slide courtesy Pieter Peers

[Stone 2003]

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23 slide courtesy Pieter Peers

[Stone 2003]

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23 slide courtesy Pieter Peers

[Stone 2003]

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23 slide courtesy Pieter Peers

[Stone 2003]

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

S,M,L responses are what we ‘see’

slide courtesy Pieter Peers

[Stone 2003]

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Reflection from colored surface

[Stone 2003] 40

• Problem for humans: same object produces different

color observations under different illuminants

• Evolved solution: color constancy

– a collection of mechanisms in the human visual system – reasonably effective at discounting illuminant – lets us recognize colors in different situations – …within limits: colors really do look a bit different under different lighting

• Rough mechanism: judge colors relative to white

– visual system decides what objects must be white or gray – uses those colors as a reference to interpret others

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color constancy

41

• Observer of an image judges colors relative to what?

– for a print: relative to the white paper – for a display: more complex, but likely relative to the display’s white

• Upshot: neutral colors should have R=G=B

– these are colors of objects with wavelength-independent reflectance spectra (whites, grays) – R=G=B means it will be perceived as neutral relative to display white

• How to ensure this for photographs?

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color constancy in imaging

42

• Camera works very much like the eye

– RGB pixel measurements are projections of spectra – measured colors depend on illuminant

• Simple heuristic to discount illuminant: divide by gray

– determine the RGB color

• f a neutral object
• w for “white” but any neutral color will do
• this is the illuminant color, up to a scale factor

– only relative scale factors matter, so scale to make one of them 1.0 to avoid changing the overall brightness too much – common plan is to scale R, G, B by

(Rw, Gw, Bw) (Gw/Rw,1,Gw/Bw)

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color balance aka white balance

43

44

Eyestocker on YouTube | https://youtu.be/MUh3vZT0tbo

• How to get a gray color?
• Reliable approach used by professionals: photograph a

known neutral object (a gray card)

• Automatic approach needed in practice

– cameras use heuristics based on image content – roughly, assume lighter colors probably average to gray – take advantage of priors on what illuminants usually occur – many heuristics used in different systems – recent research is applying machine learning to this problem

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color balancing

45

2: Color constancy

Steve Marschner • Cornell CS4620 Fall 2019 • Lecture 23

Color Science

46