Efficient Regression for Computational Imaging: from Color - - PowerPoint PPT Presentation

Efficient Regression for Computational Imaging: from Color Management to Omnidirectional Superresolution

Maya R. Gupta Eric Garcia Raman Arora

Regression

2

Regression

Regression

Linear Regression: fast, not good enough

Problem: Device Dependent Colors Depend on Device

Color Management

For each device, characterize the mapping between the native color space and a device independent color space.

8/5/2009 7

CIELab (Lab)

ICC Profile ICC Profile ICC Profile ICC Profile

Color Management

• For each device, characterize the mapping between the native

color space and a device independent color space.

8/5/2009 8

CIELab (Lab)

ICC Profile ICC Profile ICC Profile ICC Profile

CIELab is a widely used device- independent color space that is perceptually uniform (i.e. Euclidean distance approximates human judgement of color dissimilarity)

Color Management

• For each device, characterize the mapping between the native

color space and a device independent color space.

8/5/2009 9

CIELab (Lab)

ICC Profile ICC Profile ICC Profile ICC Profile

Mapping from RGB -> CIELab and CIELab -> CMYK can be highly nonlinear

Gamut mapping: linear transforms not adequate

Skin tones Skin tones

Original gamut Extended gamut Original Gamut Linear regression Nonlinear regression

Creating Custom Color Enhancements

• riginal

transformed by artist to “sunset” 2 hrs. work in Photoshop Ex: simulating illumination effects

Example

Convert an image to how it would look in Cinecolor based on 16 sample color pairs

www.widescreenmuseum.org

Original cinecolor

Color management: speed by LUT

8/5/2009 13

Color management: speed by LUT

8/5/2009 14

Color management: speed by LUT

15

Color management: speed by LUT

Color management: speed by LUT

Color management: speed by LUT

Color management: speed by LUT

Linear Interpolation is linear in the outputs

Linear Interpolation is linear in the outputs

Linear Interpolation is linear in the outputs

Lattice Regression

Choose the lattice outputs to minimize the post-linear interpolation empirical risk on the data:

8/5/2009 23

Lattice Regression

Choose the lattice outputs to minimize the post-linear interpolation empirical risk on the data:

8/5/2009 24

Lattice Regression

Choose the lattice outputs to minimize the post-linear interpolation empirical risk on the data:

Effect of Different Lattice Regression Regularizers

8/5/2009 26

Effect of Different Lattice Regression Regularizers

8/5/2009 27

Lattice Regression Closed Form Solution

9/15/2010 28

Sparse: No more than 7dm non-zero entries (of m2) with cubic interpolation.

Example Color Management Results

Example Color Management Results

9/15/2010 31

Omnidirectional Super-resolution:

Omnidirectional Superres Related Work

State of the Art: Arican and Frossard 2008-2009 (ICPR 2008 Best Paper Award)

• Interpolation with spherical harmonics
• Alignment with an iterative conjugate gradient

approach.

Lattice Regression Approach

Finding the correct registration of the low-resolution images is challenging non-convex optimization problem. Evaluate a candidate registration: use lattice regression on image subset -> high-res spherical grid sum interpolation error for all left-out low res image data

Lattice Regression Approach

Finding the correct registration of the low-resolution images is challenging non-convex optimization problem. Evaluate a candidate registration: use lattice regression on image subset -> high-res spherical grid sum interpolation error for all left-out low res image data Finding the optimal joint registration is a 3(N-1)-d opt. problem  We use FIPS to find the global optimum.

9/15/2010 36

Visual Homing

START . . . HOME . . . . Lattice Regression Better For Visual Homing

Some Conclusions

Some Conclusions

Some Conclusions

Some Conclusions

For details, see:

• “Optimized Regression for Efficient Function Evaluation,” Eric K. Garcia,

Raman Arora, and Maya R. Gupta, (in review – draft upon request).

• “Lattice Regression”, Eric K. Garcia, Maya R. Gupta, Neural Information

Processing Systems (NIPS) 2009.

• “Building Accurate and Smooth ICC Profiles by Lattice Regression,” Eric K.

Garcia, Maya R. Gupta, 17th IS&T Color Imaging Conference 2009.

• "Adaptive Local Linear Regression with Application to Printer Color

Management," Maya R. Gupta, Eric K. Garcia, and Erika Chin, IEEE Trans.

• n Image Processing , vol. 17, no. 6, 936-945, 2008.
• "Learning Custom Color Transformations with Adaptive Neighborhoods,"

Maya R. Gupta, Eric K. Garcia, and Andrey Stroilov, Journal of Electronic Imaging, vol. 17, no. 3, 2008.

• "Gamut Expansion for Video and Image Sets," Hyrum Anderson, Eric K.

Garcia, and Maya R. Gupta, Computational Color Imaging Workshop, 2007.

8/5/2009 42

Color is an event

light source human cones respond: human perceives color L = long wave = red M = medium wave = green S = short wave = blue reflection

What does it mean to see black?

light source human cones respond ??? human perceives color L = long wave = red M = medium wave = green S = short wave = blue

What does it mean to see white?

light source human cones respond ??? human perceives color L = long wave = red M = medium wave = green S = short wave = blue

What does it mean to see white?

images from: www.omatrix.com/uscolors.html

You can see “white” given light made up of 2-spectra

Color Science Crash Course

• What we see can be

represented by three primaries.

8/5/2009 47

Stiles-Burch 10° color matching functions averaged across 37

• bservers . Adapted from (Wyszecki

& Stiles, 1982) by handprint.com.

monochromatic light at some wavelength match mixture of three primary colors

Color Distances

• CIELab
• Based on spectral

measurements

• f color,

integrated over CMF envelopes.

• Euclidean distance between two

colors approximates the perceptual difference noticed by a human observer.

• Distance metrics created to

correct for perceptual non- uniformities in the space:

8/5/2009 48 image source: www.handprint.com

2-D and 3-D Simulation

8/5/2009 49

d=2 d=3

Color printer 8 bit RGB color patch printed color patch Human eye Measure CIEL*a*b*

Color management for printers

Goal: Print a given CIEL*a*b* value. Problem: What RGB value to input?

Inverse Device Characterization

8/5/2009 51

CIELab

Step 1 Sample the device Step 2 Build an inverse look-up-table

Regression

Look-up-table

Output Measure

8/5/2009 52

Gaussian Process Regression

• Models data as being drawn from a Gaussian Process

8/5/2009 53

(L large, σ2 small) (L small, σ2 small) (L large, σ2 large)

• A leading method in geostatistics (2-d regression) also known as Kriging.
• Generally considered a state-of-the-art method by machine learning folks
• Parameters: Covariance Function (length scale L), Noise Power σ2.

Gaussian Process Regression

• Models data as being drawn from a Gaussian Process

8/5/2009 54

(L large, σ2 small) (L small, σ2 small) (L large, σ2 large)

• A leading method in geostatistics (2-d regression) also known as Kriging.
• Generally considered a state-of-the-art method by machine learning folks
• Parameters: Covariance Function (length scale L), Noise Power σ2.
• Given Covariance form, parameters can be learned by maximizing

marginal likelihood. (i.e. automatically from data).

2-D Simulation

8/5/2009 55

Gaussian Process Regression (Direct) Gaussian Process Regression (to nodes of lattice) Lattice Regression (GPR bias) Lattice Regression (Bilinear bias)

50 Training Samples 1000 Training Samples

3-D Simulation

8/5/2009 56

Gaussian Process Regression (Direct) Gaussian Process Regression (to nodes of lattice) Lattice Regression (GPR bias) Lattice Regression (Bilinear bias)

50 Training Samples 1000 Training Samples