Efficient Regression for Computational Imaging: from Color - PowerPoint PPT Presentation

Efficient Regression for Computational Imaging: from Color Management to Omnidirectional Superresolution Maya R. Gupta Eric Raman Garcia Arora

Regression 2

Regression

Regression

Linear Regression: fast, not good enough

Problem : Device Dependent Colors Depend on Device

Color Management For each device, characterize the mapping between the native color space and a device independent color space. ICC ICC Profile Profile CIELab (Lab) ICC ICC Profile Profile 8/5/2009 7

Color Management • For each device, characterize the mapping between the native color space and a device independent color space. ICC ICC Profile Profile CIELab (Lab) ICC ICC Profile Profile CIELab is a widely used device- independent color space that is perceptually uniform (i.e. Euclidean distance approximates human judgement of color dissimilarity) 8/5/2009 8

Color Management • For each device, characterize the mapping between the native color space and a device independent color space. ICC ICC Profile Profile CIELab (Lab) ICC ICC Profile Profile Mapping from RGB -> CIELab and CIELab -> CMYK can be highly nonlinear 8/5/2009 9

Gamut mapping: linear transforms not adequate Original Extended gamut gamut Skin Skin tones tones Original Gamut Linear regression Nonlinear regression

Creating Custom Color Enhancements Ex: simulating illumination effects transformed by artist to “sunset” original 2 hrs. work in Photoshop

Example Convert an image to how it would look in Cinecolor based on 16 sample color pairs Original cinecolor www.widescreenmuseum.org

Color management: speed by LUT 8/5/2009 13

Color management: speed by LUT 8/5/2009 14

Color management: speed by LUT 15

Color management: speed by LUT

Color management: speed by LUT

Color management: speed by LUT

Color management: speed by LUT

Linear Interpolation is linear in the outputs

Linear Interpolation is linear in the outputs

Linear Interpolation is linear in the outputs

Lattice Regression Choose the lattice outputs to minimize the post-linear interpolation empirical risk on the data: 8/5/2009 23

Lattice Regression Choose the lattice outputs to minimize the post-linear interpolation empirical risk on the data: 8/5/2009 24

Lattice Regression Choose the lattice outputs to minimize the post-linear interpolation empirical risk on the data:

Effect of Different Lattice Regression Regularizers 8/5/2009 26

Effect of Different Lattice Regression Regularizers 8/5/2009 27

Lattice Regression Closed Form Solution Sparse: No more than 7 d m non-zero entries (of m 2 ) with cubic interpolation. 9/15/2010 28

Example Color Management Results

Example Color Management Results

Omnidirectional Super-resolution: 9/15/2010 31

Omnidirectional Superres Related Work State of the Art: Arican and Frossard 2008-2009 (ICPR 2008 Best Paper Award) • Interpolation with spherical harmonics • Alignment with an iterative conjugate gradient approach.

Lattice Regression Approach Finding the correct registration of the low-resolution images is challenging non-convex optimization problem. Evaluate a candidate registration: use lattice regression on image subset -> high-res spherical grid sum interpolation error for all left-out low res image data

Lattice Regression Approach Finding the correct registration of the low-resolution images is challenging non-convex optimization problem. Evaluate a candidate registration: use lattice regression on image subset -> high-res spherical grid sum interpolation error for all left-out low res image data Finding the optimal joint registration is a 3(N-1)-d opt. problem  We use FIPS to find the global optimum.

9/15/2010 36

Visual Homing START . . Lattice Regression Better . HOME . For Visual Homing . . .

Some Conclusions

Some Conclusions

Some Conclusions

Some Conclusions

For details, see: • “Optimized Regression for Efficient Function Evaluation,” Eric K. Garcia, Raman Arora, and Maya R. Gupta, (in review – draft upon request). • “Lattice Regression”, Eric K. Garcia, Maya R. Gupta, Neural Information Processing Systems (NIPS) 2009. • “Building Accurate and Smooth ICC Profiles by Lattice Regression,” Eric K. Garcia, Maya R. Gupta, 17 th IS&T Color Imaging Conference 2009. • "Adaptive Local Linear Regression with Application to Printer Color Management," Maya R. Gupta, Eric K. Garcia, and Erika Chin, IEEE Trans. on Image Processing , vol. 17, no. 6, 936-945, 2008. • "Learning Custom Color Transformations with Adaptive Neighborhoods," Maya R. Gupta, Eric K. Garcia, and Andrey Stroilov, Journal of Electronic Imaging, vol. 17, no. 3, 2008. • "Gamut Expansion for Video and Image Sets," Hyrum Anderson, Eric K. Garcia, and Maya R. Gupta, Computational Color Imaging Workshop, 2007. 8/5/2009 42

Color is an event human light source perceives color reflection human cones respond: L = long wave = red M = medium wave = green S = short wave = blue

What does it mean to see black ? human light source perceives ??? color human cones respond L = long wave = red M = medium wave = green S = short wave = blue

What does it mean to see white? human light source perceives ??? color human cones respond L = long wave = red M = medium wave = green S = short wave = blue

What does it mean to see white? images from: www.omatrix.com/uscolors.html You can see “white” given light made up of 2-spectra

Color Science Crash Course • What we see can be represented by three primaries. match monochromatic mixture of three light at some primary colors wavelength Stiles-Burch 10° color matching functions averaged across 37 observers . Adapted from (Wyszecki & Stiles, 1982) by handprint.com. 8/5/2009 47

Color Distances • CIELab • Based on spectral measurements of color, integrated over CMF envelopes. • Euclidean distance between two colors approximates the perceptual difference noticed by a human observer. • Distance metrics created to correct for perceptual non- uniformities in the space: image source: www.handprint.com 8/5/2009 48

2-D and 3-D Simulation d=2 d=3 8/5/2009 49

Color management for printers 8 bit RGB color patch Color printed Human eye printer color patch Measure CIEL*a*b* Goal: Print a given CIEL*a*b* value. Problem: What RGB value to input?

Inverse Device Characterization Step 1 Sample the device CIELab Output Measure Step 2 Build an inverse look-up-table Look-up-table Regression 8/5/2009 51

8/5/2009 52

Gaussian Process Regression • Models data as being drawn from a Gaussian Process • A leading method in geostatistics (2-d regression) also known as Kriging. • Generally considered a state-of-the-art method by machine learning folks • Parameters: Covariance Function ( length scale L ), Noise Power σ 2 . (L large, σ 2 small) (L small, σ 2 small) (L large, σ 2 large) 8/5/2009 53

Gaussian Process Regression • Models data as being drawn from a Gaussian Process • A leading method in geostatistics (2-d regression) also known as Kriging. • Generally considered a state-of-the-art method by machine learning folks • Parameters: Covariance Function ( length scale L ), Noise Power σ 2 . (L large, σ 2 small) (L small, σ 2 small) (L large, σ 2 large) • Given Covariance form, parameters can be learned by maximizing marginal likelihood . (i.e. automatically from data). 8/5/2009 54

2-D Simulation 1000 Training Samples 50 Training Samples Gaussian Process Regression (Direct) Gaussian Process Regression (to nodes of lattice) Lattice Regression (GPR bias) Lattice Regression (Bilinear bias) 8/5/2009 55

3-D Simulation 1000 Training Samples 50 Training Samples Gaussian Process Regression (Direct) Gaussian Process Regression (to nodes of lattice) Lattice Regression (GPR bias) Lattice Regression (Bilinear bias) 8/5/2009 56