Conceiving a Fast and Practical Multispectral Stereo System Raju - - PowerPoint PPT Presentation
Conceiving a Fast and Practical Multispectral Stereo System Raju Shrestha Supervisors: Jon Yngve Hardeberg, GUC Alamin Mansouri, University of Burgundy Two I maging Technologies A Multispectral system captures image data at specific
Raju Shrestha
Jon Yngve Hardeberg, GUC Alamin Mansouri, University of Burgundy
Supervisors:
Two I maging Technologies
A Multispectral system captures image data at specific wavelengths across electromagnetic spectrum A Stereo system records 3D visual information from two different views (like our HVS)
Multispectral I mage Acquisition
Existing state-of-the art multispectral imaging systems are slow, complex and costly
Goal
Fast and Practical Multispectral System Using Two Digital Cameras At the same time, capable of acquiring 3D stereo images
Main I dea
Use a pair of digital cameras or a stereo camera; in stereoscopic configurations Best pair of filters among a number readily available filters Resulting a Six Channel Multispectral- Stereo System
Six-Channel Multispectral system
Multispectral Camera Model
Camera sensitivities: (s1, s2), Illumination: l, Filter transmittances: (t1, t2), and Surface reflectance: r Model:
Estimating Spectral Reflectance
Problem of estimating Nx1 vector R from Kx1 vector of camera response, C [N>K] => Infinite possible solutions. Linear Model: R’ = QC From training data: Q = RtrainC+ [regularize] Based on spectral (RMSE) &colorimetric (∆E*ab) evaluation metrics Investigated with popular methods: Maloney and Wandell, Imai and Berns, Least Square Wiener, Polynomial and NN MCC240 - Training; MCC24 - Test targets
Simulation Experimental
Measurements
SPD of the light source & spectral reflectance
Fujifilm3D camera has been characterized using Benthem TMc300 monochromator
Simulation Approach
Carried out with several real and imaginary cameras Nikon D70, Canon 20D, Fujifilm 3D 265 filters from Omega Simulated random Gaussian noise (nshot), and quantization noise (nquant), - realistic Selection of Filters: Exhaustive Search that gives minimum estimation error (RMSE, ∆E*ab)
A paper has been accepted in EUSIPCO2010, Denmark
Experimental Approach
Fujifilm FinePix REAL 3D W1 camera A multiband filter: Schott BG-36 Macbeth color charts (MCC240,MCC24) Camera Settings:
Capture 10 images of each color charts Linearize with Gamma values computed from central gray patches of MCC240 (C=Yγ) :1/(1.35,1.37,1.38) Perform DC and Non-uniformity corrections (10 images)
I mage Capture and Corrections
Stereo Analysis
Qualitative: Observation of 3D images Quantitative: From Disparity Maps
Observations show clear 3D Images with slightly reduced brightness
Stereo Camera Model
A scene view is formed by projecting 3D points into the image plane using a perspective transformation:
(X,Y,Z) – A 3D point in world coordinates (u,v) – projection point A – Camera matrix (fx, fy) – focal lengths (cx,cy) – principal point [R|t] – Matrix of extrinsic parameters
k1, k2, k3 – Radial distortion coeffs. p1, p2 – Tangential ,, ,,
l r
x x bf Z − = P t R A sp ] | [ '=
Camera & Stereo Calibration
Using 20 chess board images taken in different
Camera calibration outputs intrinsic parameters (A & dist. coeffs.) Stereo calibration computes extrinsic parameters ([R|t])
Disparity Maps
Compute rectification transforms – to undistort and simplify correspondence problem (epipolar geometry=>one line search) Most commonly used Block Matching (BM) algorithm used Compare disparity maps from our multispectral system with pure stereo
Conclusion and Future Perspectives
Both simulation and experiments show promising results => Proposed system well function as a fast and practical multispectral-stereo system Choice of filters is important for accuracy Slight reduction in the brightness of the 3D images and disparity info, However, quality 3D images are still viewable Relating 3D info with the physical property would be an interesting further work