Revisiting 3D Geometric Models for Accurate Object Shape and Pose M. Zeeshan Zia 1 Michael Stark 2,3 Bernt Schiele 3 Konrad Schindler 1 3 Max-Planck-Institute for Informatics 1 Photogrammetry and Remote Sensing Laboratory 2 Artificial Intelligence Lab Saarbrücken, Germany Swiss Federal Institute of Technology (ETH), Zurich Stanford University, USA
Current object models: coarse grained estimates 1 Zeeshan Zia
Our goal: finer-grained models to aid scene-level reasoning 2 Zeeshan Zia
Revival of 3D geometric representations 1970 [Marr, Nishihara ’ 78] 1980 [ Brooks ’81] [Pentland ’ 86] [ Lowe ’87] 1990 [Koller, Daniilidis, Nagel ’93] [Sullivan, Worrall, Ferryman ’ 95] [Haag, Nagel ’ 99] 2000 2010 3 Zeeshan Zia
Revival of 3D geometric representations 1970 [Marr, Nishihara ’ 78] 1980 [ Brooks ’81] [Pentland ’ 86] [ Lowe ’87] 1990 [Koller, Daniilidis, Nagel ’93] [Sullivan, Worrall, Ferryman ’ 95] [Haag, Nagel ’ 99] 2000 [Hoiem, Efros , Hebert ’08] [Ess, Leibe, Schindler, Van Gool ’09] [Wang, Gould, Koller ’ 10] [Hedau, Hoiem, Forsyth ’ 10] [Barinova, Lempitsky, Tretyak, Kohli ’ 10] [Gupta, Efros, Hebert ’ 10] [Wojek, Roth, Schindler, Schiele ’ 10] 2010 3 Zeeshan Zia
Related work in viewpoint invariant detection Multiple, viewpoint dependent representations (connected in different ways) [Thomas et al., ’06] 1) [Yan, Khan, Shah ’07] [Ozuysal, Lepetit, Fua ’ 09] [Nachimson, Basri ’ 09] [Su, Sun, Fei-Fei, Savarese ’ 09] [Gu, Ren ’ 10] [Stark, Goesele, Schiele ’ 10] 1) 2) Explicit 3D geometry representation [Liebelt, Schmid ’ 10] 2) [Sun, Xu, Bradski, Savarese ’ 10] [Gupta, Efros, Hebert ’ 10] [Chen, Kim, Cipolla ‘10] [Gupta, Satkin, Efros, Hebert ’11] 4 Zeeshan Zia
Overview Simplify 3D Active Shape Model PCA 3D CAD Models 5 Zeeshan Zia
Overview Simplify 3D Active Shape Model PCA 3D CAD Models Render Positive examples (per part) 5 Zeeshan Zia
Overview Simplify 3D Active Shape Model PCA 3D CAD Models Render Positive examples (per part) AdaBoost Negative examples (background) 5 Zeeshan Zia
Overview Simplify 3D Active Shape Model PCA 3D CAD Models Render Positive examples (per part) AdaBoost Negative examples (background) Detection maps Test image 5 Zeeshan Zia
Overview Simplify 3D Active Shape Model PCA Inference 3D CAD Models Render Positive examples (per part) AdaBoost Negative examples (background) Detection maps Test image 5 Zeeshan Zia
Representation: 3D geometry Simplified 3D wireframes : fixed number of vertices 6 Zeeshan Zia
Learning: 3D geometry Eigen-Cars Principal Components Analysis (PCA) Tightly constrained global geometry 7 Zeeshan Zia
Representation: Local appearance Accurate foreground shape Very cheap training data, dense sampling of viewpoints! 8 Zeeshan Zia
Learning: Local appearance Dense Shape Context features [Belongie , Malik. ’00] AdaBoost classifiers (per part-viewpoint) - + … … Annotated vertices are our ‘parts’. Related work: [Andriluka, Roth, Schiele ’09] 9 Zeeshan Zia
Inference Test Image 10 Zeeshan Zia
Inference Test Image Detection … … maps 10 Zeeshan Zia
Inference Test Image Detection … … maps Sample 3D wireframes, project, compute image likelihood … … 10 Zeeshan Zia
Inference Detection image evidence Projection matrix … … maps local part scale recognition hypothesis part likelihood shape of wireframe self-occlusion indicator camera focal length Sample 3D cars, project, compute image likelihood viewpoint parameters, azimuth and elevation image space translation and scaling … … 11 Zeeshan Zia
Experimental evaluation – Test Dataset Evaluations on 3D Object Classes dataset [Savarese et al., 2007] Car class (8 azimuth angles, 2 elevation angles, 3 distances, varying backgrounds) – 240 images, 5 cars 12 Zeeshan Zia
Experimental evaluation - Training 38 3D CAD models 36 vertices as model points, 20 annotations per model (due to symmetry). Separate local part shape detectors trained from: - 72 different azimuth angles, - 2 different elevation angles (7.5 ° , 15 ° from ground plane) 13 Zeeshan Zia
Experimental evaluation - Initialization 20 ° Two initializations : Stark et al., 2010 (full system) True initial value (tight bounding box, rough azimuth) 14 Zeeshan Zia
Experimental evaluation - Inference 35 ° 35 ° 20 ° 14 Zeeshan Zia
Example wireframe fits Parts correctly localized Full system: 74.2% True initial value: 83.4% 15 Zeeshan Zia
Fine-grained 3D geometry estimation Accurate estimation of closest 3D CAD model, camera parameters, and ground plane 16 Zeeshan Zia
Ultra-wide baseline matching UW-Baseline matching using only model fits (corresponding part locations) Impossible using interest point matching Related work: [Bao, Savarese ’11] 17 Zeeshan Zia
Ultra-wide baseline matching UW-Baseline matching using only model fits (corresponding part locations) Impossible using interest point matching Related work: [Bao, Savarese ’11] 18 Zeeshan Zia
Ultra-wide baseline matching No. of Part True initial Full Azimuth Image SIFT detections value system Difference Pairs only 45 ° 53 91% 55% 2% 27% 90 ° 35 91% 60% 0% 27% 135 ° 29 69% 52% 0% 10% 180 ° 17 59% 41% 0% 24% Correct fit = Sampson error < E max on ground truth correspondences 3D Geometric model improves significantly over part detections only 19 Zeeshan Zia
Multiview recognition Rescored hypotheses Good 2D localization 20 Zeeshan Zia
Continuous viewpoint estimation Average Average Total True % correct error error Images Positives azimuth azimuth elevation 4.2 ° 4.0 ° Stark et al., 2010 48 46 67.4% 3.8 ° 3.6 ° Full system 48 45 73.3% 4.2 ° 3.6 ° True initial value* 48 48 89.6% Comparison against ground truth pose, manually labeled. Full system improves 6% over Stark et al., 2010. * Approximate pose initialization quantized to 45 ° steps 21 Zeeshan Zia
Conclusion 3D deformable object class model have potential for accurate geometric reasoning on scene level. - accurate object localization - geometric parts in 2D - 3D pose estimation Novel application examples - fine-grained object categorization - ultra-wide baseline matching Future extensions - efficient multi-class methods for part likelihoods - analyze importance of geometric model vs. local appearance - occlusion invariance 22 Zeeshan Zia
OLD SLIDES
Learning: 3D Geometry any wireframe mean wireframe weight of k th principal component standard deviation of j th principal component Eigen-Cars direction of j th principal component residual (if r < m) Zeeshan Zia
Part localization correct localization ~ localized within 4% of car length from ground truth Zeeshan Zia
Experimental evaluation - Inference 35 ° 35 ° 20 ° 14 Zeeshan Zia
Experimental evaluation - Inference 35 ° 35 ° 20 ° 14 Zeeshan Zia
Recommend
More recommend
