Robust Camera Location Estimation by Convex Programming sil and - PowerPoint PPT Presentation

Robust Camera Location Estimation by Convex Programming sil † and Amit Singer ‡ Onur ¨ Ozye¸ † INTECH Investment Management LLC 1 ‡ PACM and Department of Mathematics, Princeton University DIMACS Workshop on Distance Geometry 07/28/2016, DIMACS Center, Rutgers University 1 Work conducted as part of ¨ Ozye¸ sil’s Ph.D. work at Princeton University O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 1 / 19

Table of contents Structure from Motion (SfM) Problem 1 The Abstract Problem 2 Well-posedness of the Location Estimation Problem 3 Robust Location Estimation from Pairwise Directions 4 Robust Pairwise Direction Estimation 5 Experimental Results 6 O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 2 / 19

SfM Problem Structure from Motion (SfM) Problem Given a collection of 2 D photos of a 3 D object, recover the 3D structure by estimating the camera motion , i.e. camera locations and orientations 3D Structure ? Camera Locations = ⇒ We are primarily interested in the camera location estimation part O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 3 / 19

SfM Problem Structure from Motion Classical Approach Find corresponding points between images, estimate relative poses Estimate camera orientations and locations , i.e. camera motion Estimate the 3 D structure (e.g., by reprojection error minimization) O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 4 / 19

SfM Problem Structure from Motion Classical Approach Find corresponding points between images, estimate relative poses Estimate camera orientations and locations , i.e. camera motion Estimate the 3 D structure (e.g., by reprojection error minimization) Previous Methods Incremental methods : Incorporate images one by one (or in small groups) to maintain efficiency ⇒ prone to accumulation of errors Joint structure and motion estimation : Computationally hard, usually non-convex methods, no guarantees of convergence to global optima Orientation estimation methods : Relatively stable and efficient solvers O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 4 / 19

SfM Problem Structure from Motion Classical Approach Find corresponding points between images, estimate relative poses Estimate camera orientations and locations , i.e. camera motion Estimate the 3 D structure (e.g., by reprojection error minimization) Previous Methods Incremental methods : Incorporate images one by one (or in small groups) to maintain efficiency ⇒ prone to accumulation of errors Joint structure and motion estimation : Computationally hard, usually non-convex methods, no guarantees of convergence to global optima Orientation estimation methods : Relatively stable and efficient solvers Global Location Estimation Ill-conditioned problem (because of undetermined relative scales ) Current methods: Usually not well-formulated, not stable, inefficient O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 4 / 19

The Abstract Problem Problem: Location Estimation from Pairwise Directions Estimate the locations t 1 , t 2 , . . . , t n ∈ R d , for arbitrary d ≥ 2 , from a subset of (noisy) measurements of the pairwise directions, where the direction between t i and t j is given by the unit norm vector γ ij : t i − t j γ ij = � t i − t j � Pairwise Directions Locations t 1 γ 12 γ 1 3 γ 24 γ 1 5 t 2 γ 1 5 γ 12 t 5 γ 2 5 γ 36 γ 2 5 t 6 γ 1 3 γ 45 γ 4 6 γ 36 t 3 γ 24 γ 45 γ 4 6 t 4 A (noiseless) instance in R 3 , with n = 6 locations and m = 8 pairwise directions. O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 5 / 19

Well-posedness Well-posedness of the Location Estimation Problem • We represent the total pairwise information using a graph G t = ( V t , E t ) and endow each edge ( i, j ) with the direction measurement γ ij O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 6 / 19

Well-posedness Well-posedness of the Location Estimation Problem • We represent the total pairwise information using a graph G t = ( V t , E t ) and endow each edge ( i, j ) with the direction measurement γ ij Fundamental Questions Is the problem well-posed, i.e. do we have enough information to estimate the locations { t i } i ∈ V t stably (or, exactly in the noiseless case) ? O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 6 / 19

Well-posedness Well-posedness of the Location Estimation Problem • We represent the total pairwise information using a graph G t = ( V t , E t ) and endow each edge ( i, j ) with the direction measurement γ ij Fundamental Questions Is the problem well-posed, i.e. do we have enough information to estimate the locations { t i } i ∈ V t stably (or, exactly in the noiseless case) ? What does well-posedness depend on, is it a (generic) property of G t ? O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 6 / 19

Well-posedness Well-posedness of the Location Estimation Problem • We represent the total pairwise information using a graph G t = ( V t , E t ) and endow each edge ( i, j ) with the direction measurement γ ij Fundamental Questions Is the problem well-posed, i.e. do we have enough information to estimate the locations { t i } i ∈ V t stably (or, exactly in the noiseless case) ? What does well-posedness depend on, is it a (generic) property of G t ? Can it be decided efficiently? O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 6 / 19

Well-posedness Well-posedness of the Location Estimation Problem • We represent the total pairwise information using a graph G t = ( V t , E t ) and endow each edge ( i, j ) with the direction measurement γ ij Fundamental Questions Is the problem well-posed, i.e. do we have enough information to estimate the locations { t i } i ∈ V t stably (or, exactly in the noiseless case) ? What does well-posedness depend on, is it a (generic) property of G t ? Can it be decided efficiently? What can we do if an instance is not well-posed? O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 6 / 19

Well-posedness Well-posedness of the Location Estimation Problem • We represent the total pairwise information using a graph G t = ( V t , E t ) and endow each edge ( i, j ) with the direction measurement γ ij Fundamental Questions Is the problem well-posed, i.e. do we have enough information to estimate the locations { t i } i ∈ V t stably (or, exactly in the noiseless case) ? What does well-posedness depend on, is it a (generic) property of G t ? Can it be decided efficiently? What can we do if an instance is not well-posed? O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 6 / 19

Well-posedness Well-posedness of the Location Estimation Problem • We represent the total pairwise information using a graph G t = ( V t , E t ) and endow each edge ( i, j ) with the direction measurement γ ij Fundamental Questions Is the problem well-posed, i.e. do we have enough information to estimate the locations { t i } i ∈ V t stably (or, exactly in the noiseless case) ? What does well-posedness depend on, is it a (generic) property of G t ? Can it be decided efficiently? What can we do if an instance is not well-posed? ⇒ Well-posedness was previously studied in various contexts, under the general title of parallel rigidity theory . O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 6 / 19

Well-posedness Well-posedness: Simple Examples Consider the following noiseless instance: γ 34 γ 13 1 2 γ 35 γ 23 γ 12 + 3 γ 45 5 4 G t = ( V t , E t ) { γ ij } ( i,j ) ∈ E t O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 7 / 19

Well-posedness Well-posedness: Simple Examples Consider the following noiseless instance: γ 34 t 1 γ 13 1 2 γ 35 t 2 γ 23 ⇒ γ 12 + 3 t 3 γ 45 5 4 G t = ( V t , E t ) { γ ij } ( i,j ) ∈ E t t 5 t 4 O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 7 / 19

Well-posedness Well-posedness: Simple Examples Consider the following noiseless instance: γ 34 t 1 t 1 γ 13 1 2 γ 35 t 2 t 2 γ 23 ⇒ γ 12 + 3 t 3 γ 45 t 3 5 4 t ′ t ′ 5 4 G t = ( V t , E t ) { γ ij } ( i,j ) ∈ E t t 5 t 4 O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 7 / 19

Well-posedness Well-posedness: Simple Examples Consider the following noiseless instance: γ 34 t 1 t 1 γ 13 1 2 γ 35 t 2 t 2 γ 23 ⇒ γ 12 + 3 t 3 γ 45 t 3 5 4 t ′ t ′ 5 4 G t = ( V t , E t ) { γ ij } ( i,j ) ∈ E t t 5 t 4 Well-posedness depends on the dimension: t 2 t 1 ⇒ Well-posed in R 3 , but not in R 2 t 4 t 3 O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 7 / 19

Well-posedness Main Results of Parallel Rigidity Theory Unique Realization of Locations Unique solution exists if and only if the formation is parallel rigid . O. Ozyesil and A. Singer Robust Camera Location Estimation Workshop on Distance Geometry, 2016 8 / 19