Global MAP-Optimality by Shrinking the Combinatorial Search Area with Convex Relaxation Bogdan Savchynskyy, J¨ org Kappes, Paul Swoboda, Christoph Schn¨ orr Heidelberg Collaboratory for Image Processing (HCI) University of Heidelberg Acknowledgement: Thanks to A. Shekhovtsov, B. Flach, T. Werner, K. Antoniuk, V. Franc from CMP of TU Prague for the extreme patience and fruitful discussions 1/35
MRF Energy Minimization ÿ ÿ x P X E p x q : “ min θ v p x v q ` θ uv p x u , x v q min x P X v P V uv P E Segmentation [Rother et al. 2004], [Nowozin, Lampert 2010] Multi-camera stereo [Kolmogorov, Zabih 2002] Stereo and Motion [Kim et al. 2003] Clustering [Zabih, Kolmogorov. 2004] Medical imaging [Raj et al. 2007] Pose Estimation [Bergtholdt et al. 2010], [Bray et al. 2006] . . . Computer Vision energy minimization benchmarks: [Szeliski et al. 2008], [Kappes et al. CVPR, 2013] 2/35
MRF Energy Minimization ÿ ÿ x P X E p x q : “ min θ v p x v q ` θ uv p x u , x v q min x P X v P V uv P E θ u p x u q θ v p x v q θ uv p x u , x v q x u x v u v graph p V , E q 3/35
Integer LP Formulation ÿ ÿ ÿ ÿ θ v p x v q µ v p x v q ` θ uv p x u , x v q µ uv p x u , x v q min µ ě 0 v P V x v P X v uv P E x u , x v P X uv ř x v P V µ v p x v q “ 1 , v P V s.t. ř x v P V µ uv p x u , x v q “ µ u p x u q , x u P X u , uv P E ř x u P V µ uv p x u , x v q “ µ v p x v q , x v P X v , uv P E . µ P t 0 , 1 u N µ uv p x u , x v q 0 1 µ u p x u q µ v p x v q 1 0 0 0 u v 4/35
Integer LP Formulation ÿ ÿ ÿ ÿ θ v p x v q µ v p x v q ` θ uv p x u , x v q µ uv p x u , x v q min µ ě 0 v P V x v P X v uv P E x u , x v P X uv ř x v P V µ v p x v q “ 1 , v P V s.t. ř x v P V µ uv p x u , x v q “ µ u p x u q , x u P X u , uv P E ř x u P V µ uv p x u , x v q “ µ v p x v q , x v P X v , uv P E . µ P t 0 , 1 u N µ P r 0 , 1 s N µ uv p x u , x v q 0 . 4 1 µ u p x u q µ v p x v q 0 . 6 0 0 . 0 0 u v 5/35
LP Relaxation: typical solution color segmentation problem integer and fractional labelings Is the integer part of the solution correct? In general - NO! In practice - mostly YES. How can it be exploited to find an optimal integer solution? 6/35
Related Approach: Partial Optimality QPBO:[Hammer et al. 1984],[Boros, Hammer 2002],[Rother et al. 2007], [Kohli et al. 2008],[Windheuser et al. 2012], [Kahl,Strandmark 2012]; Submodular relaxation:[Kovtun 2003], [Kovtun PhD Thesis 2005], [Shekhovtsov,Hlavaˇ c 2011]; LP relaxation:[Swoboda et al. 2013, 2014],[Shekhovtsov 2014]. ñ integer and fractional solved partial solution labeling loooooooooooooomoooooooooooooon loooooooooooooooooooooooooooooooomoooooooooooooooooooooooooooooooon [Kovtun 2003] Our approach 7/35
Related Approach: Partial Optimality QPBO:[Hammer et al. 1984],[Boros, Hammer 2002],[Rother et al. 2007], [Kohli et al. 2008],[Windheuser et al. 2012], [Kahl,Strandmark 2012]; Submodular relaxation:[Kovtun 2003], [Kovtun PhD Thesis 2005], [Shekhovtsov,Hlavaˇ c 2011]; LP relaxation:[Swoboda et al. 2013, 2014],[Shekhovtsov 2014]. ñ integer and fractional NOT solved partial solution labeling loooooooooooooomoooooooooooooon loooooooooooooooooooooooooooooooomoooooooooooooooooooooooooooooooon [Kovtun 2003] Our approach 8/35
Algorithm Idea 0) Initialize: Identify LP and ILP parts. t) Iterate till agreement on the border : Ñ Ñ Ñ Ñ solve ILP ( + ) increase ILP check subproblem + and LP ( + ) agreement separately if disagree on the border 9/35
From Idea to Algorithm Is agreement on the border sufficient for optimality? How to select the initial LP/ILP splitting? How to encourage agreement on the border? How to avoid re-solving the LP part? (Do we need to solve the LP relaxation to optimality?) 10/35
Is agreement on the border sufficient for optimality? 2 2 0 0 ´ 3 0 0 0 0 0 Counterexample due to A. Shekhovtsov 11/35
Is consistency on the border sufficient for optimality? 2 0 ´ 3 0 0 0 Counterexample due to A. Shekhovtsov 12/35
Is consistency on the border sufficient for optimality? 2 0 ´ 3 0 0 0 Counterexample due to A. Shekhovtsov 13/35
Is consistency on the border sufficient for optimality? 2 2 0 0 ´ 3 0 0 0 0 0 Counterexample due to A. Shekhovtsov 14/35
Background: Reparametrization (Equivalent transformations) ÿ ÿ ÿ ÿ θ φ ˜ θ φ ˜ θ v p x v q ` θ uv p x u , x v q ” v p x v q ` uv p x u , x v q v P V uv P E v P V uv P E ´ φ u , v p x u q θ φ ˜ u p x u q ` φ u , v p x u q θ φ ˜ uv p x u , x v q x u ô u v u v 15/35
Background: Reparametrization, Dual problem Primal: E p x q “ ř θ v p x v q ` ř θ uv p x u , x v q min x v P V uv P E θ φ θ φ ˜ ˜ ř ` ř Dual: D p φ q “ max min v p x v q min uv p x uv q x v x uv φ v P V uv P E ´ φ u , v p x u q ˜ θ φ u p x u q ` φ u , v p x u q θ φ ˜ uv p x u , x v q x u ô u v u v 16/35
Background: Reparametrization, Dual problem θ φ θ φ ˜ ˜ ř ` ř Primal: E p x q “ min v p x v q uv p x u , x v q x v P V uv P E θ φ ˜ θ φ ˜ Dual: D p φ q “ ř v p x v q ` ř uv p x uv q max min min φ x v x uv v P V uv P E D p φ q ď E p x q ´ φ u , v p x u q θ φ ˜ u p x u q ` φ u , v p x u q θ φ ˜ uv p x u , x v q x u ô u v u v 17/35
Background: Arc Consistency ÿ ÿ θ φ ˜ θ φ ˜ Dual: D p φ q “ max v p x v q ` uv p x uv q min min x v x uv φ v P V uv P E loooomoooon looooomooooon γ v γ uv θ φ ˜ θ φ ˜ ˜ v p x v q ˜ v p x v q ˜ θ φ v p x v q ˜ θ φ θ φ uv p x u , x v q θ φ uv p x u , x v q uv p x u , x v q γ u γ u γ u γ uv γ uv γ u γ v γ v γ v γ uv γ uv u v u v u v strict arc consistency arc consistency strict arc consistency 18/35
Background: Trivial Problem, Strict Arc Consistency Strict arc consistency in all nodes Theorem. ó the non-relaxed problem is solved. Proof. Strict arc consistency ñ D p φ q “ E p x ˚ q , x ˚ consists of γ v , γ uv . D p φ q ď E p x q ñ x ˚ is the solution. ˜ θ φ v p x v q ˜ θ φ uv p x u , x v q γ u γ uv γ v u v strict arc consistency 19/35
Consistency on the border sufficient for optimality. Theorem. Let θ φ be strictly arc consistent on + . Then if LP ( + ) and ILP ( + ) solutions agree on the border ( ) their concatenation is globally optimal. 0 . 2 0 . 2 0 . 4 0 . 6 0 0 0 0 0 0 20/35
LP Relaxation: typical (approximate) solution Blue - strictly arc consistent, red - otherwise. 21/35
Algorithm 0) Initialize: Solve LP relaxation and reparametrize: θ Ñ ˜ θ φ ’Blue’ = the strictly arc consistent nodes. t) Iterate till agreement on the border: Ñ Ñ Ñ Ñ increase ILP apply ILP solver check subproblem + to + agreement if disagree on the border 22/35
Why reparametrize? Reparametrization provides: optimality condition ( “ consistency on border ( )) initial splitting criterion (to and ) encouraging of border consistency Optimal labels ”vote“ for themselves in both LP ( + ) and ILP ( + ) subproblems potential speed-up of combinatorial solvers Acts as LP pre-solving Moreover... An LP solver needs to be executed only once Because due to strict consistency local decisions are optimal: nodes removal does not change the remaining part of the solution Suboptimal reparametrization can be used as well Because we did not employ optimality of the reparametrization 23/35
Experimental Evaluation: OpenGM Library and Benchmark Open Library for Graphical Models: inference algorithms; benchmark data; includes Middlebury MRF benchmark comparison tables; Just type ’OpenGM’ in Google... 24/35
Experimental Evaluation: OpenGM Library and Benchmark Open Library for Graphical Models: inference algorithms; benchmark data; includes Middlebury MRF benchmark comparison tables; Just type ’OpenGM’ in Google... Our code is freely available as a part of OpenGM! 25/35
Experimental Evaluation: Methods Algorithms, available in OpenGM: We used: TRW-S [Kolmogorov 2005] as LP solver CPLEX [IBM] as ILP solver. 26/35
Middlebury MRF Benchmark venus tsukuba family 434 ˆ 383 , 20 labels 384 ˆ 288 , 16 labels 752 ˆ 566 , 5 labels “ 3048043 “ 369218 E min “ 184813 E min E min E TRWS “ 3048296 E TRWS “ 369218 E TRWS “ 184825 27/35
Middlebury MRF Benchmark panorama teddy 450 ˆ 375 , 60 labels 1071 ˆ 480 , 7 labels 1 iteration of ILP 1 iteration of ILP = out of memory = out of memory 28/35
Middlebury MRF Benchmark Dataset Step (1) LP (TRWS) Step (3) ILP (CPLEX) | B | name it time, s it time, s min max E E tsukuba 250 186 369537 24 36 369218 130 656 venus 2000 3083 3048296 10 69 3048043 66 233 teddy 10000 14763 1345214 1 ´ ´ 2062 ´ family 10000 20156 184825 18 2 184813 11 109 pano ´ ´ ´ 10000 34092 169224 1 24474 Table : Results on Middlebury datasets 29/35
Color Segmentation: 26 Potts models Solved 30/35
Recommend
More recommend
