Adiabatic quantum optimization applied to the stereo matching problem William Cruz-Santos 1 Salvador E. Venegas Andraca 2 Marco Lanzagorta 3 1 Computer Engineering, CU-UAEM Valle de Chalco, Edo. de M´ exico, M´ exico 2 Quantum Information Processing Group at Tecnol´ ogico de Monterrey, Escuela de Ciencias e Ingenier´ ıa 3 US Naval Research Laboratory, 4555 Overlook Ave. SW Washington DC 20375, USA Quantum Techniques in Machine Learning (QTML) Verona, 6-8 November 2017 1 / 15
1 Introduction In this talk, ⋄ I would like to discuss the quantum annealing approach to the stereo matching problem: Problem formulation → QUBO → Embedding into the hardware The stereo matching is an important problem in computer vision. ⋄ We discuss the advantages and limitations of our quantum annealing formulation and future applications. This presentation is based on Salvador E. Venegas-Andraca, William Cruz-Santos, Catherine McGeoch, and Marco Lanzagorta, “A cross-disciplinary introduction to quantum annealing-based algorithms”, to be published in contemporary physics. We thank to USRA for give us access to the D-Wave 2X. 2 / 15
Contents Section 2: The stereo matching problem Section 3: Multi-labeling problem formulation Section 4: Mapping to QUBO Section 5: Work in progress Section 6: Conclusions 3 / 15
2 The stereo matching problem (1) left image right image disparity map http://vision.middlebury.edu/stereo/data/scenes2003/ The depth Z of a point P projected onto the left and right image planes with column positions x l and x r respectively, is T Z = f . x r − x l � �� � disparity where f is the focal length and T is the baseline. 4 / 15
2 The stereo matching problem (2) left image right image disparity map http://vision.middlebury.edu/stereo/data/scenes2003/ Problem (stereo matching) Given a stereo pair images ( I l , I r ) find for every pixel I l ( x 1 , y ) in the left image, its corresponding projection I r ( x 2 , y ) in the right image. 5 / 15
3 Multi-labeling problem formulation Let P be the image domain in the left image and L be the set of labels or disparities. A labeling is a map l : P → L , ∀ p ∈ P : l ( p ) = l p . The cost of a labeling l is defined as � � E ( l ) = D p ( l p ) + V pq ( l p , l q ) p ∈P { p,q }∈N where − D p models the penalty of assigning label l p to the pixel p , − V pq models the cost of assigning l p to p and l q to q , with p and q being adjacent pixels, and − N is the set of neighbor pairs { p, q } . Remark The stereo matching problem is equivalent to finding a labeling l such that E ( l ) is minimum. The minimum E ( l ) can be obtained by finding a multi-cut with minimum cost in a weighted graph. 6 / 15
3.1 Graph construction (1) Let G = ( V, E, c ) be a weighted graph with two special vertices s, t ∈ V . Let L = { 0 , . . . , L } be a set of possible disparities or labels. The graph G is constructed as follows: 1. For each pixel p there is a chain of L + 2 vertices, said p 0 , p 1 , . . . , p L +2 joined by edges or t-links as e sp 0 , e p 0 p 1 , . . . , e p L +2 t . s t p 0 p L+2 p 1 2. For each pair of neighbor pixels p and q with corresponding chains p 0 , p 1 , . . . , p L +2 and q 0 , q 1 , . . . , q L +2 joined by edges or n-links as e p 0 q 0 , e p 1 q 1 , . . . , e p L +2 q L +2 . p 0 p 1 p L+2 s t q L+2 q 0 q 1 7 / 15
3.1 Graph construction (2) − Graph topology for a 5 × 5 image. − A 4-neighborhood is assumed for every pixel. − A cut separating the vertices s and t severs t -links as well as n -links. − The severed t -links defines the labels assigned to each pixel. − If a cut severs the t -link t p j = { p j , p j +1 } , with 1 ≤ j < L − 1 , for a pixel p , then the assigned label to p is j . 8 / 15
3.1 Graph construction (3) The cost of a t -link t p d = { p d , p d − 1 } , with 1 ≤ d < L , can be defined as c ( t p d ) = | I l ( x, y ) − I r ( x − d, y ) | 2 + C. The cost function V pq depends on the difference between their labels l p and l q defined as V pq ( l p , l q ) = λ pq | l p − l q | . Given an n -link n p k q k for two neighbor pixels p and q for all 1 ≤ k < L , the cost of n p k q k is equal to c ( n p k q k ) = λ pq where − C is a constant chosen sufficiently large enough in order to ensure that exactly one t-link is severed by the cut for each pixel − λ pq is a weighting factor for setting the relative importance of the smoothness term. 9 / 15
4 Mapping to QUBO � σ x H ( τ ) = A ( s ) i + B ( s ) H 1 i N N � � h i σ z J ij σ z i σ z H 1 = i + j i j>i − → [Lanting et al, 2014] 10 / 15
4 Mapping to QUBO Given a weighted graph G = ( V, E, c ) and a pair ( s, t ) . Mapping: − For each { u, w } ∈ E , y uw = 1 (0) if { u, w } is (not) selected for a cut. − For each v ∈ V , x v = 1 (0) if v is (not) in U where U is a subset of V . Construction: Let f qubo be defined as G � f qubo y uw · c ( { u, w } ) + α · penalty ( x v , y uw , s, t ) = G { u,w }∈ E where � penalty = ¬ ( x s ⊕ x t ) + ( x u ⊕ x w ) ⊕ y uw { u,w }∈ E � = 1 − x s − x t + 2 x s x t + ( x u + x w + y uw − { u,w }∈ E 2 x u x w − 2 x u y uw − 2 x w y uw + 4 x u x w y uw ) � �� � Easily reduced 11 / 15
Using the Ishikawa method [Ishikawa, 2011] we obtain penalty = 1 − x s − x t + 2 x s x t + � ( x u + x w + y uw + 2 x u x w + 2 x u y uw + 2 x w y uw + { u,w }∈ E 4(1 − x u − x w − y uw ) ) . z uw ���� Ancilla var The QUBO function f G uses | V | + | E | + m logical variables where 0 ≤ m ≤ | E | . α is upper bounded by � { u,w }∈ E y uw . 12 / 15
5 Work in progress Figure: A composition strategy. Example Tsukuba stereo pair: Image size 384 × 288 pixels Disparity range L = 12 | V | = 384 × 288 × L + 2 = 1 , 327 , 106 Finding the disparity for each 15 × 15 window. 13 / 15
6 Conclusions − We have proposed a composition strategy − The stereo matching is a special case of minimum multicut problem − We will use the D-Wave 2X for comparisons − Minimize the number of variables in the QUBO formulation − Future application in computer vision such as in autonomous navigation 14 / 15
Thanks for your kind attention! We thank to USRA ( Universities Space Research Association ) for give us access to the D-Wave 2X. Contact: wdelacruzd@uaemex.mx 15 / 15
Recommend
More recommend
