Adiabatic quantum optimization applied to the stereo matching - - PowerPoint PPT Presentation

Adiabatic quantum optimization applied to the stereo matching problem

William Cruz-Santos1 Salvador E. Venegas Andraca2 Marco Lanzagorta3

1Computer Engineering, CU-UAEM Valle de Chalco, Edo. de M´

exico, M´ exico

2Quantum Information Processing Group at Tecnol´

• gico de Monterrey, Escuela de Ciencias e Ingenier´

ıa

3US Naval Research Laboratory, 4555 Overlook Ave. SW Washington DC 20375, USA

Quantum Techniques in Machine Learning (QTML) Verona, 6-8 November 2017

1 / 15

1 Introduction In this talk, ⋄ I would like to discuss the quantum annealing approach to the stereo matching problem: Problem formulation → QUBO → Embedding into the hardware The stereo matching is an important problem in computer vision. ⋄ We discuss the advantages and limitations of our quantum annealing formulation and future applications.

This presentation is based on Salvador E. Venegas-Andraca, William Cruz-Santos, Catherine McGeoch, and Marco Lanzagorta, “A cross-disciplinary introduction to quantum annealing-based algorithms”, to be published in contemporary physics. We thank to USRA for give us access to the D-Wave 2X.

2 / 15

Contents Section 2: The stereo matching problem Section 3: Multi-labeling problem formulation Section 4: Mapping to QUBO Section 5: Work in progress Section 6: Conclusions

3 / 15

2 The stereo matching problem (1) left image right image disparity map

http://vision.middlebury.edu/stereo/data/scenes2003/

The depth Z of a point P projected onto the left and right image planes with column positions xl and xr respectively, is Z = f T xr − xl

disparity

. where f is the focal length and T is the baseline.

4 / 15

2 The stereo matching problem (2) left image right image disparity map

http://vision.middlebury.edu/stereo/data/scenes2003/

Problem (stereo matching)

Given a stereo pair images (Il, Ir) find for every pixel Il(x1, y) in the left image, its corresponding projection Ir(x2, y) in the right image.

5 / 15

3 Multi-labeling problem formulation Let P be the image domain in the left image and L be the set of labels or

• disparities. A labeling is a map

l : P → L, ∀p ∈ P : l(p) = lp. The cost of a labeling l is defined as E(l) =

• p∈P

Dp(lp) +

• {p,q}∈N

Vpq(lp, lq) where − Dp models the penalty of assigning label lp to the pixel p, − Vpq models the cost of assigning lp to p and lq to q, with p and q being adjacent pixels, and − N is the set of neighbor pairs {p, q}.

Remark

The stereo matching problem is equivalent to finding a labeling l such that E(l) is minimum. The minimum E(l) can be obtained by finding a multi-cut with minimum cost in a weighted graph.

6 / 15

3.1 Graph construction (1) Let G = (V, E, c) be a weighted graph with two special vertices s, t ∈ V . Let L = {0, . . . , L} be a set of possible disparities or labels. The graph G is constructed as follows:

• 1. For each pixel p there is a chain of L + 2 vertices, said p0, p1, . . . , pL+2

joined by edges or t-links as esp0, ep0p1, . . . , epL+2t. p0 p1 pL+2 s t

• 2. For each pair of neighbor pixels p and q with corresponding chains

p0, p1, . . . , pL+2 and q0, q1, . . . , qL+2 joined by edges or n-links as ep0q0, ep1q1, . . . , epL+2qL+2. q0 q1 qL+2 s t p0 p1 pL+2

7 / 15

3.1 Graph construction (2) − Graph topology for a 5 × 5 image. − A 4-neighborhood is assumed for every pixel. − A cut separating the vertices s and t severs t-links as well as n-links. − The severed t-links defines the labels assigned to each pixel. − If a cut severs the t-link tpj = {pj, pj+1}, with 1 ≤ j < L − 1, for a pixel p, then the assigned label to p is j.

8 / 15

3.1 Graph construction (3) The cost of a t-link tpd = {pd, pd−1}, with 1 ≤ d < L, can be defined as c(tpd) = |Il(x, y) − Ir(x − d, y)|2 + C. The cost function Vpq depends on the difference between their labels lp and lq defined as Vpq(lp, lq) = λpq|lp − lq|. Given an n-link npkqk for two neighbor pixels p and q for all 1 ≤ k < L, the cost of npkqk is equal to c(npkqk) = λpq where − C is a constant chosen sufficiently large enough in order to ensure that exactly one t-link is severed by the cut for each pixel − λpq is a weighting factor for setting the relative importance of the smoothness term.

9 / 15

4 Mapping to QUBO H(τ) = A(s)

• i

σx

i + B(s)H1

H1 =

N

• i

hiσz

i + N

• j>i

Jijσz

i σz j

− →

[Lanting et al, 2014]

10 / 15

4 Mapping to QUBO Given a weighted graph G = (V, E, c) and a pair (s, t). Mapping: − For each {u, w} ∈ E, yuw = 1 (0) if {u, w} is (not) selected for a cut. − For each v ∈ V , xv = 1 (0) if v is (not) in U where U is a subset of V . Construction: Let f qubo

G

be defined as f qubo

G

=

• {u,w}∈E

yuw · c({u, w}) + α · penalty(xv, yuw, s, t) where penalty = ¬(xs ⊕ xt) +

• {u,w}∈E

(xu ⊕ xw) ⊕ yuw = 1 − xs − xt + 2xsxt +

• {u,w}∈E

(xu + xw + yuw − 2xuxw − 2xuyuw − 2xwyuw + 4xuxwyuw

• Easily reduced

)

11 / 15

Using the Ishikawa method [Ishikawa, 2011] we obtain penalty = 1 − xs − xt + 2xsxt +

• {u,w}∈E

(xu + xw + yuw + 2xuxw + 2xuyuw + 2xwyuw + 4(1 − xu − xw − yuw) zuw

• Ancilla var

). The QUBO function fG uses |V | + |E| + m logical variables where 0 ≤ m ≤ |E|. α is upper bounded by

{u,w}∈E yuw.

12 / 15

5 Work in progress

Figure: A composition strategy.

Example Tsukuba stereo pair: Image size 384 × 288 pixels Disparity range L = 12 |V | = 384 × 288 × L + 2 = 1, 327, 106 Finding the disparity for each 15 × 15 window.

13 / 15

6 Conclusions − We have proposed a composition strategy − The stereo matching is a special case of minimum multicut problem − We will use the D-Wave 2X for comparisons − Minimize the number of variables in the QUBO formulation − Future application in computer vision such as in autonomous navigation

14 / 15

Thanks for your kind attention! We thank to USRA (Universities Space Research Association) for give us access to the D-Wave 2X. Contact: wdelacruzd@uaemex.mx

15 / 15