Phase synchronization An example of global optimality on manifolds - - PowerPoint PPT Presentation

Phase synchronization

An example of global optimality

• n manifolds

Nicolas Boumal, Inria & ENS Paris Joint work with Afonso Bandeira and Amit Singer

One-day workshop on Riemannian and nonsmooth optimization

• Sept. 25, 2015

Note for the reader

A tighter version of the theorems in these slides will appear in an update to the arxiv paper 1411.3272 (version 1 from

• Nov. 2014 is less tight).

The version in these slides was chosen as it is relatively simple to derive in the allotted time. In particular, the bound on 𝑨 − 𝑦 2 can be reduced to 𝑃(𝜏) and the tightness rate can be improved to 𝜏 ≤ 𝑑 ⋅ 𝑜1/4 as a result. Nicolas, September 28, 2015.

The goal: estimate individual orientations, from pairwise comparisons (up to global shift)

The goal: estimate individual orientations, from pairwise comparisons (up to global shift)

Estimate phases from relative info

Unknowns

𝑨1, … , 𝑨𝑜 ∈ ℂ, with 𝑨1 = ⋯ = 𝑨𝑜 = 1

Data

Noisy measurements of relative phases: 𝐷𝑗𝑘 = 𝑨𝑗𝑨

𝑘 ∗ + 𝜏𝑋 𝑗𝑘

ℂ1

𝑜

What does additive Gaussian noise mean here?

Noise affects the phase of the measurement

𝐷𝑗𝑘 = 𝑨𝑗𝑨

𝑘 ∗ + 𝜏𝑋 𝑗𝑘

Maximum likelihood estimation

𝑨 ∈ ℂ1

𝑜,

𝐷 = 𝑨𝑨∗ + 𝜏𝑋

Under Gaussian noise, the MLE solves a least-squares

min

𝑦∈ℂ1

𝑜

𝐷 − 𝑦𝑦∗

F 2

≡ max

𝑦∈ℂ1

𝑜 𝑦∗𝐷𝑦

𝑦 (the MLE) 𝑨 (the signal) ℓ2 ball of radius 12𝜏 𝑜 level sets of 𝑦∗𝐷𝑦

Lemma: The MLE 𝑦 is close to the signal 𝑨

The MLE is more likely than the signal: 𝑨∗𝐷𝑨 ≤ 𝑦∗𝐷𝑦. Hence, with 𝐷 = 𝑨𝑨∗ + 𝜏𝑋 and 𝑋 op ≤ 3 𝑜, 𝑜2 + 𝜏𝑨∗𝑋𝑨 ≤ 𝑨∗𝑦 2 + 𝜏𝑦∗𝑋𝑦 𝑜2 − 𝑨∗𝑦 2 ≤ 𝜏 𝑦∗𝑋𝑦 − 𝑨∗𝑋𝑨 ≤ 𝜏 ⋅ 2𝑜 𝑋 op ≤ 6𝜏𝑜3/2 Divide by 𝑜 + 𝑨∗𝑦 ≥ 𝑜. Implies: min

𝜄

𝑨 − 𝑦𝑓𝑗𝜄

2 2 = 2(𝑜 − |𝑨∗𝑦|) ≤ 12𝜏 𝑜

The MLE is NP-hard to compute

The problem max

𝑦∈ℂ1

𝑜 𝑦∗𝐷𝑦

has a quadratic cost 𝑦∗𝐷𝑦, and nonconvex quadratic constraints 𝑦𝑗 2 = 1.

Surprisingly, global optimality can be tested (sometimes)

Given 𝑦 ∈ ℂ1

𝑜, consider

𝑇 = 𝑇 𝑦 = ℜ ddiag 𝐷𝑦𝑦∗ − 𝐷 If 𝑇 is positive semidefinite, then 𝑦 is optimal: ∀𝑧 ∈ ℂ1

𝑜, 0 ≤ 𝑧∗𝑇𝑧 = 𝑦∗𝐷𝑦 − 𝑧∗𝐷𝑧

Optimization on manifolds finds a certified optimum quite often

Noise Level 𝜏 Number of phases 𝑜

Certified optimum: 𝑇 ≽ 0 Don’t know

How is that possible?

The problem is convex in a lifted space

Classic lifting trick: rewrite everything in terms of 𝑌 = 𝑦𝑦∗

max

𝑦∈ℂ1

𝑜 𝑦∗𝐷𝑦

The cost 𝑦∗𝐷𝑦 = Trace 𝑦∗𝐷𝑦 = Trace(𝐷𝑌) The constraints 𝑦𝑗 2 = 1 ⇔ 𝑦𝑗𝑦𝑗

∗ = 1 ⇔ 𝑌𝑗𝑗 = 1

The knot ∃𝑦: 𝑌 = 𝑦𝑦∗ ⇔ 𝑌 ≽ 0, rank 𝑌 = 1

Suggests a semidefinite relaxation

Recast the QCQP max

𝑦∈ℂ1

𝑜 𝑦∗𝐷𝑦

Into max

𝑌∈ℂ𝑜×𝑜 Trace(𝐷𝑌)

diag 𝑌 = 𝟐 𝑌 ≽ 0 rank 𝑌 = 1

Suggests a semidefinite relaxation

Relax the QCQP max

𝑦∈ℂ1

𝑜 𝑦∗𝐷𝑦

Into the SDP max

𝑌∈ℂ𝑜×𝑜 Trace(𝐷𝑌)

diag 𝑌 = 𝟐 𝑌 ≽ 0 rank 𝑌 = 1

The SDP seems tight roughly when Riemannian optimization succeeds

Noise Level 𝜏 Number of phases 𝑜

Tight: SDP solution has rank 1 Not tight

We prove the SDP has a unique solution of rank 1

This is with high probability for large 𝑜, And if 𝜏 ≤ 𝑑 ⋅ 𝑜1/6 (empirically, 𝑃(𝑜1/2) ok).

General idea: dual certification

Lemma:

𝑌 solves the SDP if and only if 𝑇 𝑌 = ℜ ddiag 𝐷𝑌 − 𝐷 ≽ 0 Proof via KKT conditions. Thus, the certificate works iff the SDP is tight.

max

𝑌∈ℂ𝑜×𝑜 Trace(𝐷𝑌)

diag 𝑌 = 𝟐 𝑌 ≽ 0

General idea: dual certification

Let 𝑦 be the MLE, solution of max

𝑦∈ℂ1

𝑜 𝑦∗𝐷𝑦

We aim to prove that 𝑇 𝑦𝑦∗ ≽ 0. Challenge: we don’t know 𝑦.

𝑦 (the MLE) 𝑨 (the signal) level sets of 𝑦∗𝐷𝑦

Step 1: characterize the MLE 𝑦

𝑦 is second-order critical, close to 𝑨.

Step 1: characterize the MLE 𝑦

The MLE problem lives on a smooth manifold max

𝑦∈ℂ1

𝑜 𝑦∗𝐷𝑦

𝑦 is critical simply if its projected gradient vanishes: 𝑦 is critical ⇔ 𝑇𝑦 = 0 𝑇 = 𝑇 𝑦 = ℜ ddiag 𝐷𝑦𝑦∗ − 𝐷

Step 1: characterize the MLE 𝑦

𝑇 = 𝑇 𝑦 = ℜ ddiag 𝐷𝑦𝑦∗ − 𝐷 𝑦 is critical ⇔ 𝑇𝑦 = 0 ⇔ diag 𝐷𝑦𝑦∗ is real If 𝑦 is also second-order critical, then 𝑇 is positive semidefinite on the tangent space. 𝑦 is second-order critical ⇒ diag 𝐷𝑦𝑦∗ ≥ 𝟐

Step 2: certify

Assuming 𝑦 is second-order critical and close to 𝑨, show that 𝑇 = ddiag 𝐷𝑦𝑦∗ − 𝐷 ≽ 0. For all 𝑣 ∈ ℂ𝑜 with 𝑣∗𝑦 = 0, seven lines give: 𝑣∗𝑇𝑣 ≥ 𝑣 2

2 𝑜 − 𝜏 21 𝑜 + 𝑋𝑦 ∞

(Used 𝑋 op ≤ 3 𝑜 again.)

Step 2: certify

Sufficient condition for tightness of the SDP: 𝑜 ≥ 𝜏 21 𝑜 + 𝑋𝑦 ∞ Note: 𝑦 and 𝑋 are not independent. Suboptimal argument (whp): 𝑋𝑦 ∞ ≤ 𝑋𝑨 ∞ + 𝑋 𝑦 − 𝑨

∞

≤ 𝑜 log 𝑜 + 𝑋 op 𝑦 − 𝑨 2 ≤ 𝑜 log 𝑜 + 3 12𝜏𝑜3/2

Theorem (ArXiv 1411.3272)

With high probability for large (finite) 𝑜, If 𝜏 ≤ 𝑑 ⋅ 𝑜1/6, Then a second-order critical point 𝑦 close to 𝑨 is optimal, with certificate 𝑇, And 𝑦𝑦∗ is the unique solution of the SDP.