One-bit compressed sensing with Gaussian circulant matrices Sjoerd - - PowerPoint PPT Presentation

One-bit compressed sensing with Gaussian circulant matrices

Sjoerd Dirksen (RWTH Aachen University) based on joint work with Hans Christian Jung and Holger Rauhut (RWTH Aachen) Supported by DFG project QuaCoSS Compressed Sensing and its Applications 2017 December 7, 2017

Sjoerd Dirksen 1 / 25

Quantized compressed sensing

Sjoerd Dirksen 2 / 25

Quantized compressed sensing

Goal: recover an s-sparse vector x ∈ Rn from quantized underdetermined linear measurements y = Q(Ax), A ∈ Rm×n, where Q : Rm → Am, A=quantization alphabet. Q quantizes each measurement to a finite bit string. Q can be

◮ memoryless, i.e., each measurement ai, x is quantized

independently (e.g. uniform scalar quantization with A = δZ).

◮ adaptive, i.e., quantize ai, x based on previous (quantized)

measurements (e.g. Σ-∆ quantization). Ideal design goals: minimal # measurements, # bits, energy consumption, computational cost, hardware cost.

Sjoerd Dirksen 3 / 25

One-bit compressed sensing

This talk: one-bit compressed sensing model y = sign(Ax + τ), (1) where

◮ sign is the sign function applied element-wise ◮ τ ∈ Rm is a vector consisting of thresholds (→ dithering). ◮ Memoryless if the τi are fixed or independent random,

adaptive if the τi are chosen adaptively. First considered for τ = 0 by Baraniuk and Boufounos ’09. In this case, can only recover x/x2.

Sjoerd Dirksen 4 / 25

Quantization consistency (τ = 0)

Baraniuk and Boufounos proposed to reconstruct via min z0

• s. t.

sign(Ax) = sign(Az), z2 = 1. sign(Ax) = sign(Az) enforces quantization consistency.

Theorem (Jacques-Laska-Boufounos-Baraniuk ’13)

Let A be an m × n Gaussian matrix. If m δ−1(s log(n/δ) + log(η−1)), then w.p. at least 1 − η, for all x, x′ with x2 = x′2 = 1 and x, x′0 ≤ s, sgn(Ax) = sgn(Ax′) ⇒ x − x′2 ≤ δ.

Sjoerd Dirksen 5 / 25

Quantization consistency (τ = 0)

Baraniuk and Boufounos proposed to reconstruct via min z0

• s. t.

sign(Ax) = sign(Az), z2 = 1. sign(Ax) = sign(Az) enforces quantization consistency.

Theorem (Jacques-Laska-Boufounos-Baraniuk ’13)

Let A be an m × n Gaussian matrix. If m δ−1s log(n/δ), then w.h.p., for all x, x′ with x2 = x′2 = 1 and x, x′0 ≤ s, sgn(Ax) = sgn(Ax′) ⇒ x − x′2 ≤ δ.

Sjoerd Dirksen 6 / 25

Pie cutting

Sjoerd Dirksen 7 / 25

Pie cutting I

b b b b b

a1

Figure: We draw a first Gaussian a1. Its direction a1/a12 is distributed uniformly on the unit sphere

Sjoerd Dirksen 8 / 25

Pie cutting II

b b b b b

a1 a⊥

1

Figure: a1 determines a hyperplane a⊥

1 = {x ∈ Rn : a1, x = 0}, which

divides Rn into {x ∈ Rn : a1, x > 0} and {x ∈ Rn : a1, x < 0}

Sjoerd Dirksen 9 / 25

Pie cutting III

b b b

+ −

b b

a⊥

1

Figure: sgn(a1, x) measures on which side of the hyperplane x lies.

Sjoerd Dirksen 10 / 25

Pie cutting IV

b b b b b

a⊥

1

a⊥

2

Figure: If we draw a second Gaussian a2, we obtain a new hyperplane a⊥

2 .

The hyperplanes a⊥

1 and a⊥ 2 cut the pie into 4 pieces.

Sjoerd Dirksen 11 / 25

Pie cutting V

b b b

++ −+ −− +−

b b

a⊥

1

a⊥

2

Figure: The bit string (sgn(a1, x), sgn(a2, x)) ∈ {−1, 1}2 encodes on which of the 4 pieces of pie x is located. A vector z is quantization consistent if it lies on the same piece of pie as x.

Sjoerd Dirksen 12 / 25

Quantization consistency (τ = 0)

Theorem (Jacques-Laska-Boufounos-Baraniuk ’13)

Let A be an m × n Gaussian matrix. If m δ−1s log(n/δ), then w.h.p., for all x, x′ with x2 = x′2 = 1 and x, x′0 ≤ s, sgn(Ax) = sgn(Ax′) ⇒ x − x′2 ≤ δ. Underlying reason: fA(v) = sgn(Av) is w.h.p. a √ε-binary embedding on all s-sparse vectors:

• 1

mdH(fA(v), fA(w)) − dSn−1(v, w)

• ≤ √ε

∀s-sparse v, w. In particular, if dH(fA(v), fA(w)) = 0, then dSn−1(v, w) ≤ √ε. Results for general signal sets are known: Plan-Vershynin ’14, Oymak-Recht ’15.

Sjoerd Dirksen 13 / 25

One-bit CS: linear program

Baraniuk and Boufounos proposed to reconstruct via min z1

• s. t.

sign(Ax) = sign(Az), z2 = 1. Plan and Vershynin ’13 considered min

z∈Rn z1

s.t. sign(Az) = sign(Ax), 1 m π 2 Az1 = 1. (LP) If A is standard Gaussian, then EAz1 =

• 2

πmz2.

Theorem (Plan-Vershynin ’13)

If m δ−1s log2(n/s) then whp: for every x with x1 ≤ √s and x2 = 1 the solution x# of (LP) satisfies x − x#2 ≤ δ1/5.

Sjoerd Dirksen 14 / 25

Gaussian circulant matrices

Sjoerd Dirksen 15 / 25

Subsampled Gaussian circulant matrix

For a g = (g1, . . . , gn) a Gaussian vector, the Gaussian circulant matrix is Cg =        gn g1 g2 · · · gn−2 gn−1 gn−1 gn g1 · · · gn−3 gn−2 gn−2 gn−1 gn · · · gn−4 gn−3 . . . . . . . . . . . . . . . . . . g1 g2 g3 · · · gn−1 gn       

◮ Model: A consists of m (uniformly) randomly selected rows of

Cg.

◮ Ax is a random sample of the discrete convolution g ∗ x. ◮ Applications (Romberg ’09): SAR radar imaging, Fourier

• ptical imaging, channel estimation.

Sjoerd Dirksen 16 / 25

Circulant case: one cut fixes all cuts

b b b b b

a1 a⊥

1

Figure: We draw a first Gaussian a1 to form the first hyperplane. This fixes all other hyperplanes.

Sjoerd Dirksen 17 / 25

Guarantee via RIP1,2

min

z∈Rn z1

s.t. sign(Az) = sign(Ax), Az1 = 1. (LP) A satisfies RIPeff

1,2(s, δ) if

(1 − δ)x2 ≤ Ax1 ≤ (1 + δ)x2 (2) for all x ∈ Rn with x1 ≤ √sx2.

Theorem (Foucart, ’17)

Let δ ≤ 1/5. If A satisfies RIPeff

1,2(9s, δ) then for all x ∈ Rn with

x1 ≤ s and x2 = 1, the LP-reconstruction x#

LP satisfies

x − x#

LP2 ≤ 2

√ 5δ.

1 m

π

2 A, A standard Gaussian, satisfies (2) if m δ−2s log(n/s)

(Schechtman ’06).

Sjoerd Dirksen 18 / 25

Recovery of direction

Cg =Gaussian circulant matrix, RI(x) = (xi)i∈I. Let I = {i ∈ [n] : θi = 1}, where θi are i.i.d. random selectors with mean m/n.

Theorem (D., Jung, Rauhut ’17+)

Let A = RICg and δ < log−1/2(s) log−1/4(n) and suppose that m δ−1s log(n/s) s

• δn/ log n.

(3) Then w.h.p.: for every x ∈ Rn with x1 ≤ s and x2 = 1, the LP-reconstruction x#

LP satisfies x − x# LP2 δ1/8.

For recovery of all exactly s-sparse vectors on the unit sphere a dependence δ1/4 can be obtained.

Sjoerd Dirksen 19 / 25

Full vector recovery: Gaussian case

y = sign(Ax + τ). Observation if A is Gaussian (Knudson-Saab-Ward ’16, Baraniuk-Foucart-Needell-Plan-Wootters ’17): can recover the full signal by taking Gaussian thresholds. Consider min

z∈Rn z1

s.t. sign(Az + τ) = sign(Ax + τ), z2 ≤ R (CP)

Theorem (Baraniuk-Foucart-Needell-Plan-Wootters ’17)

A standard Gaussian, τ1, . . . , τm independent N(0, R2)-distributed. If m δ−1s log(n/s), then w.h.p.: for any x ∈ Rn with x1 ≤ √sx2 and x2 ≤ R, any solution x#

CP to (CP) satisfies x − x# CP2 ≤ Rδ1/4.

Sjoerd Dirksen 20 / 25

Full vector recovery: circulant case

Theorem (D., Jung, Rauhut ’17+)

A = RICg, τ1, . . . , τm independent N(0, R2)-distributed. Suppose δ < log−1/2(s) log−1/4(n), m δ−1s log(n/s) s

• δn/ log n.

Then w.h.p.: for any x ∈ Rn with x1 ≤ √sx2 and x2 ≤ R, any solution x#

CP to (CP) satisfies x − x# CP2 ≤ Rδ1/8.

Sjoerd Dirksen 21 / 25

Dithering ↔ hyperplane shifts

b b b b b

a1 a⊥

1

Figure: a1 determines a hyperplane a⊥

1 = {x ∈ Rn : a1, x = 0}.

a⊥

1,τ1 = {x ∈ Rn : a1, x + τ1 = 0} divides Rn into

{x ∈ Rn : a1, x > −τ1} and {x ∈ Rn : a1, x < −τ1}

Sjoerd Dirksen 22 / 25

Dithering ↔ back to direction recovery

Let A = RICg, h ∈ Rn standard Gaussian. Then min

z∈RN z1

s.t. sign(Az + τ) = sign(Ax + τ), z2 ≤ R is min

z∈RN z1

s.t. sign(RI[Cg h] ¯ z) = sign(RI[Cg h] ¯ x), z2 ≤ R with ¯ z = [z, R]/[z, R]2, ¯ x = [x, R]/[x, R]2. Easy calculation: x − z2 ≤ 2R¯ x − ¯ z2 if x2, z2 ≤ R. Conclusion: sufficient to show RIP1,2-property for (rescaled version

• f) [Cg h].

Sjoerd Dirksen 23 / 25

Proof sketch

Fix δ > 0. Let A = RICg. Need to show:

1 m

π

2 A satisfies

RIPeff

1,2(s, δ) w.h.p., i.e.,

• 1

m π 2 Ax1 − 1

• ≤ δ

for any every effectively sparse vector x on the unit sphere. Pick y s.t. y − x2 ≤ δ.

• 1

m π 2 Ax1 − 1

• ≤
• 1

m π 2 Ax1 − 1 m π 2 Ay1

• +
• 1

m π 2 Ay1 − 1 N π 2 Cgy1

• +
• 1

N π 2 Cgy1 − 1

• .

Sjoerd Dirksen 24 / 25

Uniform scalar quantization

Let Qδ : Rm → (δZ + δ/2)m be the uniform scalar quantizer with resolution δ defined by Qδ(z) =

• δ⌊zi/δ⌋ + δ/2

m

i=1.

Theorem (D., Jung, Rauhut ’17+)

A = RICg, τ1, . . . , τm, independent N(0, R2)-distributed, ui independent uniformly distributed on [0, δ]. Suppose ε < log−1/2(s) log−1/4(n), m R2δ−2ǫ−6s log(en/s), s

• log(n)/n.

Then w.h.p.: for any x ∈ Rn with x1 ≤ √sx2 and x2 ≤ R, any solution x# to min z1 s.t. Qδ π 2 (Az+τ)+u

• = Qδ

π 2 (Ax+τ)+u

• , z2 ≤ R

satisfies x# − x2 δǫ. Proof uses a connection with RIP1,2 (Jacques,Cambareri ’16)

Sjoerd Dirksen 25 / 25