Adaptive Sparse Recovery Eric Price MIT 2012-04-26 Joint work - - PowerPoint PPT Presentation

Adaptive Sparse Recovery

Eric Price

MIT

2012-04-26 Joint work with Piotr Indyk and David Woodruff, 2011-2012

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 1 / 29

Outline

1

Motivating Example

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 2 / 29

Outline

1

Motivating Example

2

Formal Introduction to Sparse Recovery/Compressive Sensing

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 2 / 29

Outline

1

Motivating Example

2

Formal Introduction to Sparse Recovery/Compressive Sensing

3

Algorithm

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 2 / 29

Outline

1

Motivating Example

2

Formal Introduction to Sparse Recovery/Compressive Sensing

3

Algorithm

4

Conclusion

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 2 / 29

Outline

1

Motivating Example

2

Formal Introduction to Sparse Recovery/Compressive Sensing

3

Algorithm

4

Conclusion

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 3 / 29

Carrier screening

Want to figure out who carries a genetic mutation.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 4 / 29

Carrier screening

Want to figure out who carries a genetic mutation.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 4 / 29

Carrier screening

Want to figure out who carries a genetic mutation. Test everyone!

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 4 / 29

Carrier screening

Want to figure out who carries a genetic mutation. Test everyone! Test 1,000,000 people, find 1,000 carriers.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 4 / 29

Carrier screening

Want to figure out who carries a genetic mutation. Test everyone! Test 1,000,000 people, find 1,000 carriers. Very inefficient.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 4 / 29

Carrier screening

Want to figure out who carries a genetic mutation. Test everyone! Test 1,000,000 people, find 1,000 carriers. Very inefficient. Idea: mix together samples.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 4 / 29

Group testing

Goal: find k carriers among n people.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 5 / 29

Group testing

Goal: find k carriers among n people. Group testing: test groups to see if any member is positive.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 5 / 29

Group testing

Goal: find k carriers among n people. Group testing: test groups to see if any member is positive. Doable with Θ(log n

k

• ) = Θ(k log(n/k)) tests.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 5 / 29

Group testing vs. compressive sensing

Goal: find k carriers among n people. Group testing: test groups to see if any member is positive. Doable with Θ(log n

k

• ) = Θ(k log(n/k)) tests.

Compressive sensing: estimate the number of positives in group.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 5 / 29

Group testing vs. compressive sensing

Goal: find k carriers among n people. Group testing: test groups to see if any member is positive. Doable with Θ(log n

k

• ) = Θ(k log(n/k)) tests.

Compressive sensing: estimate the number of positives in group.

◮ Trying to learn x ∈ Rn. (Here, x ∈ {0, 1, 2}n.) Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 5 / 29

Group testing vs. compressive sensing

Goal: find k carriers among n people. Group testing: test groups to see if any member is positive. Doable with Θ(log n

k

• ) = Θ(k log(n/k)) tests.

Compressive sensing: estimate the number of positives in group.

◮ Trying to learn x ∈ Rn. (Here, x ∈ {0, 1, 2}n.) ◮ Choose coefficients v ∈ Rn. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 5 / 29

Group testing vs. compressive sensing

Goal: find k carriers among n people. Group testing: test groups to see if any member is positive. Doable with Θ(log n

k

• ) = Θ(k log(n/k)) tests.

Compressive sensing: estimate the number of positives in group.

◮ Trying to learn x ∈ Rn. (Here, x ∈ {0, 1, 2}n.) ◮ Choose coefficients v ∈ Rn. ◮ Measure v, x with noise. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 5 / 29

Group testing vs. compressive sensing

Goal: find k carriers among n people. Group testing: test groups to see if any member is positive. Doable with Θ(log n

k

• ) = Θ(k log(n/k)) tests.

Compressive sensing: estimate the number of positives in group.

◮ Trying to learn x ∈ Rn. (Here, x ∈ {0, 1, 2}n.) ◮ Choose coefficients v ∈ Rn. ◮ Measure v, x with noise.

Want to minimize number of tests.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 5 / 29

Number of Measurements

Non-adaptive Adaptive Group Testing Ω(k2) Θ(k log(n/k)) Compressive Sensing

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 6 / 29

Number of Measurements

Non-adaptive Adaptive Group Testing Ω(k2) Θ(k log(n/k)) Compressive Sensing Θ(k log(n/k))

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 6 / 29

Number of Measurements

Non-adaptive Adaptive Group Testing Ω(k2) Θ(k log(n/k)) Compressive Sensing Θ(k log(n/k)) ???

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 6 / 29

Number of Measurements

Non-adaptive Adaptive Group Testing Ω(k2) Θ(k log(n/k)) Compressive Sensing Θ(k log(n/k)) ??? Expected fraction of DNA with mutation is k/n = 0.1%.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 6 / 29

Number of Measurements

Non-adaptive Adaptive Group Testing Ω(k2) Θ(k log(n/k)) Compressive Sensing Θ(k log(n/k)) ??? Expected fraction of DNA with mutation is k/n = 0.1%. Group testing possible: machine distinguishes 0% and 0.1%.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 6 / 29

Number of Measurements

Non-adaptive Adaptive Group Testing Ω(k2) Θ(k log(n/k)) Compressive Sensing Θ(k log(n/k)) ??? Expected fraction of DNA with mutation is k/n = 0.1%. Group testing possible: machine distinguishes 0% and 0.1%. Also distinguishes 50.0% and 50.1%.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 6 / 29

Number of Measurements

Non-adaptive Adaptive Group Testing Ω(k2) Θ(k log(n/k)) Compressive Sensing Θ(k log(n/k)) ??? Expected fraction of DNA with mutation is k/n = 0.1%. Group testing possible: machine distinguishes 0% and 0.1%. Also distinguishes 50.0% and 50.1%. Hope for log(n/k) bits/test, or Θ(k) measurements.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 6 / 29

Number of Measurements

Non-adaptive Adaptive Group Testing Ω(k2) Θ(k log(n/k)) Compressive Sensing Θ(k log(n/k)) ??? Expected fraction of DNA with mutation is k/n = 0.1%. Group testing possible: machine distinguishes 0% and 0.1%. Also distinguishes 50.0% and 50.1%. Hope for log(n/k) bits/test, or Θ(k) measurements. Problem: any mixture has expected 0.1% mutation, so O(1) bits.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 6 / 29

Number of Measurements

Non-adaptive Adaptive Group Testing Ω(k2) Θ(k log(n/k)) Compressive Sensing Θ(k log(n/k)) ??? Expected fraction of DNA with mutation is k/n = 0.1%. Group testing possible: machine distinguishes 0% and 0.1%. Also distinguishes 50.0% and 50.1%. Hope for log(n/k) bits/test, or Θ(k) measurements. Problem: any mixture has expected 0.1% mutation, so O(1) bits. Idea: use knowledge from early measurements to make later mixtures more concentrated with mutations.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 6 / 29

Number of Measurements

Non-adaptive Adaptive Group Testing Ω(k2) Θ(k log(n/k)) Compressive Sensing Θ(k log(n/k)) O(k log log(n/k)) Expected fraction of DNA with mutation is k/n = 0.1%. Group testing possible: machine distinguishes 0% and 0.1%. Also distinguishes 50.0% and 50.1%. Hope for log(n/k) bits/test, or Θ(k) measurements. Problem: any mixture has expected 0.1% mutation, so O(1) bits. Idea: use knowledge from early measurements to make later mixtures more concentrated with mutations. This talk: O(k log log(n/k)) adaptive linear measurements.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 6 / 29

Outline

1

Motivating Example

2

Formal Introduction to Sparse Recovery/Compressive Sensing

3

Algorithm

4

Conclusion

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 7 / 29

General Compressive Sensing

Want to observe n-dimensional vector x

◮ Which of n people have a genetic mutation. ◮ Image ◮ Traffic pattern of packets on a network. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 8 / 29

General Compressive Sensing

Want to observe n-dimensional vector x

◮ Which of n people have a genetic mutation. ◮ Image ◮ Traffic pattern of packets on a network.

Normally takes n observations to find.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 8 / 29

General Compressive Sensing

Want to observe n-dimensional vector x

◮ Which of n people have a genetic mutation. ◮ Image ◮ Traffic pattern of packets on a network.

Normally takes n observations to find. But we know some structure on the input:

◮ Genetics: most people don’t have the mutation. ◮ Images: mostly smooth with some edges. ◮ Traffic: Zipf distribution. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 8 / 29

General Compressive Sensing

Want to observe n-dimensional vector x

◮ Which of n people have a genetic mutation. ◮ Image ◮ Traffic pattern of packets on a network.

Normally takes n observations to find. But we know some structure on the input:

◮ Genetics: most people don’t have the mutation. ◮ Images: mostly smooth with some edges. ◮ Traffic: Zipf distribution.

We use this structure to compress space (e.g. JPEG).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 8 / 29

General Compressive Sensing

Want to observe n-dimensional vector x

◮ Which of n people have a genetic mutation. ◮ Image ◮ Traffic pattern of packets on a network.

Normally takes n observations to find. But we know some structure on the input:

◮ Genetics: most people don’t have the mutation. ◮ Images: mostly smooth with some edges. ◮ Traffic: Zipf distribution.

We use this structure to compress space (e.g. JPEG). Can we use structure to save on observations?

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 8 / 29

Cameras

5 megapixel camera takes 15 million byte-size observations.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 9 / 29

Cameras

5 megapixel camera takes 15 million byte-size observations. Compresses it (JPEG) down to a million bytes.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 9 / 29

Cameras

5 megapixel camera takes 15 million byte-size observations. Compresses it (JPEG) down to a million bytes. Why do we need to bother with so many observations? [Donoho,Cand` es-Tao]

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 9 / 29

Cameras

5 megapixel camera takes 15 million byte-size observations. Compresses it (JPEG) down to a million bytes. Why do we need to bother with so many observations? [Donoho,Cand` es-Tao] Cheap in visible light (silicon), very expensive in infrared.

◮ $30,000 for 256x256 IR sensor. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 9 / 29

Cameras

5 megapixel camera takes 15 million byte-size observations. Compresses it (JPEG) down to a million bytes. Why do we need to bother with so many observations? [Donoho,Cand` es-Tao] Cheap in visible light (silicon), very expensive in infrared.

◮ $30,000 for 256x256 IR sensor.

Use structure to take only a few million observations.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 9 / 29

Cameras

5 megapixel camera takes 15 million byte-size observations. Compresses it (JPEG) down to a million bytes. Why do we need to bother with so many observations? [Donoho,Cand` es-Tao] Cheap in visible light (silicon), very expensive in infrared.

◮ $30,000 for 256x256 IR sensor.

Use structure to take only a few million observations.

◮ What structure? Sparsity. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 9 / 29

Sparsity

A vector is k-sparse if k components are non-zero. Images are almost sparse in the wavelet basis:

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 10 / 29

Sparsity

A vector is k-sparse if k components are non-zero. Images are almost sparse in the wavelet basis:

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 10 / 29

Sparsity

A vector is k-sparse if k components are non-zero. Images are almost sparse in the wavelet basis: Same kind of structure as in genetic testing!

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 10 / 29

Linear sketching/Compressive sensing

Suppose an n-dimensional vector x is k-sparse in known basis. Given Ax, a set of m << n linear products.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 11 / 29

Linear sketching/Compressive sensing

Suppose an n-dimensional vector x is k-sparse in known basis. Given Ax, a set of m << n linear products. Why linear? Many applications:

◮ Genetic testing: mixing blood samples. ◮ Streaming updates: A(x + ∆) = Ax + A∆. ◮ Camera optics: filter in front of lens. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 11 / 29

Linear sketching/Compressive sensing

Suppose an n-dimensional vector x is k-sparse in known basis. Given Ax, a set of m << n linear products. Why linear? Many applications:

◮ Genetic testing: mixing blood samples. ◮ Streaming updates: A(x + ∆) = Ax + A∆. ◮ Camera optics: filter in front of lens.

Then it is possible to recover x from Ax.

◮ Quickly ◮ Robustly: get close to x if x is close to k-sparse. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 11 / 29

Linear sketching/Compressive sensing

Suppose an n-dimensional vector x is k-sparse in known basis. Given Ax, a set of m << n linear products. Why linear? Many applications:

◮ Genetic testing: mixing blood samples. ◮ Streaming updates: A(x + ∆) = Ax + A∆. ◮ Camera optics: filter in front of lens.

Then it is possible to recover x from Ax.

◮ Quickly ◮ Robustly: get close to x if x is close to k-sparse.

Note: impossible without using sparsity (A is underdetermined).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 11 / 29

Standard Sparse Recovery Framework

Specify distribution on m × n matrices A (independent of x). Given linear sketch Ax, recover ˆ x. Satisfying the recovery guarantee: ˆ x − x2 (1 + ǫ) min

k-sparse xk

x − xk2 with probability 2/3.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 12 / 29

Standard Sparse Recovery Framework

Specify distribution on m × n matrices A (independent of x). Given linear sketch Ax, recover ˆ x. Satisfying the recovery guarantee: ˆ x − x2 (1 + ǫ) min

k-sparse xk

x − xk2 with probability 2/3. Solvable with O( 1

ǫk log n k ) measurements

[Cand` es-Romberg-Tao ’06, Gilbert-Li-Porat-Strauss ’10]

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 12 / 29

Standard Sparse Recovery Framework

Specify distribution on m × n matrices A (independent of x). Given linear sketch Ax, recover ˆ x. Satisfying the recovery guarantee: ˆ x − x2 (1 + ǫ) min

k-sparse xk

x − xk2 with probability 2/3. Solvable with O( 1

ǫk log n k ) measurements

[Cand` es-Romberg-Tao ’06, Gilbert-Li-Porat-Strauss ’10] Matching lower bound. [Do Ba-Indyk-P-Woodruff ’10, P-Woodruff ’11]

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 12 / 29

Adaptive Sparse Recovery Framework

For i = 1 . . . r:

◮ Choose matrix Ai based on previous observations (possibly

randomized).

◮ Observe Aix. ◮ Number of measurements m is total number of rows in all Ai. ◮ Number of rounds is r.

Given linear sketch Ax, recover ˆ x. Satisfying the recovery guarantee: ˆ x − x2 (1 + ǫ) min

k-sparse xk

x − xk2 with probability 2/3. Solvable with O( 1

ǫk log n k ) measurements

[Cand` es-Romberg-Tao ’06, Gilbert-Li-Porat-Strauss ’10] Matching lower bound. [Do Ba-Indyk-P-Woodruff ’10, P-Woodruff ’11]

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 12 / 29

Adaptive result in comparison to previous work

Nonadaptive: Θ( 1

ǫk log n k ).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 13 / 29

Adaptive result in comparison to previous work

Nonadaptive: Θ( 1

ǫk log n k ).

Adaptive: O(k log n

k ) with ǫ = o(1)

([Haupt-Baraniuk-Castro-Nowak ’09], in a slightly different setting)

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 13 / 29

Adaptive result in comparison to previous work

Nonadaptive: Θ( 1

ǫk log n k ).

Adaptive: O(k log n

k ) with ǫ = o(1)

([Haupt-Baraniuk-Castro-Nowak ’09], in a slightly different setting) This talk: O( 1

ǫk log log n k ). [Indyk-P-Woodruff ’11]

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 13 / 29

Adaptive result in comparison to previous work

Nonadaptive: Θ( 1

ǫk log n k ).

Adaptive: O(k log n

k ) with ǫ = o(1)

([Haupt-Baraniuk-Castro-Nowak ’09], in a slightly different setting) This talk: O( 1

ǫk log log n k ). [Indyk-P-Woodruff ’11]

◮ Using r = O(log log n log∗ k) rounds. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 13 / 29

Adaptive result in comparison to previous work

Nonadaptive: Θ( 1

ǫk log n k ).

Adaptive: O(k log n

k ) with ǫ = o(1)

([Haupt-Baraniuk-Castro-Nowak ’09], in a slightly different setting) This talk: O( 1

ǫk log log n k ). [Indyk-P-Woodruff ’11]

◮ Using r = O(log log n log∗ k) rounds.

Even when r = 2, can get O(k log n + 1

ǫk log(k/ǫ))

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 13 / 29

Adaptive result in comparison to previous work

Nonadaptive: Θ( 1

ǫk log n k ).

Adaptive: O(k log n

k ) with ǫ = o(1)

([Haupt-Baraniuk-Castro-Nowak ’09], in a slightly different setting) This talk: O( 1

ǫk log log n k ). [Indyk-P-Woodruff ’11]

◮ Using r = O(log log n log∗ k) rounds.

Even when r = 2, can get O(k log n + 1

ǫk log(k/ǫ))

◮ Separating dependence on n and ǫ. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 13 / 29

Applications of Adaptivity

When does adaptivity make sense?

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

Applications of Adaptivity

When does adaptivity make sense? Genetic testing:

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

Applications of Adaptivity

When does adaptivity make sense? Genetic testing:

◮ Yes, but multiple rounds can be costly. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

Applications of Adaptivity

When does adaptivity make sense? Genetic testing:

◮ Yes, but multiple rounds can be costly.

Cameras:

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

Applications of Adaptivity

When does adaptivity make sense? Genetic testing:

◮ Yes, but multiple rounds can be costly.

Cameras:

◮ Programmable pixels (mirrors or LCD display): Yes. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

Applications of Adaptivity

When does adaptivity make sense? Genetic testing:

◮ Yes, but multiple rounds can be costly.

Cameras:

◮ Programmable pixels (mirrors or LCD display): Yes. ◮ Hardwired lens: No. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

Applications of Adaptivity

When does adaptivity make sense? Genetic testing:

◮ Yes, but multiple rounds can be costly.

Cameras:

◮ Programmable pixels (mirrors or LCD display): Yes. ◮ Hardwired lens: No.

Streaming algorithms:

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

Applications of Adaptivity

When does adaptivity make sense? Genetic testing:

◮ Yes, but multiple rounds can be costly.

Cameras:

◮ Programmable pixels (mirrors or LCD display): Yes. ◮ Hardwired lens: No.

Streaming algorithms:

◮ Adaptivity corresponds to multiple passes. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

Applications of Adaptivity

When does adaptivity make sense? Genetic testing:

◮ Yes, but multiple rounds can be costly.

Cameras:

◮ Programmable pixels (mirrors or LCD display): Yes. ◮ Hardwired lens: No.

Streaming algorithms:

◮ Adaptivity corresponds to multiple passes. ◮ Router finding most common connections: No. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

Applications of Adaptivity

When does adaptivity make sense? Genetic testing:

◮ Yes, but multiple rounds can be costly.

Cameras:

◮ Programmable pixels (mirrors or LCD display): Yes. ◮ Hardwired lens: No.

Streaming algorithms:

◮ Adaptivity corresponds to multiple passes. ◮ Router finding most common connections: No. ◮ Mapreduce finding most frequent URLs: Yes. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

Outline

1

Motivating Example

2

Formal Introduction to Sparse Recovery/Compressive Sensing

3

Algorithm

4

Conclusion

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 15 / 29

Outline of Algorithm

Theorem

Adaptive 1 + ǫ-approximate k-sparse recovery is possible with O( 1

ǫk log log(n/k)) measurements.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 16 / 29

Outline of Algorithm

Theorem

Adaptive 1 + ǫ-approximate k-sparse recovery is possible with O( 1

ǫk log log(n/k)) measurements.

Lemma

Adaptive O(1)-approximate 1-sparse recovery is possible with O(log log n) measurements.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 16 / 29

Outline of Algorithm

Theorem

Adaptive 1 + ǫ-approximate k-sparse recovery is possible with O( 1

ǫk log log(n/k)) measurements.

Lemma

Adaptive O(1)-approximate 1-sparse recovery is possible with O(log log n) measurements. Lemma implies theorem using standard tricks ([GLPS10]).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 16 / 29

1-sparse recovery: non-adaptive lower bound

Lemma

Adaptive C-approximate 1-sparse recovery is possible with O(log log n) measurements for some C = O(1).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 17 / 29

1-sparse recovery: non-adaptive lower bound

Lemma

Adaptive C-approximate 1-sparse recovery is possible with O(log log n) measurements for some C = O(1). Non-adaptive lower bound: why is this hard?

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 17 / 29

1-sparse recovery: non-adaptive lower bound

Lemma

Adaptive C-approximate 1-sparse recovery is possible with O(log log n) measurements for some C = O(1). Non-adaptive lower bound: why is this hard? Hard case: x is random ei plus Gaussian noise w.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 17 / 29

1-sparse recovery: non-adaptive lower bound

Lemma

Adaptive C-approximate 1-sparse recovery is possible with O(log log n) measurements for some C = O(1). Non-adaptive lower bound: why is this hard? Hard case: x is random ei plus Gaussian noise w. Noise w2

2 = Θ(1) so C-approximate recovery requires finding i.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 17 / 29

1-sparse recovery: non-adaptive lower bound

Lemma

Adaptive C-approximate 1-sparse recovery is possible with O(log log n) measurements for some C = O(1). Non-adaptive lower bound: why is this hard? Hard case: x is random ei plus Gaussian noise w. Noise w2

2 = Θ(1) so C-approximate recovery requires finding i.

Observations v, x = vi + v, w = vi + v2

√n z, for z ∼ N(0, Θ(1)).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 17 / 29

1-sparse recovery: non-adaptive lower bound

Observe v, x = vi + v2

√n z, where z ∼ N(0, Θ(1))

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

1-sparse recovery: non-adaptive lower bound

Observe v, x = vi + v2

√n z, where z ∼ N(0, Θ(1))

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

1-sparse recovery: non-adaptive lower bound

Observe v, x = vi + v2

√n z, where z ∼ N(0, Θ(1))

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

1-sparse recovery: non-adaptive lower bound

Observe v, x = vi + v2

√n z, where z ∼ N(0, Θ(1))

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

1-sparse recovery: non-adaptive lower bound

Observe v, x = vi + v2

√n z, where z ∼ N(0, Θ(1))

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

1-sparse recovery: non-adaptive lower bound

Observe v, x = vi + v2

√n z, where z ∼ N(0, Θ(1))

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

1-sparse recovery: non-adaptive lower bound

Observe v, x = vi + v2

√n z, where z ∼ N(0, Θ(1))

Information capacity I(i, v, x) 1 2 log(1 + SNR) where SNR denotes the “signal-to-noise ratio,” SNR = E[signal2] E[noise2]

• E[v2

i ]

v2

2/n = 1

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

1-sparse recovery: non-adaptive lower bound

Observe v, x = vi + v2

√n z, where z ∼ N(0, Θ(1))

Information capacity I(i, v, x) 1 2 log(1 + SNR) where SNR denotes the “signal-to-noise ratio,” SNR = E[signal2] E[noise2]

• E[v2

i ]

v2

2/n = 1

Finding i needs Ω(log n) non-adaptive measurements.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

1-sparse recovery: changes in adaptive setting

Information capacity I(i, v, x) 1 2 log(1 + SNR). where SNR denotes the “signal-to-noise ratio,” SNR = Θ

• E[v2

i ]

v2

2/n

• .

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 19 / 29

1-sparse recovery: changes in adaptive setting

Information capacity I(i, v, x) 1 2 log(1 + SNR). where SNR denotes the “signal-to-noise ratio,” SNR = Θ

• E[v2

i ]

v2

2/n

• .

If i is independent of v, this is O(1).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 19 / 29

1-sparse recovery: changes in adaptive setting

Information capacity I(i, v, x) 1 2 log(1 + SNR). where SNR denotes the “signal-to-noise ratio,” SNR = Θ

• E[v2

i ]

v2

2/n

• .

If i is independent of v, this is O(1). As we learn about i, we can increase E[v2

i ] for constant v2.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 19 / 29

1-sparse recovery: changes in adaptive setting

Information capacity I(i, v, x) 1 2 log(1 + SNR). where SNR denotes the “signal-to-noise ratio,” SNR = Θ

• E[v2

i ]

v2

2/n

• .

If i is independent of v, this is O(1). As we learn about i, we can increase E[v2

i ] for constant v2.

◮ Equivalently, for constant E[v2

i ] we can decrease v2.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 19 / 29

1-sparse recovery: idea

x = ei + w

0 bits v Candidate set Signal

SNR = 2 I(i, v, x) log SNR = 1 v, x = vi + v, w

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 20 / 29

1-sparse recovery: idea

x = ei + w

0 bits 1 bit v Candidate set Signal

SNR = 22 I(i, v, x) log SNR = 2 v, x = vi + v, w

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 20 / 29

1-sparse recovery: idea

x = ei + w

0 bits 1 bit 2 bits v Candidate set Signal

SNR = 24 I(i, v, x) log SNR = 4 v, x = vi + v, w

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 20 / 29

1-sparse recovery: idea

x = ei + w

0 bits 1 bit 2 bits 4 bits v Candidate set Signal

SNR = 28 I(i, v, x) log SNR = 8 v, x = vi + v, w

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 20 / 29

1-sparse recovery: idea

x = ei + w

0 bits 1 bit 2 bits 4 bits 8 bits v Candidate set Signal

SNR = 216 I(i, v, x) log SNR = 16 v, x = vi + v, w

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 20 / 29

Goal

Shown intuition for specific distribution on x

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 21 / 29

Goal

Shown intuition for specific distribution on x Match previous convergence for arbitrary x = αei + w?

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 21 / 29

Goal

Shown intuition for specific distribution on x Match previous convergence for arbitrary x = αei + w?

◮ α may not be 1. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 21 / 29

Goal

Shown intuition for specific distribution on x Match previous convergence for arbitrary x = αei + w?

◮ α may not be 1. ◮ Work for a specific x with 3/4 probability. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 21 / 29

Goal

Shown intuition for specific distribution on x Match previous convergence for arbitrary x = αei + w?

◮ α may not be 1. ◮ Work for a specific x with 3/4 probability. ◮ Distribution over A, for fixed w. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 21 / 29

Goal

Find i from x = αei + w using log log n adaptive measurements.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 22 / 29

Goal

Find i from x = αei + w using log log n adaptive measurements. Define the signal-to-noise ratio SNR(x) = α2/w2

2.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 22 / 29

Goal

Find i from x = αei + w using log log n adaptive measurements. Define the signal-to-noise ratio SNR(x) = α2/w2

2.

For Gaussian w, can fit roughly √ SNR distinct Gaussians.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 22 / 29

Goal

Find i from x = αei + w using log log n adaptive measurements. Define the signal-to-noise ratio SNR(x) = α2/w2

2.

For Gaussian w, can fit roughly √ SNR distinct Gaussians. Given O(1) measurements, find S ∋ i with SNR(xS) (SNR(x))3/2

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 22 / 29

Goal

Find i from x = αei + w using log log n adaptive measurements. Define the signal-to-noise ratio SNR(x) = α2/w2

2.

For Gaussian w, can fit roughly √ SNR distinct Gaussians. Given O(1) measurements, find S ∋ i with SNR(xS) δ2(SNR(x))3/2 with probability 1 − O(δ).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 22 / 29

Goal

Find i from x = αei + w using log log n adaptive measurements. Define the signal-to-noise ratio SNR(x) = α2/w2

2.

For Gaussian w, can fit roughly √ SNR distinct Gaussians. Given O(1) measurements, find S ∋ i with SNR(xS) δ2(SNR(x))3/2 with probability 1 − O(δ). Repeat on xS.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 22 / 29

Goal

Find i from x = αei + w using log log n adaptive measurements. Define the signal-to-noise ratio SNR(x) = α2/w2

2.

For Gaussian w, can fit roughly √ SNR distinct Gaussians. Given O(1) measurements, find S ∋ i with SNR(xS) δ2(SNR(x))3/2 with probability 1 − O(δ). Repeat on xS. Once SNR(x) reaches O(n2), will have S = {i}.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 22 / 29

Recovery when SNR > n2

Getting log n bits when SNR is n2

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 23 / 29

Recovery when SNR > n2

Getting log n bits when SNR is n2

Find i in 2 measurements with probability 1 − O(δ).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 23 / 29

Recovery when SNR > n2

Getting log n bits when SNR is n2

Find i in 2 measurements with probability 1 − O(δ). Observe a =

• jxj

b =

• xj

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 23 / 29

Recovery when SNR > n2

Getting log n bits when SNR is n2

Find i in 2 measurements with probability 1 − O(δ). Observe a =

• jxj

b =

• xj

Then i ≈ a/b

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 23 / 29

Recovery when SNR > n2

Getting log n bits when SNR is n2

Find i in 2 measurements with probability 1 − O(δ). Observe a =

• jxj

b =

• xj

Then i ≈ a/b, with error proportional to w1.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 23 / 29

Recovery when SNR > n2

Getting log n bits when SNR is n2

Find i in 2 measurements with probability 1 − O(δ). Observe, for s ∈ {±1}n pairwise independently: a =

• jxjsj

b =

• xjsj

Then i ≈ a/b, with error proportional to w2.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 23 / 29

Recovery when SNR > n2

Getting log n bits when SNR is n2

Find i in 2 measurements with probability 1 − O(δ). Observe, for s ∈ {±1}n pairwise independently: a =

• jxjsj

b =

• xjsj

Then i ≈ a/b, with error proportional to w2. |i − a/b| < n/(δ √ SNR) with probability 1 − O(δ) (over s).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 23 / 29

Recovery when SNR > n2

Getting log n bits when SNR is n2

Find i in 2 measurements with probability 1 − O(δ). Observe, for s ∈ {±1}n pairwise independently: a =

• jxjsj

b =

• xjsj

Then i ≈ a/b, with error proportional to w2. |i − a/b| < n/(δ √ SNR) with probability 1 − O(δ) (over s). For SNR > (n/δ)2, done.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 23 / 29

Getting log SNR bits for general SNR

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 24 / 29

Getting log SNR bits for general SNR

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 24 / 29

Getting log SNR bits for general SNR

Still |i − a/b| < n/(2δ √ SNR) with 1 − O(δ) probability.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 24 / 29

Getting log SNR bits for general SNR

Still |i − a/b| < n/(2δ √ SNR) with 1 − O(δ) probability. So given a and b, know i in S of size |S| = n/(δ √ SNR)

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 24 / 29

Getting log SNR bits for general SNR

Still |i − a/b| < n/(2δ √ SNR) with 1 − O(δ) probability. So given a and b, know i in S of size |S| = n/(δ √ SNR) Want SNR(xS) ≈ (δ

• SNR(x))SNR(x).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 24 / 29

Getting log SNR bits for general SNR

Still |i − a/b| < n/(2δ √ SNR) with 1 − O(δ) probability. So given a and b, know i in S of size |S| = n/(δ √ SNR) Want SNR(xS) ≈ (δ

• SNR(x))SNR(x).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 24 / 29

Getting log SNR bits for general SNR

Still |i − a/b| < n/(2δ √ SNR) with 1 − O(δ) probability. So given a and b, know i in S of size |S| = n/(δ √ SNR) Want SNR(xS) ≈ (δ

• SNR(x))SNR(x).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 24 / 29

Getting log SNR bits for general SNR

Still |i − a/b| < n/(2δ √ SNR) with 1 − O(δ) probability. So given a and b, know i in S of size |S| = n/(δ √ SNR) Want SNR(xS) ≈ (δ

• SNR(x))SNR(x).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 24 / 29

Getting log SNR bits for general SNR

Still |i − a/b| < n/(2δ √ SNR) with 1 − O(δ) probability. So given a and b, know i in S of size |S| = n/(δ √ SNR) Want SNR(xS) ≈ (δ

• SNR(x))SNR(x).

Randomly permute x beforehand! Then SNR shrinks in expectation. SNR(xS) (δSNR(x))3/2

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 24 / 29

Getting log SNR bits for general SNR

Still |i − a/b| < n/(2δ √ SNR) with 1 − O(δ) probability. So given a and b, know i in S of size |S| = n/(δ √ SNR) Want SNR(xS) ≈ (δ

• SNR(x))SNR(x).

Randomly permute x beforehand! Then SNR shrinks in expectation. SNR(xS) (δSNR(x))3/2 Set δ = 0.1/2r in round r; still doubly exponential growth.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 24 / 29

End proof of key lemma

Lemma

Adaptive C-approximate 1-sparse recovery is possible with O(log log n) measurements for some C = O(1).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 25 / 29

End proof of key lemma

Lemma

Adaptive C-approximate 1-sparse recovery is possible with O(log log n) measurements for some C = O(1).

Theorem (Adaptive upper bound)

Adaptive 1 + ǫ-approximate k-sparse recovery is possible with O( 1

ǫk log log(n/k)) measurements.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 25 / 29

End proof of key lemma

Lemma

Adaptive C-approximate 1-sparse recovery is possible with O(log log n) measurements for some C = O(1).

Theorem (Adaptive upper bound)

Adaptive 1 + ǫ-approximate k-sparse recovery is possible with O( 1

ǫk log log(n/k)) measurements.

Lemma implies theorem using standard tricks (a la [GLPS10]):

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 25 / 29

End proof of key lemma

Lemma

Adaptive C-approximate 1-sparse recovery is possible with O(log log n) measurements for some C = O(1).

Theorem (Adaptive upper bound)

Adaptive 1 + ǫ-approximate k-sparse recovery is possible with O( 1

ǫk log log(n/k)) measurements.

Lemma implies theorem using standard tricks (a la [GLPS10]):

◮ Subsample at rate ǫ/k and apply the lemma, O(k/ǫ) times. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 25 / 29

End proof of key lemma

Lemma

Adaptive C-approximate 1-sparse recovery is possible with O(log log n) measurements for some C = O(1).

Theorem (Adaptive upper bound)

Adaptive 1 + ǫ-approximate k-sparse recovery is possible with O( 1

ǫk log log(n/k)) measurements.

Lemma implies theorem using standard tricks (a la [GLPS10]):

◮ Subsample at rate ǫ/k and apply the lemma, O(k/ǫ) times. ◮ Replace k by k/2, repeat. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 25 / 29

Experiments!

Does O(log n) → O(log log n) really matter? What about the constants?

10 20 30 40 50

SNR (dB)

5 10 15 20 25 30

Number of measurements

m as a function of SNR (n=8192,k=1)

Gaussian measurements, L1 minimization Adaptive measurements

5 10 15 20 25 30

log n

5 10 15 20 25 30

Number of measurements

m as a function of n (SNR=10db,k=1)

Gaussian measurements, L1 minimization Adaptive measurements

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 26 / 29

Experiments!

Does O(log n) → O(log log n) really matter? What about the constants?

10 20 30 40 50

SNR (dB)

5 10 15 20 25 30

Number of measurements

m as a function of SNR (n=8192,k=1)

Gaussian measurements, L1 minimization Adaptive measurements

5 10 15 20 25 30

log n

5 10 15 20 25 30

Number of measurements

m as a function of n (SNR=10db,k=1)

Gaussian measurements, L1 minimization Adaptive measurements

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 26 / 29

Round complexity

Basic algorithm

◮ O( 1

ǫk log log(n/k)) measurements.

◮ O(log∗ k log log n) rounds. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 27 / 29

Round complexity

Basic algorithm

◮ O( 1

ǫk log log(n/k)) measurements.

◮ O(log∗ k log log n) rounds.

Given O(r log∗ k) rounds, O( 1

ǫkr log1/r(n/k)) measurements.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 27 / 29

Round complexity

Basic algorithm

◮ O( 1

ǫk log log(n/k)) measurements.

◮ O(log∗ k log log n) rounds.

Given O(r log∗ k) rounds, O( 1

ǫkr log1/r(n/k)) measurements.

Lower bound: given r rounds, Ω(k/ǫ + r log1/r n) measurements. [Arias-Castro-C` andes-Davenport ’11, P-Woodruff ’12]

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 27 / 29

Round complexity

Basic algorithm

◮ O( 1

ǫk log log(n/k)) measurements.

◮ O(log∗ k log log n) rounds.

Given O(r log∗ k) rounds, O( 1

ǫkr log1/r(n/k)) measurements.

Lower bound: given r rounds, Ω(k/ǫ + r log1/r n) measurements. [Arias-Castro-C` andes-Davenport ’11, P-Woodruff ’12]

◮ For k = 1, tight up to O(log∗ k) factor in rounds. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 27 / 29

Round complexity

Basic algorithm

◮ O( 1

ǫk log log(n/k)) measurements.

◮ O(log∗ k log log n) rounds.

Given O(r log∗ k) rounds, O( 1

ǫkr log1/r(n/k)) measurements.

Lower bound: given r rounds, Ω(k/ǫ + r log1/r n) measurements. [Arias-Castro-C` andes-Davenport ’11, P-Woodruff ’12]

◮ For k = 1, tight up to O(log∗ k) factor in rounds.

Given two rounds, O( 1

ǫk log(k/ǫ) + k log(n/k)) measurements.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 27 / 29

Round complexity

Basic algorithm

◮ O( 1

ǫk log log(n/k)) measurements.

◮ O(log∗ k log log n) rounds.

Given O(r log∗ k) rounds, O( 1

ǫkr log1/r(n/k)) measurements.

Lower bound: given r rounds, Ω(k/ǫ + r log1/r n) measurements. [Arias-Castro-C` andes-Davenport ’11, P-Woodruff ’12]

◮ For k = 1, tight up to O(log∗ k) factor in rounds.

Given two rounds, O( 1

ǫk log(k/ǫ) + k log(n/k)) measurements.

◮ Separates dependence on ǫ and n. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 27 / 29

Outline

1

Motivating Example

2

Formal Introduction to Sparse Recovery/Compressive Sensing

3

Algorithm

4

Conclusion

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 28 / 29

Results and future work

Nonadaptive sparse recovery requires Θ(k log n

k ) measurements.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 29 / 29

Results and future work

Nonadaptive sparse recovery requires Θ(k log n

k ) measurements.

Adaptive algorithm uses O(r log∗ k) rounds for O( 1

ǫkr log1/r n k )

measurements.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 29 / 29

Results and future work

Nonadaptive sparse recovery requires Θ(k log n

k ) measurements.

Adaptive algorithm uses O(r log∗ k) rounds for O( 1

ǫkr log1/r n k )

measurements.

◮ Also: 2 rounds, O( 1

ǫk log(k/ǫ) + k log(n/k)) measurements.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 29 / 29

Results and future work

Nonadaptive sparse recovery requires Θ(k log n

k ) measurements.

Adaptive algorithm uses O(r log∗ k) rounds for O( 1

ǫkr log1/r n k )

measurements.

◮ Also: 2 rounds, O( 1

ǫk log(k/ǫ) + k log(n/k)) measurements.

Clearer characterization of measurement/round tradeoff?

◮ Algorithm is O(log∗ k) rounds off lower bound. ◮ Given 4 iterations, how many total blood tests do we need? Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 29 / 29

Results and future work

Nonadaptive sparse recovery requires Θ(k log n

k ) measurements.

Adaptive algorithm uses O(r log∗ k) rounds for O( 1

ǫkr log1/r n k )

measurements.

◮ Also: 2 rounds, O( 1

ǫk log(k/ǫ) + k log(n/k)) measurements.

Clearer characterization of measurement/round tradeoff?

◮ Algorithm is O(log∗ k) rounds off lower bound. ◮ Given 4 iterations, how many total blood tests do we need?

Incorporating adaptivity in constrained matrix designs?

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 29 / 29

Results and future work

Nonadaptive sparse recovery requires Θ(k log n

k ) measurements.

Adaptive algorithm uses O(r log∗ k) rounds for O( 1

ǫkr log1/r n k )

measurements.

◮ Also: 2 rounds, O( 1

ǫk log(k/ǫ) + k log(n/k)) measurements.

Clearer characterization of measurement/round tradeoff?

◮ Algorithm is O(log∗ k) rounds off lower bound. ◮ Given 4 iterations, how many total blood tests do we need?

Incorporating adaptivity in constrained matrix designs? Relate k/ǫ and n in tight bounds? Know Ω( 1

ǫk + r log1/r n).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 29 / 29

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 30 / 29

Separating ǫ and n

Hash to O(k2/ǫ2) blocks, and probably all of:

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 31 / 29

Separating ǫ and n

Hash to O(k2/ǫ2) blocks, and probably all of:

◮ A perfect hash, so heavy hitters land in different blocks. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 31 / 29

Separating ǫ and n

Hash to O(k2/ǫ2) blocks, and probably all of:

◮ A perfect hash, so heavy hitters land in different blocks. ◮ Each heavy hitter dominates the noise in the same block. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 31 / 29

Separating ǫ and n

Hash to O(k2/ǫ2) blocks, and probably all of:

◮ A perfect hash, so heavy hitters land in different blocks. ◮ Each heavy hitter dominates the noise in the same block. ◮ Overall, the noise grows by at most 1 + ǫ/2 factor Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 31 / 29

Separating ǫ and n

Hash to O(k2/ǫ2) blocks, and probably all of:

◮ A perfect hash, so heavy hitters land in different blocks. ◮ Each heavy hitter dominates the noise in the same block. ◮ Overall, the noise grows by at most 1 + ǫ/2 factor

Solve (1 + ǫ)-approximate sparse recovery in reduced space: O( 1

ǫk log(k/ǫ))

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 31 / 29

Separating ǫ and n

Hash to O(k2/ǫ2) blocks, and probably all of:

◮ A perfect hash, so heavy hitters land in different blocks. ◮ Each heavy hitter dominates the noise in the same block. ◮ Overall, the noise grows by at most 1 + ǫ/2 factor

Solve (1 + ǫ)-approximate sparse recovery in reduced space: O( 1

ǫk log(k/ǫ))

Identifies O(k) blocks to search containing enough heavy hitter mass.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 31 / 29

Separating ǫ and n

Hash to O(k2/ǫ2) blocks, and probably all of:

◮ A perfect hash, so heavy hitters land in different blocks. ◮ Each heavy hitter dominates the noise in the same block. ◮ Overall, the noise grows by at most 1 + ǫ/2 factor

Solve (1 + ǫ)-approximate sparse recovery in reduced space: O( 1

ǫk log(k/ǫ))

Identifies O(k) blocks to search containing enough heavy hitter mass. Heavy hitters are O(1)-heavy among their blocks, so O(log n) per block suffices.

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 31 / 29

Separating ǫ and n

Hash to O(k2/ǫ2) blocks, and probably all of:

◮ A perfect hash, so heavy hitters land in different blocks. ◮ Each heavy hitter dominates the noise in the same block. ◮ Overall, the noise grows by at most 1 + ǫ/2 factor

Solve (1 + ǫ)-approximate sparse recovery in reduced space: O( 1

ǫk log(k/ǫ))

Identifies O(k) blocks to search containing enough heavy hitter mass. Heavy hitters are O(1)-heavy among their blocks, so O(log n) per block suffices. Result: O( 1

ǫk log(k/ǫ) + k log n).

Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 31 / 29