adaptive sparse recovery with limited adaptivity
play

Adaptive Sparse Recovery with Limited Adaptivity Akshay Kamath Eric - PowerPoint PPT Presentation

Dec 03, 2022 •364 likes •2k views

Adaptive Sparse Recovery with Limited Adaptivity Akshay Kamath Eric Price UT Austin 2018-11-27 Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 1 / 33 Outline Introduction 1 Analysis for k = 1 2

  1. Results in adaptive sparse recovery, C = O ( 1 ) Unlimited adaptivity: with unlimited rounds, k + log log n � m ∗ � k · log log n Limited adaptivity: with R = O ( 1 ) rounds, k + log 1 / R n � m ∗ � k · log 1 / ( R − 3 ) n . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 8 / 33

  2. Results in adaptive sparse recovery, C = O ( 1 ) Unlimited adaptivity: with unlimited rounds, k + log log n � m ∗ � k · log log n Limited adaptivity: with R = O ( 1 ) rounds, k + log 1 / R n � m ∗ � k · log 1 / ( R − 3 ) n . New results: with R = O ( 1 ) rounds, k · log 1 / R n � m ∗ � k · log 1 / R n log ∗ k With caveat: Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 8 / 33

  3. Results in adaptive sparse recovery, C = O ( 1 ) Unlimited adaptivity: with unlimited rounds, k + log log n � m ∗ � k · log log n Limited adaptivity: with R = O ( 1 ) rounds, k + log 1 / R n � m ∗ � k · log 1 / ( R − 3 ) n . New results: with R = O ( 1 ) rounds, k · log 1 / R n � m ∗ � k · log 1 / R n log ∗ k With caveat: the lower bound only applies for k < 2 log 1 / R n ⇒ m ∗ > k log k . ⇐ Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 8 / 33

  4. Results in adaptive sparse recovery, C = O ( 1 ) Unlimited adaptivity: with unlimited rounds, k + log log n � m ∗ � k · log log n Limited adaptivity: with R = O ( 1 ) rounds, k + log 1 / R n � m ∗ � k · log 1 / ( R − 3 ) n . New results: with R = O ( 1 ) rounds, k · log 1 / R n � m ∗ � k · log 1 / R n log ∗ k With caveat: the lower bound only applies for k < 2 log 1 / R n ⇒ m ∗ > k log k . ⇐ For k < n o ( 1 ) , m ∗ = ω ( k ) . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 8 / 33

  5. Outline Introduction 1 Analysis for k = 1 2 General k : lower bound 3 General k : upper bound 4 Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 9 / 33

  6. Well-understood setting: k = 1 Theorem (Indyk-Price-Woodruff ’11, Price-Woodruff ’13) R-round 1 -sparse recovery requires Θ ( R log 1 / R n ) measurements. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 10 / 33

  7. Well-understood setting: k = 1 Theorem (Indyk-Price-Woodruff ’11, Price-Woodruff ’13) R-round 1 -sparse recovery requires Θ ( R log 1 / R n ) measurements. Outline of this section: Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 10 / 33

  8. Well-understood setting: k = 1 Theorem (Indyk-Price-Woodruff ’11, Price-Woodruff ’13) R-round 1 -sparse recovery requires Θ ( R log 1 / R n ) measurements. Outline of this section: ◮ R = 1 lower bound: Ω ( log n ) . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 10 / 33

  9. Well-understood setting: k = 1 Theorem (Indyk-Price-Woodruff ’11, Price-Woodruff ’13) R-round 1 -sparse recovery requires Θ ( R log 1 / R n ) measurements. Outline of this section: ◮ R = 1 lower bound: Ω ( log n ) . ◮ Adaptive upper bound: O ( log log n ) . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 10 / 33

  10. Well-understood setting: k = 1 Theorem (Indyk-Price-Woodruff ’11, Price-Woodruff ’13) R-round 1 -sparse recovery requires Θ ( R log 1 / R n ) measurements. Outline of this section: ◮ R = 1 lower bound: Ω ( log n ) . ◮ Adaptive upper bound: O ( log log n ) . ◮ Adaptive lower bound: Ω ( log log n ) . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 10 / 33

  11. Well-understood setting: k = 1 Theorem (Indyk-Price-Woodruff ’11, Price-Woodruff ’13) R-round 1 -sparse recovery requires Θ ( R log 1 / R n ) measurements. Outline of this section: ◮ R = 1 lower bound: Ω ( log n ) . ◮ Adaptive upper bound: O ( log log n ) . ◮ Adaptive lower bound: Ω ( log log n ) . Hard case: x is random e z plus Gaussian noise w with � w � 2 ≈ 1. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 10 / 33

  12. Well-understood setting: k = 1 Theorem (Indyk-Price-Woodruff ’11, Price-Woodruff ’13) R-round 1 -sparse recovery requires Θ ( R log 1 / R n ) measurements. Outline of this section: ◮ R = 1 lower bound: Ω ( log n ) . ◮ Adaptive upper bound: O ( log log n ) . ◮ Adaptive lower bound: Ω ( log log n ) . Hard case: x is random e z plus Gaussian noise w with � w � 2 ≈ 1. Robust recovery must locate z . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 10 / 33

  13. Well-understood setting: k = 1 Theorem (Indyk-Price-Woodruff ’11, Price-Woodruff ’13) R-round 1 -sparse recovery requires Θ ( R log 1 / R n ) measurements. Outline of this section: ◮ R = 1 lower bound: Ω ( log n ) . ◮ Adaptive upper bound: O ( log log n ) . ◮ Adaptive lower bound: Ω ( log log n ) . Hard case: x is random e z plus Gaussian noise w with � w � 2 ≈ 1. Robust recovery must locate z . Observations � v , x � = v z + � v , w � = v z + � v � 2 √ n z , for z ∼ N ( 0, 1 ) . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 10 / 33

  14. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v z + � v � 2 √ n z , where z ∼ N ( 0, Θ ( 1 )) Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 11 / 33

  15. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v z + � v � 2 √ n z , where z ∼ N ( 0, Θ ( 1 )) Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 11 / 33

  16. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v z + � v � 2 √ n z , where z ∼ N ( 0, Θ ( 1 )) Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 11 / 33

  17. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v z + � v � 2 √ n z , where z ∼ N ( 0, Θ ( 1 )) Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 11 / 33

  18. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v z + � v � 2 √ n z , where z ∼ N ( 0, Θ ( 1 )) Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 11 / 33

  19. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v z + � v � 2 √ n z , where z ∼ N ( 0, Θ ( 1 )) Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 11 / 33

  20. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v z + � v � 2 √ n z , where z ∼ N ( 0, Θ ( 1 )) Shannon-Hartley theorem: AWGN channel capacity is I ( z , � v , x � ) � 1 2 log ( 1 + SNR ) where SNR denotes the “signal-to-noise ratio,” Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 11 / 33

  21. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v z + � v � 2 √ n z , where z ∼ N ( 0, Θ ( 1 )) Shannon-Hartley theorem: AWGN channel capacity is I ( z , � v , x � ) � 1 2 log ( 1 + SNR ) where SNR denotes the “signal-to-noise ratio,” SNR = E [ signal 2 ] E [ noise 2 ] � E [ v 2 z ] � v � 2 2 / n Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 11 / 33

  22. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v z + � v � 2 √ n z , where z ∼ N ( 0, Θ ( 1 )) Shannon-Hartley theorem: AWGN channel capacity is I ( z , � v , x � ) � 1 2 log ( 1 + SNR ) where SNR denotes the “signal-to-noise ratio,” SNR = E [ signal 2 ] E [ noise 2 ] � E [ v 2 z ] 2 / n = 1 � v � 2 Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 11 / 33

  23. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v z + � v � 2 √ n z , where z ∼ N ( 0, Θ ( 1 )) Shannon-Hartley theorem: AWGN channel capacity is I ( z , � v , x � ) � 1 2 log ( 1 + SNR ) where SNR denotes the “signal-to-noise ratio,” SNR = E [ signal 2 ] E [ noise 2 ] � E [ v 2 z ] 2 / n = 1 � v � 2 Finding z needs Ω ( log n ) non-adaptive measurements. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 11 / 33

  24. 1-sparse recovery: changes in adaptive setting Information capacity I ( z , � v , x � ) � 1 2 log ( 1 + SNR ) . where SNR denotes the “signal-to-noise ratio,” SNR � E [ v 2 z ] 2 / n . � v � 2 Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 12 / 33

  25. 1-sparse recovery: changes in adaptive setting Information capacity I ( z , � v , x � ) � 1 2 log ( 1 + SNR ) . where SNR denotes the “signal-to-noise ratio,” SNR � E [ v 2 z ] 2 / n . � v � 2 If z is independent of v , this is 1. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 12 / 33

  26. 1-sparse recovery: changes in adaptive setting Information capacity I ( z , � v , x � ) � 1 2 log ( 1 + SNR ) . where SNR denotes the “signal-to-noise ratio,” SNR � E [ v 2 z ] 2 / n . � v � 2 If z is independent of v , this is 1. As we learn about z , we can increase the SNR. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 12 / 33

  27. 1-sparse recovery: adaptive upper bound x = e z + w Candidate set Signal 0 bits v SNR = 2 I ( z , � v , x � ) � log SNR = 1 � v , x � = v z + � v , w � Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 13 / 33

  28. 1-sparse recovery: adaptive upper bound x = e z + w Candidate set Signal 0 bits 1 bit v SNR = 2 2 I ( z , � v , x � ) � log SNR = 2 � v , x � = v z + � v , w � Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 13 / 33

  29. 1-sparse recovery: adaptive upper bound x = e z + w Candidate set Signal 0 bits 1 bit 2 bits v SNR = 2 4 I ( z , � v , x � ) � log SNR = 4 � v , x � = v z + � v , w � Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 13 / 33

  30. 1-sparse recovery: adaptive upper bound x = e z + w Candidate set Signal 0 bits 1 bit 2 bits 4 bits v SNR = 2 8 I ( z , � v , x � ) � log SNR = 8 � v , x � = v z + � v , w � Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 13 / 33

  31. 1-sparse recovery: adaptive upper bound x = e z + w Candidate set Signal 0 bits 1 bit 2 bits 4 bits 8 bits v SNR = 2 16 I ( z , � v , x � ) � log SNR = 16 � v , x � = v z + � v , w � Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 13 / 33

  32. 1-sparse recovery: adaptive lower bound Review of upper bound: ◮ Given b bits of information about z . ◮ Identifies z to set of size n / 2 b . ◮ Increases SNR , E [ v 2 z ] , by 2 b . ◮ Recover b bits of information in one measurement. ◮ 1 → 2 → · · · → log n in log log n measurements. ◮ R = 2: 1 → √ log n → log n in √ log n measurements/round. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 14 / 33

  33. 1-sparse recovery: adaptive lower bound Review of upper bound: ◮ Given b bits of information about z . ◮ Identifies z to set of size n / 2 b . ◮ Increases SNR , E [ v 2 z ] , by 2 b . ◮ Recover b bits of information in one measurement. ◮ 1 → 2 → · · · → log n in log log n measurements. ◮ R = 2: 1 → √ log n → log n in √ log n measurements/round. Lower bound outline: ◮ At each stage, have posterior distribution p on z . ◮ b = log n − H ( p ) bits known. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 14 / 33

  34. 1-sparse recovery: adaptive lower bound Review of upper bound: ◮ Given b bits of information about z . ◮ Identifies z to set of size n / 2 b . ◮ Increases SNR , E [ v 2 z ] , by 2 b . ◮ Recover b bits of information in one measurement. ◮ 1 → 2 → · · · → log n in log log n measurements. ◮ R = 2: 1 → √ log n → log n in √ log n measurements/round. Lower bound outline: ◮ At each stage, have posterior distribution p on z . ◮ b = log n − H ( p ) bits known. Lemma (Key lemma for k = 1) For any measurement vector v, I ( z ; � v , x � ) � b + 1 Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 14 / 33

  35. 1-sparse recovery: adaptive lower bound Lower bound outline: ◮ At each stage, have posterior distribution p on z . ◮ b = log n − H ( p ) bits known. ◮ Show any measurement gives O ( b + 1 ) bits of information. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 15 / 33

  36. 1-sparse recovery: adaptive lower bound Lower bound outline: ◮ At each stage, have posterior distribution p on z . ◮ b = log n − H ( p ) bits known. ◮ Show any measurement gives O ( b + 1 ) bits of information. Shannon-Hartley: � v 2 I ( z ; � v , x � ) � 1 z p z � v 2 2 log ( 1 + SNR ) � 1 + log z / n � 1 + n � p � ∞ Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 15 / 33

  37. 1-sparse recovery: adaptive lower bound Lower bound outline: ◮ At each stage, have posterior distribution p on z . ◮ b = log n − H ( p ) bits known. ◮ Show any measurement gives O ( b + 1 ) bits of information. Shannon-Hartley: � v 2 I ( z ; � v , x � ) � 1 z p z � v 2 2 log ( 1 + SNR ) � 1 + log z / n � 1 + n � p � ∞ Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 15 / 33

  38. 1-sparse recovery: adaptive lower bound Lower bound outline: ◮ At each stage, have posterior distribution p on z . ◮ b = log n − H ( p ) bits known. ◮ Show any measurement gives O ( b + 1 ) bits of information. Shannon-Hartley: � v 2 I ( z ; � v , x � ) � 1 z p z � v 2 2 log ( 1 + SNR ) � 1 + log z / n � 1 + n � p � ∞ Bound is good ( SNR ≈ 2 b ) when nonzero p z are similar. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 15 / 33

  39. 1-sparse recovery: adaptive lower bound Lower bound outline: ◮ At each stage, have posterior distribution p on z . ◮ b = log n − H ( p ) bits known. ◮ Show any measurement gives O ( b + 1 ) bits of information. Shannon-Hartley: � v 2 I ( z ; � v , x � ) � 1 z p z � v 2 2 log ( 1 + SNR ) � 1 + log z / n � 1 + n � p � ∞ Bound is good ( SNR ≈ 2 b ) when nonzero p z are similar. Can be terrible in general: b = 1 but SNR = n / log n . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 15 / 33

  40. 1-sparse recovery: adaptive lower bound Lower bound outline: ◮ At each stage, have posterior distribution p on z . ◮ b = log n − H ( p ) = � p z log np z bits known. ◮ Show any measurement gives O ( b + 1 ) bits of information. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 16 / 33

  41. 1-sparse recovery: adaptive lower bound Lower bound outline: ◮ At each stage, have posterior distribution p on z . ◮ b = log n − H ( p ) = � p z log np z bits known. ◮ Show any measurement gives O ( b + 1 ) bits of information. Partition indices into “level sets” S 0 , S 1 , . . . ⊆ [ n ] of p : Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 16 / 33

  42. 1-sparse recovery: adaptive lower bound Lower bound outline: ◮ At each stage, have posterior distribution p on z . ◮ b = log n − H ( p ) = � p z log np z bits known. ◮ Show any measurement gives O ( b + 1 ) bits of information. Partition indices into “level sets” S 0 , S 1 , . . . ⊆ [ n ] of p : ◮ S J = { z | p z ∈ [ 2 J / n , 2 J + 1 / n ] } Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 16 / 33

  43. 1-sparse recovery: adaptive lower bound Lower bound outline: ◮ At each stage, have posterior distribution p on z . ◮ b = log n − H ( p ) = � p z log np z bits known. ◮ Show any measurement gives O ( b + 1 ) bits of information. Partition indices into “level sets” S 0 , S 1 , . . . ⊆ [ n ] of p : ◮ S J = { z | p z ∈ [ 2 J / n , 2 J + 1 / n ] } ◮ E [ J ] � b . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 16 / 33

  44. 1-sparse recovery: adaptive lower bound Lower bound outline: ◮ At each stage, have posterior distribution p on z . ◮ b = log n − H ( p ) = � p z log np z bits known. ◮ Show any measurement gives O ( b + 1 ) bits of information. Partition indices into “level sets” S 0 , S 1 , . . . ⊆ [ n ] of p : ◮ S J = { z | p z ∈ [ 2 J / n , 2 J + 1 / n ] } ◮ E [ J ] � b . I ( z ; � v , x � ) � I ( z ; � v , x � | J ) + H ( J ) . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 16 / 33

  45. 1-sparse recovery: adaptive lower bound Lower bound outline: ◮ At each stage, have posterior distribution p on z . ◮ b = log n − H ( p ) = � p z log np z bits known. ◮ Show any measurement gives O ( b + 1 ) bits of information. Partition indices into “level sets” S 0 , S 1 , . . . ⊆ [ n ] of p : ◮ S J = { z | p z ∈ [ 2 J / n , 2 J + 1 / n ] } ◮ E [ J ] � b . I ( z ; � v , x � ) � I ( z ; � v , x � | J ) + H ( J ) . Shannon-Hartley: I ( z ; � v , x � | J = j ) � j + 1. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 16 / 33

  46. 1-sparse recovery: adaptive lower bound Lower bound outline: ◮ At each stage, have posterior distribution p on z . ◮ b = log n − H ( p ) = � p z log np z bits known. ◮ Show any measurement gives O ( b + 1 ) bits of information. Partition indices into “level sets” S 0 , S 1 , . . . ⊆ [ n ] of p : ◮ S J = { z | p z ∈ [ 2 J / n , 2 J + 1 / n ] } ◮ E [ J ] � b . I ( z ; � v , x � ) � I ( z ; � v , x � | J ) + H ( J ) . Shannon-Hartley: I ( z ; � v , x � | J = j ) � j + 1. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 16 / 33

  47. 1-sparse recovery: adaptive lower bound Lower bound outline: ◮ At each stage, have posterior distribution p on z . ◮ b = log n − H ( p ) = � p z log np z bits known. ◮ Show any measurement gives O ( b + 1 ) bits of information. Partition indices into “level sets” S 0 , S 1 , . . . ⊆ [ n ] of p : ◮ S J = { z | p z ∈ [ 2 J / n , 2 J + 1 / n ] } ◮ E [ J ] � b . I ( z ; � v , x � ) � I ( z ; � v , x � | J ) + H ( J ) . Shannon-Hartley: I ( z ; � v , x � | J = j ) � j + 1. Lemma (Key lemma for k = 1) I ( z ; � v , x � ) � b + 1 Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 16 / 33

  48. 1-sparse recovery: adaptive lower bound: finishing up Lemma (Key lemma for k = 1) I ( z ; � v , x � ) � b + 1 Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 17 / 33

  49. 1-sparse recovery: adaptive lower bound: finishing up Lemma (Key lemma for k = 1) I ( z ; � v , x � ) � b + 1 Suppose two rounds with m measurements each. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 17 / 33

  50. 1-sparse recovery: adaptive lower bound: finishing up Lemma (Key lemma for k = 1) I ( z ; � v , x � ) � b + 1 Suppose two rounds with m measurements each. ◮ O ( m ) bits learned in first round. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 17 / 33

  51. 1-sparse recovery: adaptive lower bound: finishing up Lemma (Key lemma for k = 1) I ( z ; � v , x � ) � b + 1 Suppose two rounds with m measurements each. ◮ O ( m ) bits learned in first round. ◮ O ( m 2 ) bits in second round. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 17 / 33

  52. 1-sparse recovery: adaptive lower bound: finishing up Lemma (Key lemma for k = 1) I ( z ; � v , x � ) � b + 1 Suppose two rounds with m measurements each. ◮ O ( m ) bits learned in first round. ◮ O ( m 2 ) bits in second round. ◮ Hence m � √ log n . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 17 / 33

  53. 1-sparse recovery: adaptive lower bound: finishing up Lemma (Key lemma for k = 1) I ( z ; � v , x � ) � b + 1 Suppose two rounds with m measurements each. ◮ O ( m ) bits learned in first round. ◮ O ( m 2 ) bits in second round. ◮ Hence m � √ log n . In general: Ω ( R log 1 / R n ) bits Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 17 / 33

  54. 1-sparse recovery: adaptive lower bound: finishing up Lemma (Key lemma for k = 1) I ( z ; � v , x � ) � b + 1 Suppose two rounds with m measurements each. ◮ O ( m ) bits learned in first round. ◮ O ( m 2 ) bits in second round. ◮ Hence m � √ log n . In general: Ω ( R log 1 / R n ) bits ◮ Ω ( log log n ) for unlimited R . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 17 / 33

  55. Outline Introduction 1 Analysis for k = 1 2 General k : lower bound 3 General k : upper bound 4 Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 18 / 33

  56. Recall the k = 1 proof outline Setting: x = e z + w for z ∼ p . p is posterior on z from previous measurements. Previous measurements had information content b := log n − H ( p ) Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 19 / 33

  57. Recall the k = 1 proof outline Setting: x = e z + w for z ∼ p . p is posterior on z from previous measurements. Previous measurements had information content b := log n − H ( p ) Lemma (Key lemma for k = 1) I ( z ; � v , x � ) � b + 1 Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 19 / 33

  58. Recall the k = 1 proof outline Setting: x = e z + w for z ∼ p . p is posterior on z from previous measurements. Previous measurements had information content b := log n − H ( p ) Lemma (Key lemma for k = 1) I ( z ; � v , x � ) � b + 1 Question : How to extend this to k > 1? Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 19 / 33

  59. Extending to general k Create k independent copies over domain N = nk . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 20 / 33

  60. Extending to general k Create k independent copies over domain N = nk . Formally: x = � k i = 1 e ni + Z i + w for Z ∈ [ n ] k , Z ∼ p . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 20 / 33

  61. Extending to general k Create k independent copies over domain N = nk . Formally: x = � k i = 1 e ni + Z i + w for Z ∈ [ n ] k , Z ∼ p . p is posterior from previous measurements. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 20 / 33

  62. Extending to general k Create k independent copies over domain N = nk . Formally: x = � k i = 1 e ni + Z i + w for Z ∈ [ n ] k , Z ∼ p . p is posterior from previous measurements. Previous measurements have information content b := k log n − H ( p ) Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 20 / 33

  63. Extending to general k Create k independent copies over domain N = nk . Formally: x = � k i = 1 e ni + Z i + w for Z ∈ [ n ] k , Z ∼ p . p is posterior from previous measurements. Previous measurements have information content b := k log n − H ( p ) Lemma (Key lemma for general k ) I ( Z ; � v , x � ) � b + 1 ???? Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 20 / 33

  64. What lemma do we want for general k ? I ( Z ; � v , x � ) � b + 1 Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 21 / 33

  65. What lemma do we want for general k ? I ( Z ; � v , x � ) � b + 1 ◮ True but too weak: would get Ω ( √ k log n ) not k √ log n . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 21 / 33

  66. What lemma do we want for general k ? I ( Z ; � v , x � ) � b + 1 ◮ True but too weak: would get Ω ( √ k log n ) not k √ log n . I ( Z ; � v , x � ) � b / k + 1 Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 21 / 33

  67. What lemma do we want for general k ? I ( Z ; � v , x � ) � b + 1 ◮ True but too weak: would get Ω ( √ k log n ) not k √ log n . I ( Z ; � v , x � ) � b / k + 1 ◮ Strong but false: if algorithm does 1-sparse recovery on first block, it really can learn Θ ( b + 1 ) bits. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 21 / 33

  68. What lemma do we want for general k ? I ( Z ; � v , x � ) � b + 1 ◮ True but too weak: would get Ω ( √ k log n ) not k √ log n . I ( Z ; � v , x � ) � b / k + 1 ◮ Strong but false: if algorithm does 1-sparse recovery on first block, it really can learn Θ ( b + 1 ) bits. ◮ But the learned bits are only about that first block. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 21 / 33

  69. What lemma do we want for general k ? I ( Z ; � v , x � ) � b + 1 ◮ True but too weak: would get Ω ( √ k log n ) not k √ log n . I ( Z ; � v , x � ) � b / k + 1 ◮ Strong but false: if algorithm does 1-sparse recovery on first block, it really can learn Θ ( b + 1 ) bits. ◮ But the learned bits are only about that first block. I ( Z W ; � v , x � ) � b / k + 1 for | W | > 0.99 k . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 21 / 33

  70. What lemma do we want for general k ? I ( Z ; � v , x � ) � b + 1 ◮ True but too weak: would get Ω ( √ k log n ) not k √ log n . I ( Z ; � v , x � ) � b / k + 1 ◮ Strong but false: if algorithm does 1-sparse recovery on first block, it really can learn Θ ( b + 1 ) bits. ◮ But the learned bits are only about that first block. I ( Z W ; � v , x � ) � b / k + 1 for | W | > 0.99 k . ◮ Strong enough, at least for constant R . Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 21 / 33

  71. What lemma do we want for general k ? I ( Z ; � v , x � ) � b + 1 ◮ True but too weak: would get Ω ( √ k log n ) not k √ log n . I ( Z ; � v , x � ) � b / k + 1 ◮ Strong but false: if algorithm does 1-sparse recovery on first block, it really can learn Θ ( b + 1 ) bits. ◮ But the learned bits are only about that first block. I ( Z W ; � v , x � ) � b / k + 1 for | W | > 0.99 k . ◮ Strong enough, at least for constant R . ◮ True for product distributions p ... Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 21 / 33

  72. What lemma do we want for general k ? I ( Z ; � v , x � ) � b + 1 ◮ True but too weak: would get Ω ( √ k log n ) not k √ log n . I ( Z ; � v , x � ) � b / k + 1 ◮ Strong but false: if algorithm does 1-sparse recovery on first block, it really can learn Θ ( b + 1 ) bits. ◮ But the learned bits are only about that first block. I ( Z W ; � v , x � ) � b / k + 1 for | W | > 0.99 k . ◮ Strong enough, at least for constant R . ◮ True for product distributions p ... ◮ but correlated p can make this false. Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 21 / 33

  73. What lemma do we want for general k ? I ( Z ; � v , x � ) � b + 1 ◮ True but too weak: would get Ω ( √ k log n ) not k √ log n . I ( Z ; � v , x � ) � b / k + 1 ◮ Strong but false: if algorithm does 1-sparse recovery on first block, it really can learn Θ ( b + 1 ) bits. ◮ But the learned bits are only about that first block. I ( Z W ; � v , x � ) � b / k + 1 for | W | > 0.99 k . ◮ Strong enough, at least for constant R . ◮ True for product distributions p ... ◮ but correlated p can make this false. I ( Z W ; � v , x � ) � b / k + log k Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 21 / 33

  74. What lemma do we want for general k ? I ( Z ; � v , x � ) � b + 1 ◮ True but too weak: would get Ω ( √ k log n ) not k √ log n . I ( Z ; � v , x � ) � b / k + 1 ◮ Strong but false: if algorithm does 1-sparse recovery on first block, it really can learn Θ ( b + 1 ) bits. ◮ But the learned bits are only about that first block. I ( Z W ; � v , x � ) � b / k + 1 for | W | > 0.99 k . ◮ Strong enough, at least for constant R . ◮ True for product distributions p ... ◮ but correlated p can make this false. I ( Z W ; � v , x � ) � b / k + log k ◮ True! Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 21 / 33

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Adaptive Sparse Recovery Eric Price MIT 2012-04-26 Joint work with Piotr Indyk and David

Adaptive Sparse Recovery Eric Price MIT 2012-04-26 Joint work with Piotr Indyk and David

Adaptive Sparse Recovery Eric Price MIT 2012-04-26 Joint work with Piotr Indyk and David Woodruff, 2011-2012 Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 1 / 29 Outline Motivating Example 1 Eric Price (MIT) Adaptive Sparse

1.92k views • 154 slides
Two added structures in sparse recovery: nonnegativity and disjointedness Simon Foucart

Two added structures in sparse recovery: nonnegativity and disjointedness Simon Foucart

Two added structures in sparse recovery: nonnegativity and disjointedness Simon Foucart University of Georgia Semester Program on High-Dimensional Approximation ICERM 7 October 2014 Part I: Nonnegative Sparse Recovery (joint work with

957 views • 69 slides
Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation

Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation

Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation (approximation theory, learning Fourier coeffs, linear sketching, finite rate of innovation, compressed sensing...) Setup: Data/signal in n-dimensional

418 views • 25 slides
Natural Analysts in Adaptive Data Analysis Tijana Zrnic joint with Moritz Hardt Adaptivity

Natural Analysts in Adaptive Data Analysis Tijana Zrnic joint with Moritz Hardt Adaptivity

Natural Analysts in Adaptive Data Analysis Tijana Zrnic joint with Moritz Hardt Adaptivity in Machine Learning Adaptivity in Machine Learning data analyst with training data Adaptivity in Machine Learning model data analyst with

819 views • 81 slides
Fostering Adaptivity in E-Learning Platforms: A Meta-Model Supporting Adaptive Courses Sabine

Fostering Adaptivity in E-Learning Platforms: A Meta-Model Supporting Adaptive Courses Sabine

Fostering Adaptivity in E-Learning Platforms: A Meta-Model Supporting Adaptive Courses Sabine Graf Vienna University of Technology, Austria Womens Postgraduate College for Internet Technologies graf@wit.tuwien.ac.at Motivation and Aims

555 views • 8 slides
Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems

Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems

Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems Evelyn Herrholz Gerd Teschke November 9, 2010 Abstract In this paper we shall be concerned with compressive sampling strategies and sparse

417 views • 30 slides
Bounds on Sparse Recovery with Additional Structures Abbas Kazemipour University of Maryland.

Bounds on Sparse Recovery with Additional Structures Abbas Kazemipour University of Maryland.

Bounds on Sparse Recovery with Additional Structures Abbas Kazemipour University of Maryland. College Park kaazemi@umd.edu March 23, 2015 Abbas Kazemipour (UMD) Sparse Recovery with Side Info. March 23, 2015 1 / 19 Overview 1 Restricted

599 views • 42 slides
Adaptive Stochastic Collocation for PDE-Constrained Optimization under Uncertainty using Sparse

Adaptive Stochastic Collocation for PDE-Constrained Optimization under Uncertainty using Sparse

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Adaptive Stochastic Collocation for PDE-Constrained Optimization under Uncertainty using Sparse Grids and Model Reduction Matthew J. Zahr Advisor: Charbel

612 views • 46 slides
Efficient PDE Optimization under Uncertainty using Adaptive Model Reduction and Sparse Grids

Efficient PDE Optimization under Uncertainty using Adaptive Model Reduction and Sparse Grids

Efficient PDE Optimization under Uncertainty using Adaptive Model Reduction and Sparse Grids Matthew J. Zahr Advisor: Charbel Farhat Computational and Mathematical Engineering Stanford University Joint work with: Kevin Carlberg (Sandia CA),

444 views • 28 slides
Sparse Recovery and Fourier Sampling Eric Price MIT Eric Price (MIT) Sparse Recovery and

Sparse Recovery and Fourier Sampling Eric Price MIT Eric Price (MIT) Sparse Recovery and

Sparse Recovery and Fourier Sampling Eric Price MIT Eric Price (MIT) Sparse Recovery and Fourier Sampling 1 / 37 The Fourier Transform Conversion between time and frequency domains Frequency Domain Time Domain Fourier Transform

2.15k views • 187 slides
fifty shades of adaptivity (in property testing) An Adaptivity Hierarchy Theorem for Property

fifty shades of adaptivity (in property testing) An Adaptivity Hierarchy Theorem for Property

fifty shades of adaptivity (in property testing) An Adaptivity Hierarchy Theorem for Property Testing Clment Canonne (Columbia University) July 9, 2017 Joint work with Tom Gur (Weizmann Institute UC Berkeley) property testing?

705 views • 67 slides
Axioms of Adaptivity Dirk Praetorius joint work with Carsten Carstensen (Berlin), Michael

Axioms of Adaptivity Dirk Praetorius joint work with Carsten Carstensen (Berlin), Michael

Adaptive algorithms for computational PDEs January 5, 2016 Axioms of Adaptivity Dirk Praetorius joint work with Carsten Carstensen (Berlin), Michael Feischl (Sydney), Kris van der Zee (Nottingham) TU Wien Institute for Analysis and Scientific

951 views • 52 slides
Q1) How important is the problem of adaptivity and its various guises as a cause of false

Q1) How important is the problem of adaptivity and its various guises as a cause of false

Q1) How important is the problem of adaptivity and its various guises as a cause of false discovery and false inference? Response: With very few exceptions scientific practice requires an approach capable of dealing with adaptivity from

384 views • 19 slides
Generating Sparse Representations by Adaptive Multiscale Approximations Angela Kunoth University

Generating Sparse Representations by Adaptive Multiscale Approximations Angela Kunoth University

Generating Sparse Representations by Adaptive Multiscale Approximations Angela Kunoth University of Cologne, Germany Angela Kunoth Generating Sparse Representations by Adaptive Multiscale Approximations 1 Generating Sparse Representations

737 views • 34 slides
Two problems from neural data analysis: Sparse entropy estimation and efficient adaptive

Two problems from neural data analysis: Sparse entropy estimation and efficient adaptive

Two problems from neural data analysis: Sparse entropy estimation and efficient adaptive experimental design Liam Paninski Department of Statistics and Center for Theoretical Neuroscience Columbia University http://www.stat.columbia.edu/

702 views • 38 slides
New Constructions of RIP Matrices with Fast Multiplication and Fewer Rows aka, sparse recovery

New Constructions of RIP Matrices with Fast Multiplication and Fewer Rows aka, sparse recovery

New Constructions of RIP Matrices with Fast Multiplication and Fewer Rows aka, sparse recovery from Fourier -like measurements with applications to fast Johnson-Lindenstrauss transforms , etc. Jelani Nelson, Eric Price, and Mary Wootters February

394 views • 24 slides
Shannons Formula &amp; Hartleys Rule: Olivier Rioul Jos Carlos Magossi

Shannons Formula &amp; Hartleys Rule: Olivier Rioul Jos Carlos Magossi

Shannons Formula &amp; Hartleys Rule: Olivier Rioul Jos Carlos Magossi Tlcom ParisTech Unicamp c l a u d e s h a n n o n 2/31 23 Sept 2014 Shannons Formula &amp; Hartleys Rule: A Mathematical Coincidence? c

1.11k views • 71 slides
Holographic Entanglement in Gauss-Bonnet gravity: time and shadows E LENA C CERES Facultad de

Holographic Entanglement in Gauss-Bonnet gravity: time and shadows E LENA C CERES Facultad de

Holographic Entanglement in Gauss-Bonnet gravity: time and shadows E LENA C CERES Facultad de Ciencias Universidad de Colima, Mexico Theory Group University of Texas at Austin (work in progress with M. Sanchez and J. Virrueta) March 25,

336 views • 29 slides
Tutorium CG2 LU Overview Shadow-Mapping Bloom / Glow Animation Institute of Computer Graphics

Tutorium CG2 LU Overview Shadow-Mapping Bloom / Glow Animation Institute of Computer Graphics

Tutorium CG2 LU Overview Shadow-Mapping Bloom / Glow Animation Institute of Computer Graphics and Algorithms Shadow Mapping Peter Houska Institute of Computer Graphics and Algorithms Vienna University of Technology Why Shadows? Institute

1.24k views • 87 slides
Assignments Please fill the dropboxes Lights Still catching up on grading.

Assignments Please fill the dropboxes Lights Still catching up on grading.

Assignments Please fill the dropboxes Lights Still catching up on grading. Questions. Final exam effects Final exam effect Deliverables: Presentation: Research (Grads only) Final exam period Shader code

457 views • 11 slides
On Mobile Edge Computing: Game Theory, Edge AI and Other New Ideas Hai-Liang Zhao

On Mobile Edge Computing: Game Theory, Edge AI and Other New Ideas Hai-Liang Zhao

On Mobile Edge Computing: Game Theory, Edge AI and Other New Ideas Hai-Liang Zhao hliangzhao97@gmail.com January 8, 2019 hliangzhao97@gmail.com On MEC: New Research Interests January 8, 2019 1 / 39 Outline 1 Game Theory and Its

1.29k views • 40 slides
Ilab2: Advanced Wireless Summer 2017 Prof. Dr.-Ing. Georg Carle Maurice Leclaire Chair of

Ilab2: Advanced Wireless Summer 2017 Prof. Dr.-Ing. Georg Carle Maurice Leclaire Chair of

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Ilab2: Advanced Wireless Summer 2017 Prof. Dr.-Ing. Georg Carle Maurice Leclaire Chair of Network Architectures and Services Department of

654 views • 63 slides
Third Quarter and Nine Months 2016 Financial Results 20 October 2016 1 Scope of Briefing

Third Quarter and Nine Months 2016 Financial Results 20 October 2016 1 Scope of Briefing

Third Quarter and Nine Months 2016 Financial Results 20 October 2016 1 Scope of Briefing Address by CEO Group Financial Highlights by CFO 2 Address by CEO 3 Macro Environment Slow global growth Continuing challenges in

712 views • 39 slides
Modern Aspects of Perturbative QFT and Gravity Bryan Larios in collaboration with J. Lorenzo

Modern Aspects of Perturbative QFT and Gravity Bryan Larios in collaboration with J. Lorenzo

Modern Aspects of Perturbative QFT and Gravity Bryan Larios in collaboration with J. Lorenzo Diaz-Cruz XXXI Annual Meeting DPyC-SMF May 24, 2017 Overview Introduction / Motivation. Basics of the Spinor Helicity Formalism (SHF).

479 views • 26 slides

More recommend