Detecting Sparse Structures in Data in Sub-Linear Time: A group - - PowerPoint PPT Presentation

▶
detecting sparse structures in data in sub linear time a
SMART_READER_LITE
LIVE PREVIEW

Detecting Sparse Structures in Data in Sub-Linear Time: A group - - PowerPoint PPT Presentation

Jan 10, 2023 155 likes •939 views

Detecting Sparse Structures in Data in Sub-Linear Time: A group testing approach Boaz Nadler The Weizmann Institute of Science Israel Joint works with Inbal Horev, Ronen Basri, Meirav Galun and Ery Arias-Castro Yi-Qing Wang, Alain Trouve,

slide-1
SLIDE 1

Detecting Sparse Structures in Data in Sub-Linear Time: A group testing approach

Boaz Nadler

The Weizmann Institute of Science Israel Joint works with Inbal Horev, Ronen Basri, Meirav Galun and Ery Arias-Castro Yi-Qing Wang, Alain Trouve, Yali Amit Roi Weiss, Chen Attias, Robert Krauthgamer

Dec 2017

Boaz Nadler Sublinear Time Group Testing 1

slide-2
SLIDE 2

Statistical Challenges related to ”big data”

In various applications (vision in particular), we collect so much data, that either 1) data does not fit / cannot be processed on single machine

Boaz Nadler Sublinear Time Group Testing 2

slide-3
SLIDE 3

Statistical Challenges related to ”big data”

In various applications (vision in particular), we collect so much data, that either 1) data does not fit / cannot be processed on single machine

  • r

2) Standard algorithms that pass over all data are too slow / take too much computing power

Boaz Nadler Sublinear Time Group Testing 2

slide-4
SLIDE 4

Statistical Challenges related to ”big data”

In various applications (vision in particular), we collect so much data, that either 1) data does not fit / cannot be processed on single machine

  • r

2) Standard algorithms that pass over all data are too slow / take too much computing power Common approach to handle setting (1) is distributed learning

Boaz Nadler Sublinear Time Group Testing 2

slide-5
SLIDE 5

Statistical Challenges related to ”big data”

In various applications (vision in particular), we collect so much data, that either 1) data does not fit / cannot be processed on single machine

  • r

2) Standard algorithms that pass over all data are too slow / take too much computing power Common approach to handle setting (1) is distributed learning not focus of this talk, take a look at [Rosenblatt & N. 16’] On the optimality of averaging in distributed statistical learning

Boaz Nadler Sublinear Time Group Testing 2

slide-6
SLIDE 6

Statistical challenges related to ”big data”

Focus of this talk: 2) Standard algorithms to solve a task are too slow

Boaz Nadler Sublinear Time Group Testing 3

slide-7
SLIDE 7

Statistical challenges related to ”big data”

Focus of this talk: 2) Standard algorithms to solve a task are too slow Two key challenges: ◮ [computational & practical] develop extremely fast algorithms (linear / sub-linear complexity) ◮ [theoretical] understand lower bounds on statistical accuracy under computational constraints

Boaz Nadler Sublinear Time Group Testing 3

slide-8
SLIDE 8

Statistical challenges related to ”big data”

Focus of this talk: 2) Standard algorithms to solve a task are too slow Two key challenges: ◮ [computational & practical] develop extremely fast algorithms (linear / sub-linear complexity) ◮ [theoretical] understand lower bounds on statistical accuracy under computational constraints In this talk: study these two challenges for (i) edge detection in large noisy images (ii) finding sparse representations in high dimensional dictionaries

Boaz Nadler Sublinear Time Group Testing 3

slide-9
SLIDE 9

Edge Detection

Observe n1 × n2 image I = array of pixel values Goal: Detect edges in image, typically boundaries between

  • bjects.

Search for curves Γ such that at direction n - normal to curve Γ, gradient ∇I · n is large

Boaz Nadler Sublinear Time Group Testing 4

slide-10
SLIDE 10

Edge Detection

Observe n1 × n2 image I = array of pixel values Goal: Detect edges in image, typically boundaries between

  • bjects.

Search for curves Γ such that at direction n - normal to curve Γ, gradient ∇I · n is large A fundamental task in low level image processing

Boaz Nadler Sublinear Time Group Testing 4

slide-11
SLIDE 11

Edge Detection

Observe n1 × n2 image I = array of pixel values Goal: Detect edges in image, typically boundaries between

  • bjects.

Search for curves Γ such that at direction n - normal to curve Γ, gradient ∇I · n is large A fundamental task in low level image processing well studied problem, many algorithms well understood theory

Boaz Nadler Sublinear Time Group Testing 4

slide-12
SLIDE 12

Edge Detection at low SNR

Our Interest: Edge detection in noisy and large 2D images and 3D video Motivation for large: high resolution images in many applications Motivation(s): for noisy images

  • 1. Images at non-ideal conditions: poor lighting, fog, rain,

night

  • 2. surveillance applications
  • 3. Real time object tracking in 3D video

Boaz Nadler Sublinear Time Group Testing 5

slide-13
SLIDE 13

Edge Detection at low SNR

Our Interest: Edge detection in noisy and large 2D images and 3D video Motivation for large: high resolution images in many applications Motivation(s): for noisy images

  • 1. Images at non-ideal conditions: poor lighting, fog, rain,

night

  • 2. surveillance applications
  • 3. Real time object tracking in 3D video

Image Prior:

  • Interested in long straight (or weakly curved) edges
  • Sparsity - image contains few edges

Boaz Nadler Sublinear Time Group Testing 5

slide-14
SLIDE 14

Example: Powerlines

500 1000 1500 2000 2500 200 400 600 800 1000 1200 1400 1600 1800 Boaz Nadler Sublinear Time Group Testing 6

slide-15
SLIDE 15

Traditional Edge Detection Algorithms

Typical Approach: Detect edges from local image gradients Example: Canny Edge Detector, complexity O(n2) linear in total number of image pixels fast, possibly suitable for real-time Limitation: Does not work well at low SNR

Boaz Nadler Sublinear Time Group Testing 7

slide-16
SLIDE 16

Example: Canny, run-time 2.5sec

Boaz Nadler Sublinear Time Group Testing 8

slide-17
SLIDE 17

Example: Canny, run-time 2.5sec

Cannot detect faint powerlines of second tower

Boaz Nadler Sublinear Time Group Testing 8

slide-18
SLIDE 18

Modern Sophisticated Methods

[Arias-Castro, Donoho, Huo, 05] [Brandt, Galun, Basri, 07] [Alpert, Galun, Nadler, Basri, 10] [Ofir, Galun, Nadler, Basri, 15]

  • Statistical theory for limits of detectability
  • (Theoretically) efficient multiscale algorithms, robust to noise

Boaz Nadler Sublinear Time Group Testing 9

slide-19
SLIDE 19

Modern Sophisticated Methods

[Arias-Castro, Donoho, Huo, 05] [Brandt, Galun, Basri, 07] [Alpert, Galun, Nadler, Basri, 10] [Ofir, Galun, Nadler, Basri, 15]

  • Statistical theory for limits of detectability
  • (Theoretically) efficient multiscale algorithms, robust to noise

and yet slow

Boaz Nadler Sublinear Time Group Testing 9

slide-20
SLIDE 20

Modern Sophisticated Methods

[Arias-Castro, Donoho, Huo, 05] [Brandt, Galun, Basri, 07] [Alpert, Galun, Nadler, Basri, 10] [Ofir, Galun, Nadler, Basri, 15]

  • Statistical theory for limits of detectability
  • (Theoretically) efficient multiscale algorithms, robust to noise

and yet slow Run time: O(min) for large images, O(hours) for video

Boaz Nadler Sublinear Time Group Testing 9

slide-21
SLIDE 21

Example: Straight Segment Detector, run-time 5 min

Boaz Nadler Sublinear Time Group Testing 10

slide-22
SLIDE 22

Challenge: Sublinear Time Edge Detection

Goal: Devise edge detection algorith, that is (i) robust to noise and (ii) extremely fast

Boaz Nadler Sublinear Time Group Testing 11

slide-23
SLIDE 23

Challenge: Sublinear Time Edge Detection

Goal: Devise edge detection algorith, that is (i) robust to noise and (ii) extremely fast Given noisy n × n image I, detect long straight edges in sublinear time

Boaz Nadler Sublinear Time Group Testing 11

slide-24
SLIDE 24

Challenge: Sublinear Time Edge Detection

Goal: Devise edge detection algorith, that is (i) robust to noise and (ii) extremely fast Given noisy n × n image I, detect long straight edges in sublinear time complexity O(nα) with α < 2

Boaz Nadler Sublinear Time Group Testing 11

slide-25
SLIDE 25

Challenge: Sublinear Time Edge Detection

Goal: Devise edge detection algorith, that is (i) robust to noise and (ii) extremely fast Given noisy n × n image I, detect long straight edges in sublinear time complexity O(nα) with α < 2 touching only a fraction of the image/video pixels!

Boaz Nadler Sublinear Time Group Testing 11

slide-26
SLIDE 26

Challenge: Sublinear Time Edge Detection

Goal: Devise edge detection algorith, that is (i) robust to noise and (ii) extremely fast Given noisy n × n image I, detect long straight edges in sublinear time complexity O(nα) with α < 2 touching only a fraction of the image/video pixels! Questions: a) Statistical: which edge strengths can one detect vs. α ? b) Computational: optimal sampling scheme ? c) Practical: sub-linear time algorithm ?

Boaz Nadler Sublinear Time Group Testing 11

slide-27
SLIDE 27

Problem Setup

Observe n × n noisy image I = I0 + ξ I0 - noise free image ξ - i.i.d. additive noise, zero mean, variance σ2

Boaz Nadler Sublinear Time Group Testing 12

slide-28
SLIDE 28

Problem Setup

Observe n × n noisy image I = I0 + ξ I0 - noise free image ξ - i.i.d. additive noise, zero mean, variance σ2 Goal: Detect edges in I0 from noisy I

Boaz Nadler Sublinear Time Group Testing 12

slide-29
SLIDE 29

Problem Setup

Observe n × n noisy image I = I0 + ξ I0 - noise free image ξ - i.i.d. additive noise, zero mean, variance σ2 Goal: Detect edges in I0 from noisy I Assumptions:

  • Clean image I0 contains few step edges (sparsity)

Boaz Nadler Sublinear Time Group Testing 12

slide-30
SLIDE 30

Problem Setup

Observe n × n noisy image I = I0 + ξ I0 - noise free image ξ - i.i.d. additive noise, zero mean, variance σ2 Goal: Detect edges in I0 from noisy I Assumptions:

  • Clean image I0 contains few step edges (sparsity)
  • Edges of interest are straight and sufficiently long

Boaz Nadler Sublinear Time Group Testing 12

slide-31
SLIDE 31

Problem Setup

Observe n × n noisy image I = I0 + ξ I0 - noise free image ξ - i.i.d. additive noise, zero mean, variance σ2 Goal: Detect edges in I0 from noisy I Assumptions:

  • Clean image I0 contains few step edges (sparsity)
  • Edges of interest are straight and sufficiently long

Definition: Edge Signal to Noise Ratio = edge contrast/σ.

Boaz Nadler Sublinear Time Group Testing 12

slide-32
SLIDE 32

Theoretical Questions

Given sub-linear budget: 1) what are optimal sampling schemes ? 2) what are fundamental limitations on sub-linear edge detection ? 3) what is the tradeoff between statistical accuracy and computational complexity ?

Boaz Nadler Sublinear Time Group Testing 13

slide-33
SLIDE 33

Optimal Sublinear Edge Detection

For theoretical analysis, consider following class of images: I = {I contains only noise or one long fiber plus noise}

Boaz Nadler Sublinear Time Group Testing 14

slide-34
SLIDE 34

Optimal Sublinear Edge Detection

For theoretical analysis, consider following class of images: I = {I contains only noise or one long fiber plus noise}

Boaz Nadler Sublinear Time Group Testing 14

slide-35
SLIDE 35

Fundamental Limitations / Design Principles

Focus on detection under worst-case scenario

Boaz Nadler Sublinear Time Group Testing 15

slide-36
SLIDE 36

Fundamental Limitations / Design Principles

Focus on detection under worst-case scenario Lemma: If number of observed pixels is nα with α < 1 then there exists I ∈ I whose edges cannot be detected

Boaz Nadler Sublinear Time Group Testing 15

slide-37
SLIDE 37

Fundamental Limitations / Design Principles

Focus on detection under worst-case scenario Lemma: If number of observed pixels is nα with α < 1 then there exists I ∈ I whose edges cannot be detected Theorem: Assume number of observed pixels is s and s/n is

  • integer. Then,

i) any optimal sampling scheme must observe exactly s/n pixels per row. ii) sampling s/n whole columns is an optimal scheme

Boaz Nadler Sublinear Time Group Testing 15

slide-38
SLIDE 38

Statistical Accuracy vs. Computational Complexity

Definition: Edge SNR = edge contrast / noise level

Boaz Nadler Sublinear Time Group Testing 16

slide-39
SLIDE 39

Statistical Accuracy vs. Computational Complexity

Definition: Edge SNR = edge contrast / noise level Theorem: At complexity O(nα), with α ≥ 1, SNR

  • ln n/nα−1

0.5 1 1.5 2 0.5 1 1.5 2 2.5 3 3.5 4 α SNR not possible ←

  • ln(n)

←

  • ln n/nα−1

Boaz Nadler Sublinear Time Group Testing 16

slide-40
SLIDE 40

Sublinear Edge Detection Algorithm

Key Idea: Sample few image strips

Boaz Nadler Sublinear Time Group Testing 17

slide-41
SLIDE 41

Sublinear Edge Detection Algorithm

Key Idea: Sample few image strips first detect edges in strips

Boaz Nadler Sublinear Time Group Testing 17

slide-42
SLIDE 42

Sublinear Edge Detection Algorithm

Key Idea: Sample few image strips first detect edges in strips next: non-maximal suppression, edge localization

Boaz Nadler Sublinear Time Group Testing 17

slide-43
SLIDE 43

Example:

NOISY IMAGE, SNR=1 Boaz Nadler Sublinear Time Group Testing 18

slide-44
SLIDE 44

Example:

NOISY IMAGE, SNR=1 CANNY Boaz Nadler Sublinear Time Group Testing 18

slide-45
SLIDE 45

Example:

NOISY IMAGE, SNR=1 CANNY SUB−LINEAR Boaz Nadler Sublinear Time Group Testing 18

slide-46
SLIDE 46

Sublinear Edge Detection, run-time few seconds

Boaz Nadler Sublinear Time Group Testing 19

slide-47
SLIDE 47

Some Results on Real Images

Detection with roughly 10% of pixels sampled.

Boaz Nadler Sublinear Time Group Testing 20

slide-48
SLIDE 48

Real Images

Boaz Nadler Sublinear Time Group Testing 21

slide-49
SLIDE 49

Part II: Sparse Representations in High Dimensions

[with Chen Attias and Roi Weiss] Problem Setup: ◮ A dictionary Φ = [ϕ1, ϕ2, . . . , ϕN] ∈ Rp×N ◮ Each atom ϕi normalized ϕi = 1 ◮ High-dimensional p ≫ 1 ◮ Possibly Redundant N = Jp with J ≥ 1 Definition: A signal s ∈ Rp is m-sparse over dictionary Φ if s = Φα where | supp(α)| = m ≪ p

Boaz Nadler Sublinear Time Group Testing 22

slide-50
SLIDE 50

Part II: Sparse Representations in High Dimensions

[with Chen Attias and Roi Weiss] Problem Setup: ◮ A dictionary Φ = [ϕ1, ϕ2, . . . , ϕN] ∈ Rp×N ◮ Each atom ϕi normalized ϕi = 1 ◮ High-dimensional p ≫ 1 ◮ Possibly Redundant N = Jp with J ≥ 1 Definition: A signal s ∈ Rp is m-sparse over dictionary Φ if s = Φα where | supp(α)| = m ≪ p Goal: Given (noisy version of) s, find α.

Boaz Nadler Sublinear Time Group Testing 22

slide-51
SLIDE 51

Part II: Sparse Representations in High Dimensions

[with Chen Attias and Roi Weiss] Problem Setup: ◮ A dictionary Φ = [ϕ1, ϕ2, . . . , ϕN] ∈ Rp×N ◮ Each atom ϕi normalized ϕi = 1 ◮ High-dimensional p ≫ 1 ◮ Possibly Redundant N = Jp with J ≥ 1 Definition: A signal s ∈ Rp is m-sparse over dictionary Φ if s = Φα where | supp(α)| = m ≪ p Goal: Given (noisy version of) s, find α. Applications: Image and signal analysis.

Boaz Nadler Sublinear Time Group Testing 22

slide-52
SLIDE 52

Computing Sparse Representation

[Davis et al, 97’] With no assumptions on Φ and on m, problem is NP-hard Key challenge: find the support. Once supp(α) is known, recovering α requires O(pm2)

  • perations.

Boaz Nadler Sublinear Time Group Testing 23

slide-53
SLIDE 53

Computing Sparse Representation

[Davis et al, 97’] With no assumptions on Φ and on m, problem is NP-hard Key challenge: find the support. Once supp(α) is known, recovering α requires O(pm2)

  • perations.

Definition: The coherence µ of a dictionary Φ is µ = max

i=j |

  • ϕi, ϕj
  • |

m-sparse signal satisfies MUTUAL-INCOHERENCE-PROPERTY (MIP) if (2m − 1)µ < 1

Boaz Nadler Sublinear Time Group Testing 23

slide-54
SLIDE 54

Orthogonal Matching Pursuit

[Donoho & Elad ’03, Tropp ’04, others...]

Theorem: Suppose m-sparse signal s = Φα satisfies MIP

  • condition. Then, solution of Basis-Pursuit (BP) problem

arg min

x∈RN x1

s.t. s = Φx recovers representation α exactly Orthogonal Matching Pursuit (OMP) also recovers α exactly

Boaz Nadler Sublinear Time Group Testing 24

slide-55
SLIDE 55

Orthogonal Matching Pursuit

[Donoho & Elad ’03, Tropp ’04, others...]

Theorem: Suppose m-sparse signal s = Φα satisfies MIP

  • condition. Then, solution of Basis-Pursuit (BP) problem

arg min

x∈RN x1

s.t. s = Φx recovers representation α exactly Orthogonal Matching Pursuit (OMP) also recovers α exactly Time complexity of OMP is O(mp2) and of BP even higher.

Boaz Nadler Sublinear Time Group Testing 24

slide-56
SLIDE 56

Orthogonal Matching Pursuit

[Donoho & Elad ’03, Tropp ’04, others...]

Theorem: Suppose m-sparse signal s = Φα satisfies MIP

  • condition. Then, solution of Basis-Pursuit (BP) problem

arg min

x∈RN x1

s.t. s = Φx recovers representation α exactly Orthogonal Matching Pursuit (OMP) also recovers α exactly Time complexity of OMP is O(mp2) and of BP even higher. Can one compute a sparse representation faster?

Boaz Nadler Sublinear Time Group Testing 24

slide-57
SLIDE 57

Runtime of Orthogonal Matching Pursuit

Key quantity for identifying significant atoms Ik ck = Φ⊤rk−1 ∈ RN with all inner products between rk−1 and all N atoms in Φ

Boaz Nadler Sublinear Time Group Testing 25

slide-58
SLIDE 58

Runtime of Orthogonal Matching Pursuit

Key quantity for identifying significant atoms Ik ck = Φ⊤rk−1 ∈ RN with all inner products between rk−1 and all N atoms in Φ Computing all N inner products

Boaz Nadler Sublinear Time Group Testing 25

slide-59
SLIDE 59

Runtime of Orthogonal Matching Pursuit

Key quantity for identifying significant atoms Ik ck = Φ⊤rk−1 ∈ RN with all inner products between rk−1 and all N atoms in Φ Computing all N inner products ◮ For structured Φ — runtime ∝ N log N

(Fourier, wavelet, sparse-graph codes, ...)

Boaz Nadler Sublinear Time Group Testing 25

slide-60
SLIDE 60

Runtime of Orthogonal Matching Pursuit

Key quantity for identifying significant atoms Ik ck = Φ⊤rk−1 ∈ RN with all inner products between rk−1 and all N atoms in Φ Computing all N inner products ◮ For structured Φ — runtime ∝ N log N

(Fourier, wavelet, sparse-graph codes, ...)

◮ For non-structured Φ — runtime ∝ Np

Boaz Nadler Sublinear Time Group Testing 25

slide-61
SLIDE 61

Runtime of Orthogonal Matching Pursuit

Key quantity for identifying significant atoms Ik ck = Φ⊤rk−1 ∈ RN with all inner products between rk−1 and all N atoms in Φ Computing all N inner products ◮ For structured Φ — runtime ∝ N log N

(Fourier, wavelet, sparse-graph codes, ...)

◮ For non-structured Φ — runtime ∝ Np If N = Jp ⇒ runtime quadratic in signal dimension O(p2)

Boaz Nadler Sublinear Time Group Testing 25

slide-62
SLIDE 62

Runtime of Orthogonal Matching Pursuit

Key quantity for identifying significant atoms Ik ck = Φ⊤rk−1 ∈ RN with all inner products between rk−1 and all N atoms in Φ Computing all N inner products ◮ For structured Φ — runtime ∝ N log N

(Fourier, wavelet, sparse-graph codes, ...)

◮ For non-structured Φ — runtime ∝ Np If N = Jp ⇒ runtime quadratic in signal dimension O(p2) Question: Identify largest entries of ck in nearly-linear time ?

Boaz Nadler Sublinear Time Group Testing 25

slide-63
SLIDE 63

Our contribution: A group testing approach

Use (random) Tree-Based Statistical-Group-Testing data structure

Boaz Nadler Sublinear Time Group Testing 26

slide-64
SLIDE 64

Our contribution: A group testing approach

Use (random) Tree-Based Statistical-Group-Testing data structure ◮ Finds at least one atom from rk−1’s representation w.h.p.

Boaz Nadler Sublinear Time Group Testing 26

slide-65
SLIDE 65

Our contribution: A group testing approach

Use (random) Tree-Based Statistical-Group-Testing data structure ◮ Finds at least one atom from rk−1’s representation w.h.p. ◮ If coherence µ = O(1/√p) ⇒ O

  • pm2 · log(pm)
  • time

Boaz Nadler Sublinear Time Group Testing 26

slide-66
SLIDE 66

Our contribution: A group testing approach

Use (random) Tree-Based Statistical-Group-Testing data structure ◮ Finds at least one atom from rk−1’s representation w.h.p. ◮ If coherence µ = O(1/√p) ⇒ O

  • pm2 · log(pm)
  • time

◮ Memory: O(Np · log (pm))

Boaz Nadler Sublinear Time Group Testing 26

slide-67
SLIDE 67

Our contribution: A group testing approach

Use (random) Tree-Based Statistical-Group-Testing data structure ◮ Finds at least one atom from rk−1’s representation w.h.p. ◮ If coherence µ = O(1/√p) ⇒ O

  • pm2 · log(pm)
  • time

◮ Memory: O(Np · log (pm)) OMP + Statistical-Group-Testing (GT-OMP) ◮ At most m iterations for exact recovery

Boaz Nadler Sublinear Time Group Testing 26

slide-68
SLIDE 68

Our contribution: A group testing approach

Use (random) Tree-Based Statistical-Group-Testing data structure ◮ Finds at least one atom from rk−1’s representation w.h.p. ◮ If coherence µ = O(1/√p) ⇒ O

  • pm2 · log(pm)
  • time

◮ Memory: O(Np · log (pm)) OMP + Statistical-Group-Testing (GT-OMP) ◮ At most m iterations for exact recovery ◮ Total runtime O

  • pm3 · log (pm)
  • Boaz Nadler

Sublinear Time Group Testing 26

slide-69
SLIDE 69

Our contribution: A group testing approach

Use (random) Tree-Based Statistical-Group-Testing data structure ◮ Finds at least one atom from rk−1’s representation w.h.p. ◮ If coherence µ = O(1/√p) ⇒ O

  • pm2 · log(pm)
  • time

◮ Memory: O(Np · log (pm)) OMP + Statistical-Group-Testing (GT-OMP) ◮ At most m iterations for exact recovery ◮ Total runtime O

  • pm3 · log (pm)
  • ◮ For sparsity m = o(p1/3) — runtime sub-quadratic in p

(recall MIP requires m = O(p1/2))

Boaz Nadler Sublinear Time Group Testing 26

slide-70
SLIDE 70

Our contribution: A group testing approach

Use (random) Tree-Based Statistical-Group-Testing data structure ◮ Finds at least one atom from rk−1’s representation w.h.p. ◮ If coherence µ = O(1/√p) ⇒ O

  • pm2 · log(pm)
  • time

◮ Memory: O(Np · log (pm)) OMP + Statistical-Group-Testing (GT-OMP) ◮ At most m iterations for exact recovery ◮ Total runtime O

  • pm3 · log (pm)
  • ◮ For sparsity m = o(p1/3) — runtime sub-quadratic in p

(recall MIP requires m = O(p1/2))

◮ For sparsity m = O(log p) — runtime near-linear in p

Boaz Nadler Sublinear Time Group Testing 26

slide-71
SLIDE 71

Simulations results

◮ random Gaussian dictionary N = 2p, Φij ∼ N(0, 1/√p). ◮ sparsity m = 4 log2 p ◮ Compared 4 algorithms:

  • OMP [Pati et al 93’, Mallat & Zhang 92’]
  • OMP-threshold [Yang & de Hoog 15’]
  • Stagewise OMP [Donoho et al 12’]
  • GT-OMP (ours)

◮ Averaged over 25 random signals ◮ All 4 algorithms 100% success in recovering all 25 signals

Boaz Nadler Sublinear Time Group Testing 27

slide-72
SLIDE 72

Runtime vs. dimension

1024 2048 4096 8192 10

4

10

5

10

6

dimension p

  • no. of inner products

OMP OMP−T StOMP GT−OMP

Boaz Nadler Sublinear Time Group Testing 28

slide-73
SLIDE 73

Summary

Increasing need for fast (possibly sub-linear) algorithms in various “big-data” applications Considered edge detection and sparse representation

Boaz Nadler Sublinear Time Group Testing 29

slide-74
SLIDE 74

Summary

Increasing need for fast (possibly sub-linear) algorithms in various “big-data” applications Considered edge detection and sparse representation Common Theme to both problems:

  • Formulate estimation as a search in a very large space of

possible hypothesis

  • Construct coarse tests that rule out many hypothesis at once
  • Perform more expensive/accurate tests on remaining

hypotheses

Boaz Nadler Sublinear Time Group Testing 29

slide-75
SLIDE 75

Summary

Increasing need for fast (possibly sub-linear) algorithms in various “big-data” applications Considered edge detection and sparse representation Common Theme to both problems:

  • Formulate estimation as a search in a very large space of

possible hypothesis

  • Construct coarse tests that rule out many hypothesis at once
  • Perform more expensive/accurate tests on remaining

hypotheses Similar ideas: Gilbert et al, Willett et al 14’, Meinshausen et al 09’, Haupt et al,...

Boaz Nadler Sublinear Time Group Testing 29

slide-76
SLIDE 76

Summary

Open Questions:

  • Can similar approach solve other inference/learning

problems?

  • Precisely quantify statistical vs. computational tradeoffs for
  • ther problems?

Boaz Nadler Sublinear Time Group Testing 30

slide-77
SLIDE 77

The End

Still a long way to go Thank you ! www.weizmann.ac.il/math/Nadler/

Boaz Nadler Sublinear Time Group Testing 31