Detecting Sparse Structures in Data in Sub-Linear Time: A group - PowerPoint PPT Presentation

Detecting Sparse Structures in Data in Sub-Linear Time: A group testing approach Boaz Nadler The Weizmann Institute of Science Israel Joint works with Inbal Horev, Ronen Basri, Meirav Galun and Ery Arias-Castro Yi-Qing Wang, Alain Trouve, Yali Amit Roi Weiss, Chen Attias, Robert Krauthgamer Dec 2017 Boaz Nadler Sublinear Time Group Testing 1

Statistical Challenges related to ”big data” In various applications (vision in particular), we collect so much data, that either 1) data does not fit / cannot be processed on single machine Boaz Nadler Sublinear Time Group Testing 2

Statistical Challenges related to ”big data” In various applications (vision in particular), we collect so much data, that either 1) data does not fit / cannot be processed on single machine or 2) Standard algorithms that pass over all data are too slow / take too much computing power Boaz Nadler Sublinear Time Group Testing 2

Statistical Challenges related to ”big data” In various applications (vision in particular), we collect so much data, that either 1) data does not fit / cannot be processed on single machine or 2) Standard algorithms that pass over all data are too slow / take too much computing power Common approach to handle setting (1) is distributed learning Boaz Nadler Sublinear Time Group Testing 2

Statistical Challenges related to ”big data” In various applications (vision in particular), we collect so much data, that either 1) data does not fit / cannot be processed on single machine or 2) Standard algorithms that pass over all data are too slow / take too much computing power Common approach to handle setting (1) is distributed learning not focus of this talk , take a look at [Rosenblatt & N. 16’] On the optimality of averaging in distributed statistical learning Boaz Nadler Sublinear Time Group Testing 2

Statistical challenges related to ”big data” Focus of this talk: 2) Standard algorithms to solve a task are too slow Boaz Nadler Sublinear Time Group Testing 3

Statistical challenges related to ”big data” Focus of this talk: 2) Standard algorithms to solve a task are too slow Two key challenges: ◮ [ computational & practical ] develop extremely fast algorithms (linear / sub-linear complexity) ◮ [ theoretical ] understand lower bounds on statistical accuracy under computational constraints Boaz Nadler Sublinear Time Group Testing 3

Statistical challenges related to ”big data” Focus of this talk: 2) Standard algorithms to solve a task are too slow Two key challenges: ◮ [ computational & practical ] develop extremely fast algorithms (linear / sub-linear complexity) ◮ [ theoretical ] understand lower bounds on statistical accuracy under computational constraints In this talk: study these two challenges for (i) edge detection in large noisy images (ii) finding sparse representations in high dimensional dictionaries Boaz Nadler Sublinear Time Group Testing 3

Edge Detection Observe n 1 × n 2 image I = array of pixel values Goal: Detect edges in image, typically boundaries between objects. Search for curves Γ such that at direction n - normal to curve Γ , gradient ∇ I · n is large Boaz Nadler Sublinear Time Group Testing 4

Edge Detection Observe n 1 × n 2 image I = array of pixel values Goal: Detect edges in image, typically boundaries between objects. Search for curves Γ such that at direction n - normal to curve Γ , gradient ∇ I · n is large A fundamental task in low level image processing Boaz Nadler Sublinear Time Group Testing 4

Edge Detection Observe n 1 × n 2 image I = array of pixel values Goal: Detect edges in image, typically boundaries between objects. Search for curves Γ such that at direction n - normal to curve Γ , gradient ∇ I · n is large A fundamental task in low level image processing well studied problem, many algorithms well understood theory Boaz Nadler Sublinear Time Group Testing 4

Edge Detection at low SNR Our Interest: Edge detection in noisy and large 2D images and 3D video Motivation for large: high resolution images in many applications Motivation(s): for noisy images 1. Images at non-ideal conditions: poor lighting, fog, rain, night 2. surveillance applications 3. Real time object tracking in 3D video Boaz Nadler Sublinear Time Group Testing 5

Edge Detection at low SNR Our Interest: Edge detection in noisy and large 2D images and 3D video Motivation for large: high resolution images in many applications Motivation(s): for noisy images 1. Images at non-ideal conditions: poor lighting, fog, rain, night 2. surveillance applications 3. Real time object tracking in 3D video Image Prior: - Interested in long straight (or weakly curved) edges - Sparsity - image contains few edges Boaz Nadler Sublinear Time Group Testing 5

Example: Powerlines 200 400 600 800 1000 1200 1400 1600 1800 500 1000 1500 2000 2500 Boaz Nadler Sublinear Time Group Testing 6

Traditional Edge Detection Algorithms Typical Approach: Detect edges from local image gradients Example: Canny Edge Detector, complexity O ( n 2 ) linear in total number of image pixels fast, possibly suitable for real-time Limitation: Does not work well at low SNR Boaz Nadler Sublinear Time Group Testing 7

Example: Canny, run-time 2.5sec Boaz Nadler Sublinear Time Group Testing 8

Example: Canny, run-time 2.5sec Cannot detect faint powerlines of second tower Boaz Nadler Sublinear Time Group Testing 8

Modern Sophisticated Methods [Arias-Castro, Donoho, Huo, 05] [Brandt, Galun, Basri, 07] [Alpert, Galun, Nadler, Basri, 10] [Ofir, Galun, Nadler, Basri, 15] - Statistical theory for limits of detectability - (Theoretically) efficient multiscale algorithms, robust to noise Boaz Nadler Sublinear Time Group Testing 9

Modern Sophisticated Methods [Arias-Castro, Donoho, Huo, 05] [Brandt, Galun, Basri, 07] [Alpert, Galun, Nadler, Basri, 10] [Ofir, Galun, Nadler, Basri, 15] - Statistical theory for limits of detectability - (Theoretically) efficient multiscale algorithms, robust to noise and yet slow Boaz Nadler Sublinear Time Group Testing 9

Modern Sophisticated Methods [Arias-Castro, Donoho, Huo, 05] [Brandt, Galun, Basri, 07] [Alpert, Galun, Nadler, Basri, 10] [Ofir, Galun, Nadler, Basri, 15] - Statistical theory for limits of detectability - (Theoretically) efficient multiscale algorithms, robust to noise and yet slow Run time: O ( min ) for large images, O ( hours ) for video Boaz Nadler Sublinear Time Group Testing 9

Example: Straight Segment Detector, run-time 5 min Boaz Nadler Sublinear Time Group Testing 10

Challenge: Sublinear Time Edge Detection Goal: Devise edge detection algorith, that is (i) robust to noise and (ii) extremely fast Boaz Nadler Sublinear Time Group Testing 11

Challenge: Sublinear Time Edge Detection Goal: Devise edge detection algorith, that is (i) robust to noise and (ii) extremely fast Given noisy n × n image I , detect long straight edges in sublinear time Boaz Nadler Sublinear Time Group Testing 11

Challenge: Sublinear Time Edge Detection Goal: Devise edge detection algorith, that is (i) robust to noise and (ii) extremely fast Given noisy n × n image I , detect long straight edges in sublinear time complexity O ( n α ) with α < 2 Boaz Nadler Sublinear Time Group Testing 11

Challenge: Sublinear Time Edge Detection Goal: Devise edge detection algorith, that is (i) robust to noise and (ii) extremely fast Given noisy n × n image I , detect long straight edges in sublinear time complexity O ( n α ) with α < 2 touching only a fraction of the image/video pixels! Boaz Nadler Sublinear Time Group Testing 11

Challenge: Sublinear Time Edge Detection Goal: Devise edge detection algorith, that is (i) robust to noise and (ii) extremely fast Given noisy n × n image I , detect long straight edges in sublinear time complexity O ( n α ) with α < 2 touching only a fraction of the image/video pixels! Questions: a) Statistical: which edge strengths can one detect vs. α ? b) Computational: optimal sampling scheme ? c) Practical: sub-linear time algorithm ? Boaz Nadler Sublinear Time Group Testing 11

Problem Setup Observe n × n noisy image I = I 0 + ξ I 0 - noise free image ξ - i.i.d. additive noise, zero mean, variance σ 2 Boaz Nadler Sublinear Time Group Testing 12

Problem Setup Observe n × n noisy image I = I 0 + ξ I 0 - noise free image ξ - i.i.d. additive noise, zero mean, variance σ 2 Goal: Detect edges in I 0 from noisy I Boaz Nadler Sublinear Time Group Testing 12

Problem Setup Observe n × n noisy image I = I 0 + ξ I 0 - noise free image ξ - i.i.d. additive noise, zero mean, variance σ 2 Goal: Detect edges in I 0 from noisy I Assumptions: - Clean image I 0 contains few step edges (sparsity) Boaz Nadler Sublinear Time Group Testing 12

Problem Setup Observe n × n noisy image I = I 0 + ξ I 0 - noise free image ξ - i.i.d. additive noise, zero mean, variance σ 2 Goal: Detect edges in I 0 from noisy I Assumptions: - Clean image I 0 contains few step edges (sparsity) - Edges of interest are straight and sufficiently long Boaz Nadler Sublinear Time Group Testing 12