Sparse PCA refusing to graduate :-) Aviad Rubinstein (UC Berkeley) - PowerPoint PPT Presentation

Sep 17, 2023 •113 likes •166 views

Best* Case Approximability of Sparse PCA refusing to graduate :-) Aviad Rubinstein (UC Berkeley) Joint work with Siu-On Chan and Dimitris Papailliopoulos Sparse Principal Component Analysis max s.t. 2 = 1 and 0

Best* Case Approximability of Sparse PCA refusing to graduate :-) Aviad Rubinstein (UC Berkeley) Joint work with Siu-On Chan and Dimitris Papailliopoulos
Sparse Principal Component Analysis max 𝑦 ⊤ 𝐵𝑦 s.t. 𝑦 2 = 1 and 𝑦 0 ≤ 𝑙 (and 𝐵 is PSD) 7.1 1.3 ⋯ 4.5 −2.6 −3.4 ⋯ 6.2 ⋮ ⋮ ⋱ ⋮ 3.1 9.2 ⋯ −4.8 “Yes we SPCA!” -Obama, 2008
Sparse Spiked Covariance Model (“average case”) rank 1 ⋯ 0 𝑙 × 𝑙 block 𝐵 = 𝐽 𝑜 + +noise ⋮ ⋱ ⋮ 0 ⋯ 0 • Cool sample-complexity / computational-complexity tradeoff [Berthet & Rigollet ’13 , Wang et al ’14 , Gao et al ’15 , Kraugthgamer et al ’15 , Ma & Wigderson ’15] • Good news: trivial algorithm gives (1 − 𝑝 1 ) -approximation • Bad news: this is not what your data looks like!
Our results: “best - case” analysis Computationally intractable even when given the exact covariance matrix: • NP-hard to approximate to within (1 − 𝜗) • SSE-hard to approximate to within any 𝑑 • Quasi-quasi-poly ( 𝑓 𝑓 lnln𝑜 ) integrality gap • 𝑜 −1/3 -approximation algorithm “Vive la SPCA!” -Napoleon, 1808

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Sparse Matrices Example Of Sparse Matrices diagonal tridiagonal sparse many elements are

Sparse Matrices Example Of Sparse Matrices diagonal tridiagonal sparse many elements are zero lower triangular (?) dense few elements are zero These are structured sparse matrices. May be mapped into a 1D array so that a mapping

391 views • 6 slides

Sparse Matrices sparse many elements are zero dense few elements are zero Example Of

Sparse Matrices sparse many elements are zero dense few elements are zero Example Of Sparse Matrices diagonal tridiagonal lower triangular (?) These are structured sparse matrices. May be mapped into a 1D array so that a mapping

698 views • 11 slides

= + N 1 N M measurements M 1 sparse signal Problem : Solve for x Basis pursuit,

Sparse processing / Compressed sensing Model : y = Ax + n , x is sparse = + N 1 N M measurements M 1 sparse signal Problem : Solve for x Basis pursuit, LASSO (convex objective function) Matching pursuit (greedy method)

210 views • 17 slides

Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation

Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation (approximation theory, learning Fourier coeffs, linear sketching, finite rate of innovation, compressed sensing...) Setup: Data/signal in n-dimensional

418 views • 25 slides

Sparse Representations The Sparse Representations (SRs) framework was inspired by studies of

Manifold Based Sparse Representation for Robust Expression Recognition without Neutral Subtraction Raymond Ptucha, Grigorios Tsagkatakis, Andreas Savakis Department of Computer Engineering, Rochester Institute of Technology, Rochester, NY Nov

411 views • 11 slides

Tutorial: TF-Ranking for sparse features Tutorial: TF-Ranking for sparse features This tutorial

Tutorial: TF-Ranking for sparse features Tutorial: TF-Ranking for sparse features This tutorial is an end-to-end walkthrough of training a TensorFlow Ranking (TF-Ranking) neural network model which incorporates sparse textual features.

257 views • 24 slides

Extremal results for sparse pseudorandom graphs Yufei Zhao Massachusetts Institute of Technology

Extremal results for sparse pseudorandom graphs Yufei Zhao Massachusetts Institute of Technology Joint work with David Conlon and Jacob Fox Sparse extensions Extending classical results to sparse settings. For example: Sparse extensions

882 views • 55 slides

Parallel Numerical Algorithms Chapter 4 Sparse Linear Systems Section 4.1 Direct Methods

Sparse Matrices Sparse Triangular Solve Cholesky Factorization Sparse Cholesky Factorization Parallel Numerical Algorithms Chapter 4 Sparse Linear Systems Section 4.1 Direct Methods Michael T. Heath and Edgar Solomonik Department of

906 views • 63 slides

Outline Background Sparse Coding Semi-supervised Learning with Sparse Coding

Sparse-coded Net Model and Applications Y. Gwon, M. Cha, W. Campbell, H.T. Kung, C. Dagli IEEE International Workshop on Machine Learning for Signal Processing (MLSP 2016) September 16, 2016 This work is sponsored by the

416 views • 17 slides

CNBC Matlab Mini-Course Sparse Matrices Sparse matrices provide an efficient means to store

CNBC Matlab Mini-Course Sparse Matrices Sparse matrices provide an efficient means to store matrices that are mostly empty. David S. Touretzky s = sparse(1000, 1000) October 2019 s(2, 11) = 2 s(992, 875) = 3 Day 3: The Really Cool Stuff

550 views • 6 slides

sparse matrices and graphs L. Olson Department of Computer Science University of Illinois at

sparse matrices and graphs L. Olson Department of Computer Science University of Illinois at Urbana-Champaign 1 objectives Convert a graph into a sparse matrix Go over a few sparse matrix storage formats Give an example of lower

513 views • 24 slides

Exploiting GPU Caches in Sparse Matrix Vector Multiplication Yusuke Nagasaka Tokyo Institute of

Exploiting GPU Caches in Sparse Matrix Vector Multiplication Yusuke Nagasaka Tokyo Institute of Technology Sparse Matrix Generated by FEM, being as the graph data Often require solving sparse linear equation fast Iterative method :

629 views • 30 slides

Sparse and TV Kaczmarz solvers and the linearized Bregman method Dirk Lorenz, Frank Schpfer,

Sparse and TV Kaczmarz solvers and the linearized Bregman method Dirk Lorenz, Frank Schpfer, Stephan Wenger, Marcus Magnor, March, 2014 Sparse Tomo Days, DTU Motivation Split feasibility problems Sparse Kaczmarz and TV-Kaczmarz Application

1.04k views • 67 slides

MLSS 06 - Canberra Elements Hierarchical Basis Sparse Grids Sparse Grids Combination

MLSS 06 - Canberra Sparse Grids Jochen Garcke (Very) Short Course on Finite MLSS 06 - Canberra Elements Hierarchical Basis Sparse Grids Sparse Grids Combination Technique Jochen Garcke Regression / Classification via Function

977 views • 68 slides

Parameter efficient training of deep convolutional neural networks by dynamic sparse

Parameter efficient training of deep convolutional neural networks by dynamic sparse reparameterization Hesham Mostafa (Intel AI) Xin Wang (Intel AI, Cerebras Systems) Easy : post-training (sparse) compression Hard : direct training of sparse

727 views • 9 slides

Lecture 5 : Sparse Models Homework 3 discussion (Nima) Sparse Models Lecture - Reading :

Lecture 5 : Sparse Models Homework 3 discussion (Nima) Sparse Models Lecture - Reading : Murphy, Chapter 13.1, 13.3, 13.6.1 - Reading : Peter Knee, Chapter 2 Paolo Gabriel (TA) : Neural Brain Control After class - Project groups

571 views • 28 slides

Multivariate Ordination Analyses: Principal Component Analysis Dilys Vela Tatiana Boza Tatiana

Multivariate Ordination Analyses: Principal Component Analysis Dilys Vela Tatiana Boza Tatiana Boza Multivariate Analyses Multivariate Analyses A multivariate data set includes more than one variable recorded from a number of i bl d d f b f

1.16k views • 41 slides

Principal Components Analysis David Benjamin, Broad DSDE Methods February 10, 2016 What is PCA?

Principal Components Analysis David Benjamin, Broad DSDE Methods February 10, 2016 What is PCA? PCA turns high-dimensional data into low-dimensional data by throwing out directions with low variance. Keep y , throw out x . Assumption: noise

352 views • 23 slides

Lecture 14: High Dimensionality & PCA CS109A Introduction to Data Science Pavlos Protopapas,

Lecture 14: High Dimensionality & PCA CS109A Introduction to Data Science Pavlos Protopapas, Kevin Rader and Chris Tanner Lecture Outline Interaction Terms and Unique Parameterizations Big Data and High Dimensionality Principal

694 views • 37 slides

GP IT Overview NHS Scarborough & Ryedale PLT February 2020 Official-Sensitive: Commercial

GP IT Overview NHS Scarborough & Ryedale PLT February 2020 Official-Sensitive: Commercial Welcome & Introductions Michelle Cunningham Account & Relationship Manager Ian Harrison Service Delivery Manager 2

604 views • 11 slides

Big Data Analytics in Economics: What Have We Learned so Far, and Where Should We Go From Here?

Big Data Analytics in Economics: What Have We Learned so Far, and Where Should We Go From Here? Norman R. Swanson and Weiqi Xiong* June 2017 *Rutgers University prep. for Workshop on Forecasting at Deutsche Bundesbank, Sept., 2017

689 views • 38 slides

Multiple Object Tracking Using Local PCA C. Beleznai 1 , B. Frhstck 2 , H. Bischof 3 1 Advanced

Multiple Object Tracking Using Local PCA C. Beleznai 1 , B. Frhstck 2 , H. Bischof 3 1 Advanced Computer Vision GmbH ACV, Vienna, Austria 2 Siemens AG Austria, Program and System Engineering, Graz, Austria 3 Inst. for Computer Graphics and

1.72k views • 16 slides

Access Link at UHCprovider.com Sign in to Link by clicking on the Link button in the top right

Access Link at UHCprovider.com Sign in to Link by clicking on the Link button in the top right corner. 1 Doc #: PCA-1-006654-05302017_07112017 Proprietary Information of UnitedHealth Group. Do not distribute or reproduce without express

408 views • 9 slides

Connecting the dots 2 Planning the Retreat Schedule a mandatory retreat date Find a location

Connecting the dots 2 Planning the Retreat Schedule a mandatory retreat date Find a location Set the agenda Staff preparation Provide agenda Provide guidelines Materials/items to take 3 Agenda 8:00 Welcome & Expectations Overview

582 views • 15 slides