tensor factorization via matrix factorization
play

Tensor Factorization via Matrix Factorization Volodymyr Kuleshov - PowerPoint PPT Presentation

Jun 12, 2023 •303 likes •1.1k views

Tensor Factorization via Matrix Factorization Volodymyr Kuleshov Arun Tejasvi Chaganty Percy Liang Stanford University May 11, 2015 Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 1 / 28 Introduction:

  1. Tensor Factorization via Matrix Factorization Volodymyr Kuleshov ú Arun Tejasvi Chaganty ú Percy Liang Stanford University May 11, 2015 Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 1 / 28

  2. Introduction: tensor factorization An application: community detection a b c d Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 2 / 28

  3. Introduction: tensor factorization An application: community detection a b c d Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 2 / 28

  4. Introduction: tensor factorization An application: community detection a b c d Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 2 / 28

  5. Introduction: tensor factorization An application: community detection ? ? a b ? ? c d Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 2 / 28

  6. Introduction: tensor factorization An application: community detection Anandkumar, Ge, Hsu, and S. Kakade 2013 a b c d = + + · · · + Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 2 / 28

  7. Introduction: tensor factorization Applications of tensor factorization I Community detection I Anandkumar, Ge, Hsu, and S. Kakade 2013 I Parsing I Cohen, Satta, and Collins 2013 I Knowledge base completion I Chang et al. 2014 I Singh, Rockt¨ aschel, and Riedel 2015 I Topic modelling I Anandkumar, Foster, et al. 2012 I Crowdsourcing I Zhang et al. 2014 I Mixture models I Anandkumar, Ge, Hsu, S. M. Kakade, et al. 2013 I Bottlenecked models I Chaganty and Liang 2014 I . . . Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 3 / 28

  8. Introduction: tensor factorization What is tensor (CP) factorization? I Tensor analogue of matrix eigen-decomposition. k ÿ M = fi i u i ¢ u i . i =1 = + + · · · + k Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 4 / 28

  9. Introduction: tensor factorization What is tensor (CP) factorization? I Tensor analogue of matrix eigen-decomposition. k ÿ T = fi i u i ¢ u i ¢ u i . i =1 = + + · · · + k Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 4 / 28

  10. Introduction: tensor factorization What is tensor (CP) factorization? I Tensor analogue of matrix eigen-decomposition. k ÿ ‚ T = fi i u i ¢ u i ¢ u i + ‘ R . i =1 I Goal: Given T with noise, ‘ R , recover factors u i . = + + · · · + + k Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 4 / 28

  11. Introduction: tensor factorization What is tensor (CP) factorization? I Tensor analogue of matrix eigen-decomposition. k ÿ ‚ T = fi i u i ¢ u i ¢ u i + ‘ R . i =1 I Goal: Given T with noise, ‘ R , recover factors u i . l a = n + + · · · + + o g o h t r O l a n o = + + · · · + + g o h t r o - n o N Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 4 / 28

  12. Introduction: tensor factorization Existing tensor factorization algorithms I Tensor power method (Anandkumar, Ge, Hsu, S. M. Kakade, et al. 2013) I Analog of matrix power method. I Sensitive to noise. I Restricted to orthogonal tensors. Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 5 / 28

  13. Introduction: tensor factorization Existing tensor factorization algorithms I Tensor power method (Anandkumar, Ge, Hsu, S. M. Kakade, et al. 2013) I Analog of matrix power method. I Sensitive to noise. I Restricted to orthogonal tensors. I Alternating least squares (Comon, Luciani, and Almeida 2009; Anandkumar, Ge, and Janzamin 2014) I Sensitive to initialization. Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 5 / 28

  14. Introduction: tensor factorization Existing tensor factorization algorithms I Tensor power method (Anandkumar, Ge, Hsu, S. M. Kakade, et al. 2013) I Analog of matrix power method. I Sensitive to noise. I Restricted to orthogonal tensors. I Alternating least squares (Comon, Luciani, and Almeida 2009; Anandkumar, Ge, and Janzamin 2014) I Sensitive to initialization. Our approach: reduce to existing fast and robust matrix algorithms . Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 5 / 28

  15. Orthogonal Tensor factorization Outline Introduction: tensor factorization Orthogonal Tensor factorization Projections Non-orthogonal tensor factorization Related work Empirical results Conclusions Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 6 / 28

  16. Orthogonal Tensor factorization Tensor factorization via single matrix factorization fi 1 u ¢ 3 fi 2 u ¢ 3 fi 3 u ¢ 3 T = + + + ‘ R 1 2 3 Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 7 / 28

  17. Orthogonal Tensor factorization Tensor factorization via single matrix factorization u ¢ 3 u ¢ 3 u ¢ 3 T = + + 1 1 1 Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 8 / 28

  18. Orthogonal Tensor factorization Tensor factorization via single matrix factorization u ¢ 3 u ¢ 3 u ¢ 3 T = + + 1 1 1 ¿ ( w € u 1 ) u ¢ 2 ( w € u 2 ) u ¢ 2 ( w € u 3 ) u ¢ 2 T ( I , I , w ) = + + 1 2 3 Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 8 / 28

  19. Orthogonal Tensor factorization Tensor factorization via single matrix factorization u ¢ 3 u ¢ 3 u ¢ 3 T = + + 1 1 1 ¿ ( w € u 1 ) u ¢ 2 ( w € u 2 ) u ¢ 2 ( w € u 3 ) u ¢ 2 T ( I , I , w ) = + + 1 2 3 ¸ ˚˙ ˝ ¸ ˚˙ ˝ ¸ ˚˙ ˝ ⁄ 1 ⁄ 2 ⁄ 3 I Proposal: Eigen-decomposition on the projected matrix. I Return: recovered eigenvectors, u i . Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 8 / 28

  20. Orthogonal Tensor factorization Sensitivity of single matrix projection I Problem : Eigendecomposition is very sensitive to the eigengap . 1 error in factors à min(di ff erence in eigenvalues) . Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 9 / 28

  21. Orthogonal Tensor factorization Sensitivity of single matrix projection I Problem : Eigendecomposition is very sensitive to the eigengap . 1 error in factors à min(di ff erence in eigenvalues) . I Intuition: If two eigenvalues are equal, corresponding eigenvectors are arbitrary. = + + Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 9 / 28

  22. Orthogonal Tensor factorization Sensitivity of single matrix projection I Problem : Eigendecomposition is very sensitive to the eigengap . 1 error in factors à min(di ff erence in eigenvalues) . I Intuition: If two eigenvalues are equal, corresponding eigenvectors are arbitrary. = + + Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 9 / 28

  23. Orthogonal Tensor factorization Sensitivity analysis (contd.) Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 10 / 28

  24. Orthogonal Tensor factorization Sensitivity analysis (contd.) I Single matrix factorization: error in factors à 1 min di ff . in eigenvalues . Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 10 / 28

  25. Orthogonal Tensor factorization Sensitivity analysis (contd.) I Single matrix factorization: error in factors à 1 min di ff . in eigenvalues . Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 10 / 28

  26. Orthogonal Tensor factorization Sensitivity analysis (contd.) I Single matrix factorization: error in factors à 1 min di ff . in eigenvalues . Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 10 / 28

  27. Orthogonal Tensor factorization Sensitivity analysis (contd.) I Single matrix factorization: error in factors à 1 min di ff . in eigenvalues . Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 10 / 28

  28. Orthogonal Tensor factorization Sensitivity analysis (contd.) I Single matrix factorization: error in factors à 1 min di ff . in eigenvalues . Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 10 / 28

  29. Orthogonal Tensor factorization Sensitivity analysis (contd.) I Single matrix factorization: error in factors à 1 min di ff . in eigenvalues . Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 10 / 28

  30. Orthogonal Tensor factorization Sensitivity analysis (contd.) (Cardoso 1994) I Single matrix factorization: error in factors à 1 min di ff . in eigenvalues . I Simultaneous matrix factorization: error in factors à 1 min avg. di ff . in eigenvalues . Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 10 / 28

  31. Orthogonal Tensor factorization Sensitivity analysis (contd.) (Cardoso 1994) I Single matrix factorization: Every coordinate pair needs one good projection (with a error in factors à large eigengap). 1 min di ff . in eigenvalues . I Simultaneous matrix factorization: error in factors à 1 min avg. di ff . in eigenvalues . Kuleshov, Chaganty, Liang (Stanford University) Tensor Factorization May 11, 2015 10 / 28

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

L101: Matrix Factorization In a nutshell Matrix factorization/completion you know? In NLP?

L101: Matrix Factorization In a nutshell Matrix factorization/completion you know? In NLP?

L101: Matrix Factorization In a nutshell Matrix factorization/completion you know? In NLP? Word embeddings Topic models Information extraction FastText Why complete the matrix? Label Features Label Label Label Label

420 views • 20 slides
Online-Updating Regularized Kernel Matrix Factorization Models for Large-Scale Recommender

Online-Updating Regularized Kernel Matrix Factorization Models for Large-Scale Recommender

Introduction Matrix factorization Learning Matrix Factorization Models Online Updates Evaluation Online-Updating Regularized Kernel Matrix Factorization Models for Large-Scale Recommender Systems. Steffen Rendle, Lars Schmidt-Thieme

903 views • 21 slides
Matrix Factorization and Factorization Machines for Recommender Systems Chih-Jen Lin Department

Matrix Factorization and Factorization Machines for Recommender Systems Chih-Jen Lin Department

Matrix Factorization and Factorization Machines for Recommender Systems Chih-Jen Lin Department of Computer Science National Taiwan University Talk at SDM workshop on Machine Learning Methods on Recommender Systems, May 2, 2015 Chih-Jen Lin

1.14k views • 54 slides
Sparse Tensor Factorization: Algorithms, Data Structures, and Challenges Shaden Smith &

Sparse Tensor Factorization: Algorithms, Data Structures, and Challenges Shaden Smith &

Sparse Tensor Factorization: Algorithms, Data Structures, and Challenges Shaden Smith & George Karypis University of Minnesota Department of Computer Science & Engineering shaden@cs.umn.edu Sparse Tensor Factorization: Algorithms,

844 views • 80 slides
TIME-DEPENDENT PARAMETRIC AND HARMONIC TEMPLATES IN NON-NEGATIVE MATRIX FACTORIZATION 13 th

TIME-DEPENDENT PARAMETRIC AND HARMONIC TEMPLATES IN NON-NEGATIVE MATRIX FACTORIZATION 13 th

Introduction Non negative-Matrix Factorization Spectrogram model Decomposition of musical spectrograms Conclusion TIME-DEPENDENT PARAMETRIC AND HARMONIC TEMPLATES IN NON-NEGATIVE MATRIX FACTORIZATION 13 th International Conference on Digital

750 views • 26 slides
Fast Coordinate Descent methods for Non-Negative Matrix Factorization Inderjit S. Dhillon

Fast Coordinate Descent methods for Non-Negative Matrix Factorization Inderjit S. Dhillon

Non-negative Matrix Factorization Fast Coordinate Descent methods for Non-Negative Matrix Factorization Inderjit S. Dhillon University of Texas at Austin SIAM Conference on Applied Linear Algebra Valencia, Spain June 19, 2012 Joint work with

682 views • 25 slides
Matrix Factorization with Binary Components Uniqueness in a randomized model Felix Krahmer,

Matrix Factorization with Binary Components Uniqueness in a randomized model Felix Krahmer,

Matrix Factorization with Binary Components Uniqueness in a randomized model Felix Krahmer, TU M unchen Joint work with: Matthias Hein, Saarland University, David James , University of G ottingen Matrix Factorization given data

1.49k views • 44 slides
Matrix Factorization For Topic Models Dr. Derek Greene Insight Latent Space Workshop

Matrix Factorization For Topic Models Dr. Derek Greene Insight Latent Space Workshop

Matrix Factorization For Topic Models Dr. Derek Greene Insight Latent Space Workshop Non-negative Matrix Factorization NMF : an unsupervised family of algorithms that simultaneously perform dimension reduction and clustering. Also

407 views • 16 slides
Factorization on GPUs Wei Tan IBM T. J. Watson Research Center wtan@us.ibm.com

Factorization on GPUs Wei Tan IBM T. J. Watson Research Center wtan@us.ibm.com

cuMF_sgd: Fast and Scalable Matrix Factorization on GPUs Wei Tan IBM T. J. Watson Research Center wtan@us.ibm.com http://github.com/cumf http://researcher.ibm.com/person/us-wtan GTC 2017 Agenda Why accelerate matrix factorization?

571 views • 21 slides
A Medium-Grained Algorithm for Distributed Sparse Tensor Factorization Shaden Smith George

A Medium-Grained Algorithm for Distributed Sparse Tensor Factorization Shaden Smith George

A Medium-Grained Algorithm for Distributed Sparse Tensor Factorization Shaden Smith George Karypis University of Minnesota Department of Computer Science & Engineering shaden@cs.umn.edu Medium-Grained Sparse Tensor Factorization 1 / 24

424 views • 25 slides
CuMF: Large-Scale Matrix Factorization on Just One Machine with GPUs Wei Tan, IBM T. J. Watson

CuMF: Large-Scale Matrix Factorization on Just One Machine with GPUs Wei Tan, IBM T. J. Watson

CuMF: Large-Scale Matrix Factorization on Just One Machine with GPUs Wei Tan, IBM T. J. Watson Research Center wtan@us.ibm.com Agenda Why accelerate recommendation (matrix factorization) using GPUs? What are the challenges? How cuMF

403 views • 22 slides
A Model For Mixed Linear-Tropical Matrix Factorization James Hook, Sanjar Karaev, Pauli Miettinen

A Model For Mixed Linear-Tropical Matrix Factorization James Hook, Sanjar Karaev, Pauli Miettinen

A Model For Mixed Linear-Tropical Matrix Factorization James Hook, Sanjar Karaev, Pauli Miettinen University of Birmingham: 18th June 2018 Low-Rank Approximate Factorization Given a matrix A R n m , an approximate factorization of rank k

207 views • 17 slides
Greedy Orthogonal Pivoting for Non-negative Matrix Factorization Kai Zhang, Jun Liu, Jie Zhang,

Greedy Orthogonal Pivoting for Non-negative Matrix Factorization Kai Zhang, Jun Liu, Jie Zhang,

Greedy Orthogonal Pivoting for Non-negative Matrix Factorization Kai Zhang, Jun Liu, Jie Zhang, Jun Wang Infinia ML Inc., Fudan University, East China Normal University Non-negative Matrix Factorization Represent data with non-negative basis

330 views • 7 slides
Matrix Factorization and Lifting Palle Jorgensen and Myung-Sin Song The University of Iowa,

Matrix Factorization and Lifting Palle Jorgensen and Myung-Sin Song The University of Iowa,

Matrix Factorization and Lifting Palle Jorgensen and Myung-Sin Song The University of Iowa, Southern Illinois University Edwardsville January 7, 2011 Introduction Develop and refine a procedure which uses factorization of families of

544 views • 35 slides
Matrix Factorization March 17, 2020 Data Science CSCI 1951A Brown University Instructor: Ellie

Matrix Factorization March 17, 2020 Data Science CSCI 1951A Brown University Instructor: Ellie

Matrix Factorization March 17, 2020 Data Science CSCI 1951A Brown University Instructor: Ellie Pavlick HTAs: Josh Levin, Diane Mutako, Sol Zitter 1 Announcements 2 Today Matrix Factorization with SVD Applications to: Topic

1.33k views • 117 slides
Infinite Positive Semidefinite Tensor Factorization for Source Separation of Mixture Signals

Infinite Positive Semidefinite Tensor Factorization for Source Separation of Mixture Signals

Infinite Positive Semidefinite Tensor Factorization by K. Yoshii, R. Tomioka, D. Mochihashi, and M. Goto Infinite Positive Semidefinite Tensor Factorization for Source Separation of Mixture Signals Kazuyoshi Yoshii Ryota Tomioka Daichi

461 views • 8 slides
CS475/CS675 Lecture 4: May 12, 2016 Sparse Gaussian Elimination, Graph Representation Reading:

CS475/CS675 Lecture 4: May 12, 2016 Sparse Gaussian Elimination, Graph Representation Reading:

CS475/CS675 Lecture 4: May 12, 2016 Sparse Gaussian Elimination, Graph Representation Reading: [Saad] Sect 3.13.2 CS475/CS675 (c) 2016 P. Poupart & J. Wan 1 5Point Stencil An easy way to denote 2D finite difference equations

450 views • 15 slides
Model Predictive Control Model Predictive Control of Hybrid Systems of Hybrid Systems Model

Model Predictive Control Model Predictive Control of Hybrid Systems of Hybrid Systems Model

Model Predictive Control Model Predictive Control of Hybrid Systems of Hybrid Systems Model Predictive Control Model Predictive Control of Hybrid Systems of Hybrid Systems Controller Hybrid System Alberto Bemporad Alberto Bemporad

615 views • 18 slides
Creating LaTeX and HTML documents from within Stata using texdoc and webdoc Example 1 Ben Jann

Creating LaTeX and HTML documents from within Stata using texdoc and webdoc Example 1 Ben Jann

Creating LaTeX and HTML documents from within Stata using texdoc and webdoc Example 1 Ben Jann University of Bern, ben.jann@soz.unibe.ch Nordic and Baltic Stata Users Group meeting Oslo, September 13, 2016 Contents 1 The texdoc source file

394 views • 11 slides
Mechanism Feasibility Design Tutorial Session Notes Dr. James Gopsill Design & Manufacture 2

Mechanism Feasibility Design Tutorial Session Notes Dr. James Gopsill Design & Manufacture 2

2017 Mechanism Feasibility Design Tutorial Session Notes Dr. James Gopsill Design & Manufacture 2 Mechanism Feasibility Design 1 Session 4 2017 Last Week 1. Shaft Design Feedback 2. Deployment Modelling Demo: Stopping the

583 views • 15 slides
Solving Linear Systems Sanzheng Qiao Department of Computing and Software McMaster University

Solving Linear Systems Sanzheng Qiao Department of Computing and Software McMaster University

Intro GE Errors Software Summary Solving Linear Systems Sanzheng Qiao Department of Computing and Software McMaster University July, 2012 Intro GE Errors Software Summary Outline Introduction 1 Gaussian Elimination Methods 2

854 views • 72 slides
CS 240A: Solving Ax = b in parallel Dense A: Gaussian elimination with partial pivoting (LU)

CS 240A: Solving Ax = b in parallel Dense A: Gaussian elimination with partial pivoting (LU)

CS 240A: Solving Ax = b in parallel Dense A: Gaussian elimination with partial pivoting (LU) Same flavor as matrix * matrix, but more complicated Sparse A: Gaussian elimination Cholesky, LU, etc. Graph algorithms Sparse A:

589 views • 25 slides
Lecture 14: Dense Linear Algebra David Bindel 18 Oct 2010 Where we are This week: dense

Lecture 14: Dense Linear Algebra David Bindel 18 Oct 2010 Where we are This week: dense

Lecture 14: Dense Linear Algebra David Bindel 18 Oct 2010 Where we are This week: dense linear algebra Next week: sparse linear algebra Numerical linear algebra in a nutshell Basic problems Linear systems: Ax = b Least

539 views • 36 slides
Notes Linear Algebra Assignment 1 is out (questions?) Last class: we reduced the problem

Notes Linear Algebra Assignment 1 is out (questions?) Last class: we reduced the problem

Notes Linear Algebra Assignment 1 is out (questions?) Last class: we reduced the problem of optimally interpolating scattered data to solving a system of linear equations Typical: often almost all of the computational work in

215 views • 3 slides

More recommend