matrix factorization
play

Matrix Factorization March 17, 2020 Data Science CSCI 1951A Brown - PowerPoint PPT Presentation

Sep 05, 2022 •154 likes •1.33k views

Matrix Factorization March 17, 2020 Data Science CSCI 1951A Brown University Instructor: Ellie Pavlick HTAs: Josh Levin, Diane Mutako, Sol Zitter 1 Announcements 2 Today Matrix Factorization with SVD Applications to: Topic

  1. cong parli the US UK ress ame Singular Value 1 1 1 1 0 doc1 Decomposition (SVD) 1 0 1 0 1 doc2 1 1 0 1 0 doc3 doc1 in new feature space 1 0 1 0 1 doc4 cong parlia the US UK ress ment -0.65 -0.34 -0.51 -0.34 -0.31 d1 -0.60 -0.39 0.70 0.00 3.06 0.00 0.00 0.00 0.00 0.02 -0.54 0.34 -0.54 0.56 d2 -0.48 0.50 -0.12 -0.71 0.00 1.81 0.00 0.00 0.00 d3 -0.43 -0.58 -0.69 0.00 -0.42 0.02 0.79 0.02 -0.44 0.00 0.00 0.57 0.00 0.00 d4 -0.48 0.50 -0.12 0.71 -0.63 0.27 0.00 0.37 0.63 0.00 0.00 0.00 0.00 0.00 -0.04 0.73 0.00 -0.68 0.04 U D V

  2. cong parli the US UK ress ame Singular Value 1 1 1 1 0 doc1 Decomposition (SVD) 1 0 1 0 1 doc2 1 1 0 1 0 doc3 weight of component 1 for 1 0 1 0 1 doc4 doc 1 cong parlia the US UK ress ment -0.65 -0.34 -0.51 -0.34 -0.31 d1 -0.60 -0.39 0.70 0.00 3.06 0.00 0.00 0.00 0.00 0.02 -0.54 0.34 -0.54 0.56 d2 -0.48 0.50 -0.12 -0.71 0.00 1.81 0.00 0.00 0.00 d3 -0.43 -0.58 -0.69 0.00 -0.42 0.02 0.79 0.02 -0.44 0.00 0.00 0.57 0.00 0.00 d4 -0.48 0.50 -0.12 0.71 -0.63 0.27 0.00 0.37 0.63 0.00 0.00 0.00 0.00 0.00 -0.04 0.73 0.00 -0.68 0.04 U D V

  3. cong parli the US UK ress ame Singular Value 1 1 1 1 0 doc1 Decomposition (SVD) 1 0 1 0 1 doc2 1 1 0 1 0 doc3 weight of component 1 over 1 0 1 0 1 doc4 all the data cong parlia the US UK ress ment -0.65 -0.34 -0.51 -0.34 -0.31 d1 -0.60 -0.39 0.70 0.00 3.06 0.00 0.00 0.00 0.00 0.02 -0.54 0.34 -0.54 0.56 d2 -0.48 0.50 -0.12 -0.71 0.00 1.81 0.00 0.00 0.00 d3 -0.43 -0.58 -0.69 0.00 -0.42 0.02 0.79 0.02 -0.44 0.00 0.00 0.57 0.00 0.00 d4 -0.48 0.50 -0.12 0.71 -0.63 0.27 0.00 0.37 0.63 0.00 0.00 0.00 0.00 0.00 -0.04 0.73 0.00 -0.68 0.04 U D V

  4. cong parli the US UK ress ame Singular Value 1 1 1 1 0 doc1 Decomposition (SVD) 1 0 1 0 1 doc2 1 1 0 1 0 doc3 component 1 1 0 1 0 1 doc4 cong parlia the US UK ress ment -0.65 -0.34 -0.51 -0.34 -0.31 d1 -0.60 -0.39 0.70 0.00 3.06 0.00 0.00 0.00 0.00 0.02 -0.54 0.34 -0.54 0.56 d2 -0.48 0.50 -0.12 -0.71 0.00 1.81 0.00 0.00 0.00 d3 -0.43 -0.58 -0.69 0.00 -0.42 0.02 0.79 0.02 -0.44 0.00 0.00 0.57 0.00 0.00 d4 -0.48 0.50 -0.12 0.71 -0.63 0.27 0.00 0.37 0.63 0.00 0.00 0.00 0.00 0.00 -0.04 0.73 0.00 -0.68 0.04 U D V

  5. cong parli the US UK ress ame Singular Value 1 1 1 1 0 doc1 Decomposition (SVD) 1 0 1 0 1 doc2 1 1 0 1 0 doc3 contribution of “the” to 1 0 1 0 1 doc4 component 1 cong parlia the US UK ress ment -0.65 -0.34 -0.51 -0.34 -0.31 d1 -0.60 -0.39 0.70 0.00 3.06 0.00 0.00 0.00 0.00 0.02 -0.54 0.34 -0.54 0.56 d2 -0.48 0.50 -0.12 -0.71 0.00 1.81 0.00 0.00 0.00 d3 -0.43 -0.58 -0.69 0.00 -0.42 0.02 0.79 0.02 -0.44 0.00 0.00 0.57 0.00 0.00 d4 -0.48 0.50 -0.12 0.71 -0.63 0.27 0.00 0.37 0.63 0.00 0.00 0.00 0.00 0.00 -0.04 0.73 0.00 -0.68 0.04 U D V

  6. Singular Value Decomposition (SVD) x = x M U D V m x n m x m m x n n x n

  7. 67

  8. d e t a Singular Value c n u r T Decomposition (SVD) x = x M U D V m x n m x l l x l l x n

  9. d e t a Singular Value c n u r T Decomposition (SVD) x = x M U D V m x n m x l l x l l x n keep only first l components

  10. d e t a Singular Value c n u r T Decomposition (SVD) x = x M U D V m x n m x l l x l l x n keep only first l components “best l-rank approximation of M”

  11. d e t a Singular Value c n u r T Decomposition (SVD) x = x M U D V ||M - UDV|| 2 as small as possible m x n m x l l x l l x n keep only first l components “best l-rank approximation of M”

  12. 72

  13. Dimensionality Reduction • “Low Rank Assumption”: we typically assume that our features contain a large amount of redundant information • We can throw away a lot of principle components without losing too much of the signal needed for our task

  14. Clicker Question!

  15. Clicker Question! In practice, is this assumption of low rank valid? a) Yes b) No c) Yeah, sure, why not?

  16. Matrices IRL • Data is noisy, so M is most likely full-rank • We assume that M is close to a low rank matrix, and we approximate the matrix it is close to • Viewed as a “de-noised” version of M • “Original matrix exhibits redundancy and noise, low-rank reconstruction exploits the former to remove the latter”*

  17. Matrices IRL • Data is noisy, so M is most likely full-rank • We assume that M is close to a low rank matrix, and we approximate the matrix it is close to • Viewed as a “de-noised” version of M • “Original matrix exhibits redundancy and noise, low-rank reconstruction exploits the former to remove the latter”*

  18. Matrices IRL • Data is noisy, so M is most likely full-rank • We assume that M is close to a low rank matrix, and we approximate the matrix it is close to • Viewed as a “de-noised” version of M • “Original matrix exhibits redundancy and noise, low-rank reconstruction exploits the former to remove the latter”*

  19. Matrices IRL • Data is noisy, so M is most likely full-rank • We assume that M is close to a low rank matrix, and we approximate the matrix it is close to • Viewed as a “de-noised” version of M • “Original matrix exhibits redundancy and noise, low-rank reconstruction exploits the former to remove the latter”*

  20. Matrices IRL • Data is noisy, so M is most likely full-rank • We assume that M is close to a low rank matrix, and we approximate the matrix it is close to • Viewed as a “de-noised” version of M • “Original matrix exhibits redundancy and noise, low-rank reconstruction exploits the former to remove the latter”* *Matrix and Tensor Factorization Methods for Natural Language Processing. (ACL 2015)

  21. Matrices IRL • Data is also often incomplete…missing values, new observations, etc. • Can we use SVD for this? • Yes! Though we need to make a few changes…

  22. Matrices IRL • Data is also often incomplete…missing values, new observations, etc. • Can we use SVD for this? • Yes! Though we need to make a few changes…

  23. Matrices IRL • Data is also often incomplete…missing values, new observations, etc. • Can we use SVD for this? • Yes! Though we need to make a few changes…

  24. Matrices IRL • Data is also often incomplete…missing values, new observations, etc. • Can we use SVD for this? • Yes! Though we need to make a few changes…

  25. Matrix Completion to all the ballad of mud- boys i roma buster okja bound loved scruggs… before user1 1 0 1 user2 0 0 1 user3 1 0 1 0 1 0 user4 user5 1

  26. Matrix Completion to all the ballad of mud- boys i roma buster okja bound loved scruggs… before 1 0 user1 1 0 1 user2 0 0 1 1 user3 1 0 1 0 1 0 user4 user5 1 “people also liked…”

  27. Matrix Completion

  28. Matrix Completion M ≈ UDV = M’

  29. Matrix Completion M ≈ UDV = M’ original completed

  30. Matrix Completion M ≈ UDV = M’ original completed problems?

  31. Matrix Completion x x = M U D V Exact SVD assumes M is complete…

  32. Matrix Completion x x = M U D V …just gradient descent that MF!

  33. MF with Gradient Descent x x = M U D V

  34. MF with Gradient Descent = x M U V

  35. MF with Gradient Descent = x M U V Not properly SVD (fewer guarantees, e.g. components not orthonormal) but good enough

  36. <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> MF with Gradient Descent = x M U V X ( M ij − u i · v j ) 2 min U,V ij

  37. <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> MF with Gradient Descent = x M U V X ( M ij − u i · v j ) 2 min U,V ij But! Only consider cases when M ij is observed!

  38. Clicker Question!

  39. <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> Clicker Question! X ( M ij − u i · v j ) 2 min U,V ij Compute the loss given this setting of U and V… 2 1 1 2 3 2 2 0 0 M U V a) 14 b) 10 c) 6

  40. <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> Clicker Question! X ( M ij − u i · v j ) 2 min U,V ij Compute the loss given this setting of U and V… 1 2 2 1 x 1 2 = 2 4 3 2 0 0 2 0 0 M U V M’ a) 14 b) 10 c) 6

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

L101: Matrix Factorization In a nutshell Matrix factorization/completion you know? In NLP?

L101: Matrix Factorization In a nutshell Matrix factorization/completion you know? In NLP?

L101: Matrix Factorization In a nutshell Matrix factorization/completion you know? In NLP? Word embeddings Topic models Information extraction FastText Why complete the matrix? Label Features Label Label Label Label

420 views • 20 slides
Online-Updating Regularized Kernel Matrix Factorization Models for Large-Scale Recommender

Online-Updating Regularized Kernel Matrix Factorization Models for Large-Scale Recommender

Introduction Matrix factorization Learning Matrix Factorization Models Online Updates Evaluation Online-Updating Regularized Kernel Matrix Factorization Models for Large-Scale Recommender Systems. Steffen Rendle, Lars Schmidt-Thieme

903 views • 21 slides
TIME-DEPENDENT PARAMETRIC AND HARMONIC TEMPLATES IN NON-NEGATIVE MATRIX FACTORIZATION 13 th

TIME-DEPENDENT PARAMETRIC AND HARMONIC TEMPLATES IN NON-NEGATIVE MATRIX FACTORIZATION 13 th

Introduction Non negative-Matrix Factorization Spectrogram model Decomposition of musical spectrograms Conclusion TIME-DEPENDENT PARAMETRIC AND HARMONIC TEMPLATES IN NON-NEGATIVE MATRIX FACTORIZATION 13 th International Conference on Digital

750 views • 26 slides
Matrix Factorization and Factorization Machines for Recommender Systems Chih-Jen Lin Department

Matrix Factorization and Factorization Machines for Recommender Systems Chih-Jen Lin Department

Matrix Factorization and Factorization Machines for Recommender Systems Chih-Jen Lin Department of Computer Science National Taiwan University Talk at SDM workshop on Machine Learning Methods on Recommender Systems, May 2, 2015 Chih-Jen Lin

1.14k views • 54 slides
Fast Coordinate Descent methods for Non-Negative Matrix Factorization Inderjit S. Dhillon

Fast Coordinate Descent methods for Non-Negative Matrix Factorization Inderjit S. Dhillon

Non-negative Matrix Factorization Fast Coordinate Descent methods for Non-Negative Matrix Factorization Inderjit S. Dhillon University of Texas at Austin SIAM Conference on Applied Linear Algebra Valencia, Spain June 19, 2012 Joint work with

682 views • 25 slides
Chapter IX: Matrix factorizations* 1. The general idea 2. Matrix factorization methods 3. Latent

Chapter IX: Matrix factorizations* 1. The general idea 2. Matrix factorization methods 3. Latent

Chapter IX: Matrix factorizations* 1. The general idea 2. Matrix factorization methods 3. Latent topic models 4. Dimensionality reduction *Zaki &amp; Meira, Ch. 8; Tan, Steinbach &amp; Kumar, App. B; Manning, Raghavan &amp; Schtze, Ch. 18

798 views • 53 slides
Matrix Factorization with Binary Components Uniqueness in a randomized model Felix Krahmer,

Matrix Factorization with Binary Components Uniqueness in a randomized model Felix Krahmer,

Matrix Factorization with Binary Components Uniqueness in a randomized model Felix Krahmer, TU M unchen Joint work with: Matthias Hein, Saarland University, David James , University of G ottingen Matrix Factorization given data

1.49k views • 44 slides
Matrix Factorization For Topic Models Dr. Derek Greene Insight Latent Space Workshop

Matrix Factorization For Topic Models Dr. Derek Greene Insight Latent Space Workshop

Matrix Factorization For Topic Models Dr. Derek Greene Insight Latent Space Workshop Non-negative Matrix Factorization NMF : an unsupervised family of algorithms that simultaneously perform dimension reduction and clustering. Also

407 views • 16 slides
CuMF: Large-Scale Matrix Factorization on Just One Machine with GPUs Wei Tan, IBM T. J. Watson

CuMF: Large-Scale Matrix Factorization on Just One Machine with GPUs Wei Tan, IBM T. J. Watson

CuMF: Large-Scale Matrix Factorization on Just One Machine with GPUs Wei Tan, IBM T. J. Watson Research Center wtan@us.ibm.com Agenda Why accelerate recommendation (matrix factorization) using GPUs? What are the challenges? How cuMF

403 views • 22 slides
Greedy Orthogonal Pivoting for Non-negative Matrix Factorization Kai Zhang, Jun Liu, Jie Zhang,

Greedy Orthogonal Pivoting for Non-negative Matrix Factorization Kai Zhang, Jun Liu, Jie Zhang,

Greedy Orthogonal Pivoting for Non-negative Matrix Factorization Kai Zhang, Jun Liu, Jie Zhang, Jun Wang Infinia ML Inc., Fudan University, East China Normal University Non-negative Matrix Factorization Represent data with non-negative basis

330 views • 7 slides
Large-Scale Matrix Factorization with Distributed Stochastic Gradient Descent Rainer Gemulla

Large-Scale Matrix Factorization with Distributed Stochastic Gradient Descent Rainer Gemulla

Large-Scale Matrix Factorization with Distributed Stochastic Gradient Descent Rainer Gemulla December 17, 2011 Peter J. Haas Yannis Sismanis Christina Teflioudi Faraz Makari Outline Matrix Factorization Stochastic Gradient Descent

1.24k views • 75 slides
A Model For Mixed Linear-Tropical Matrix Factorization James Hook, Sanjar Karaev, Pauli Miettinen

A Model For Mixed Linear-Tropical Matrix Factorization James Hook, Sanjar Karaev, Pauli Miettinen

A Model For Mixed Linear-Tropical Matrix Factorization James Hook, Sanjar Karaev, Pauli Miettinen University of Birmingham: 18th June 2018 Low-Rank Approximate Factorization Given a matrix A R n m , an approximate factorization of rank k

207 views • 17 slides
Factorization on GPUs Wei Tan IBM T. J. Watson Research Center wtan@us.ibm.com

Factorization on GPUs Wei Tan IBM T. J. Watson Research Center wtan@us.ibm.com

cuMF_sgd: Fast and Scalable Matrix Factorization on GPUs Wei Tan IBM T. J. Watson Research Center wtan@us.ibm.com http://github.com/cumf http://researcher.ibm.com/person/us-wtan GTC 2017 Agenda Why accelerate matrix factorization?

571 views • 21 slides
Large-Scale Matrix Factorization with Distributed Stochastic Gradient Descent Rainer Gemulla

Large-Scale Matrix Factorization with Distributed Stochastic Gradient Descent Rainer Gemulla

Large-Scale Matrix Factorization with Distributed Stochastic Gradient Descent Rainer Gemulla August 23, 2011 Peter J. Haas Yannis Sismanis Erik Nijkamp Outline Matrix Factorization Stochastic Gradient Descent Distributed SGD with MapReduce

1.05k views • 81 slides
Structured sparse methods for matrix factorization Francis Bach Sierra team, INRIA - Ecole

Structured sparse methods for matrix factorization Francis Bach Sierra team, INRIA - Ecole

Structured sparse methods for matrix factorization Francis Bach Sierra team, INRIA - Ecole Normale Sup erieure March 2011 Joint work with R. Jenatton, J. Mairal, G. Obozinski Structured sparse methods for matrix factorization Outline

1.07k views • 73 slides
Nonnegative matrix factorization and applications in audio signal processing C edric F

Nonnegative matrix factorization and applications in audio signal processing C edric F

Nonnegative matrix factorization and applications in audio signal processing C edric F evotte Laboratoire Lagrange, Nice Machine Learning Crash Course Genova, June 2015 1 Outline Generalities Matrix factorisation models Nonnegative

581 views • 42 slides
Growing Global Leaders Advancing Palliative Care Finding Your Voice Ron Cameron-Lewis, BA,

Growing Global Leaders Advancing Palliative Care Finding Your Voice Ron Cameron-Lewis, BA,

Growing Global Leaders Advancing Palliative Care Finding Your Voice Ron Cameron-Lewis, BA, AGSM Professor Emeritus, Sheridan Institute, Canada LDI C2 RC2 October 21-28, 2012 Steps for this morning 1. Relaxation and breath 2. Turning

329 views • 12 slides
Introducing InfluxDB, an open source distributed time series database Paul Dix @pauldix

Introducing InfluxDB, an open source distributed time series database Paul Dix @pauldix

Introducing InfluxDB, an open source distributed time series database Paul Dix @pauldix paul@errplane.com About me Co-founder, CEO of Errplane (YC W13) Organizer of NYC Machine Learning Author of Service Oriented Design with

959 views • 67 slides
moodwatchers The Psychology of Flow SESSION TWO Some principles from last session The Role

moodwatchers The Psychology of Flow SESSION TWO Some principles from last session The Role

moodwatchers The Psychology of Flow SESSION TWO Some principles from last session The Role of Thinking in Mood Illogical and irrational thinking ( Is it helpful? Fair? True?) Its not events per se which determine our feelings but

367 views • 5 slides
Automatic writer identification: the paleographers new best friend? Jinna Smit University of

Automatic writer identification: the paleographers new best friend? Jinna Smit University of

Automatic writer identification: the paleographers new best friend? Jinna Smit University of Amsterdam/Nationaal Archief Thinking of meat makes people less social and lonely. Article in Dutch newspapers on an experiment of three

587 views • 27 slides
RAFAL trained as an architect in Warsaw. In 1981 he emigrated to the US, where he worked as a

RAFAL trained as an architect in Warsaw. In 1981 he emigrated to the US, where he worked as a

Rafal Olbinski is a surrealist artist, who has become world famous for his posters and designs . Rafal was born in Poland, and originally RAFAL trained as an architect in Warsaw. In 1981 he emigrated to the US, where he worked as a

548 views • 29 slides
LEADING IN TURBULENT TIMES How to Lead and Develop an Action Plan for Your Office Though no

LEADING IN TURBULENT TIMES How to Lead and Develop an Action Plan for Your Office Though no

LEADING IN TURBULENT TIMES How to Lead and Develop an Action Plan for Your Office Though no one can go back and make a brand new start, anyone can start from now and make a brand new ending. Carl Bard BEING POSITIVE IN A NEGATIVE

558 views • 24 slides
Militarization of Space by Bard Wormdal NSA-friends and enemies NSA secret agrement with Sweden

Militarization of Space by Bard Wormdal NSA-friends and enemies NSA secret agrement with Sweden

Militarization of Space by Bard Wormdal NSA-friends and enemies NSA secret agrement with Sweden Similar secret NSA-agrement with Norway Norways geographical curse US-Norwegian intelligence today Fauske - stealing information from Russian

342 views • 30 slides
UEC Report 48 th Annual Users Meeting William Louis, June 11, 2015 2014-2015 Users Executive

UEC Report 48 th Annual Users Meeting William Louis, June 11, 2015 2014-2015 Users Executive

UEC Report 48 th Annual Users Meeting William Louis, June 11, 2015 2014-2015 Users Executive Committee Sandra Biedron Tulika Bose Fernanda Garcia Andre de Gouvea Fabio Happacher Bill Lee Bill Louis Vivian O'Dell Jesus Orduna Lee Roberts

335 views • 14 slides

More recommend