http cs246 stanford edu high dimensional many features
play

http://cs246.stanford.edu High dimensional == many features Find - PowerPoint PPT Presentation

Apr 11, 2024 •36 likes •606 views

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu High dimensional == many features Find concepts/topics/genres: Documents: Features: Thousands of words, millions of word pairs

  1. CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu

  2.  High ‐ dimensional == many features  Find concepts/topics/genres:  Documents:  Features: Thousands of words, millions of word pairs  Surveys – Netflix: 480k users x 177k movies 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 2

  3.  Compress / reduce dimensionality:  10 6 rows; 10 3 columns; no updates  random access to any cell(s); small error: OK 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 3

  4.  Assumption: Data lies on or near a low d ‐ dimensional subspace  Axes of this subspace are effective representation of the data 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 4

  5. Why reduce dimensions?  Discover hidden correlations/topics  Words that occur commonly together  Remove redundant and noisy features  Not all words are useful  Interpretation and visualization  Easier storage and processing of the data 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 5

  6. A [m x n] = U [m x r]   r x r] ( V [n x r] ) T  A : Input data matrix  m x n matrix (e.g., m documents, n terms)  U : Left singular vectors  m x r matrix ( m documents, r concepts)   : Singular values  r x r diagonal matrix (strength of each ‘concept’) ( r : rank of the matrix A )  V : Right singular vectors  n x r matrix ( n terms, r concepts) 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 6

  7. n n   V T m m A U 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 7

  8. n  1 u 1 v 1  2 u 2 v 2  + m A σ i … scalar u i … vector v i … vector 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 8

  9. It is always possible to decompose a real matrix A into A = U  V T , where  U,  , V : unique  U, V : column orthonormal:  U T U = I ; V T V = I ( I : identity matrix)  (Cols. are orthogonal unit vectors)   : diagonal  Entries ( singular values ) are positive, and sorted in decreasing order ( σ 1  σ 2  ...  0) 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 9

  10.  A = U  V T ‐ example: Casablanca Serenity Amelie Matrix Alien 0.18 0 1 1 1 0 0 0.36 0 2 2 2 0 0 SciFi 9.64 0 0.18 0 1 1 1 0 0 x x = 0 5.29 0.90 0 5 5 5 0 0 0 0 0 2 2 0 0.53 0 0 0 3 3 0 0.80 0.58 0.58 0.58 0 0 Romnce 0 0 0 1 1 0 0.27 0 0 0 0.71 0.71 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 10

  11.  A = U  V T ‐ example: Casablanca SciFi ‐ concept Serenity Amelie Matrix Romance ‐ concept Alien 0.18 0 1 1 1 0 0 0.36 0 2 2 2 0 0 SciFi 9.64 0 0.18 0 1 1 1 0 0 x x = 0 5.29 5 5 5 0 0 0.90 0 0 0 0 2 2 0 0.53 0 0 0 3 3 0 0.80 0.58 0.58 0.58 0 0 Romnce 0 0 0 1 1 0 0.27 0 0 0 0.71 0.71 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 11

  12.  A = U  V T ‐ example: U is “user ‐ to ‐ concept” similarity matrix Casablanca SciFi ‐ concept Serenity Amelie Matrix Romance ‐ concept Alien 0.18 0 1 1 1 0 0 0.36 0 2 2 2 0 0 SciFi 9.64 0 0.18 0 1 1 1 0 0 x x = 0 5.29 5 5 5 0 0 0.90 0 0 0 0 2 2 0 0.53 0 0 0 3 3 0 0.80 0.58 0.58 0.58 0 0 Romnce 0 0 0 1 1 0 0.27 0 0 0 0.71 0.71 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 12

  13.  A = U  V T ‐ example: Casablanca Serenity Amelie Matrix Alien ‘strength’ of SciFi ‐ concept 0.18 0 1 1 1 0 0 0.36 0 2 2 2 0 0 SciFi 9.64 0 0.18 0 1 1 1 0 0 x x = 0 5.29 5 5 5 0 0 0.90 0 0 0 0 2 2 0 0.53 0 0 0 3 3 0 0.80 0.58 0.58 0.58 0 0 Romnce 0 0 0 1 1 0 0.27 0 0 0 0.71 0.71 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 13

  14.  A = U  V T ‐ example: V is “movie ‐ to ‐ concept” similarity matrix Casablanca Serenity Amelie Matrix Alien 0.18 0 1 1 1 0 0 SciFi ‐ concept 0.36 0 2 2 2 0 0 SciFi 9.64 0 0.18 0 1 1 1 0 0 x x = 0 5.29 5 5 5 0 0 0.90 0 0 0 0 2 2 0 0.53 0 0 0 3 3 0 0.80 0.58 0.58 0.58 0 0 Romnce 0 0 0 1 1 0 0.27 0 0 0 0.71 0.71 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 14

  15.  A = U  V T ‐ example: V is “movie ‐ to ‐ concept” similarity matrix Casablanca Serenity Amelie Matrix Alien 0.18 0 1 1 1 0 0 SciFi ‐ concept 0.36 0 2 2 2 0 0 SciFi 9.64 0 0.18 0 1 1 1 0 0 x x = 0 5.29 5 5 5 0 0 0.90 0 0 0 0 2 2 0 0.53 0 0 0 3 3 0 0.80 0.58 0.58 0.58 0 0 Romnce 0 0 0 1 1 0 0.27 0 0 0 0.71 0.71 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 15

  16. ‘ movies ’, ‘ users ’ and ‘ concepts ’:  U : user ‐ to ‐ concept similarity matrix  V : movie ‐ to ‐ concept sim. matrix   : its diagonal elements: ‘strength’ of each concept 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 16

  17. SVD gives best axis Movie 2 rating to project on:  ‘best’ = min sum first singular of squares of vector projection errors  minimum reconstruction v 1 error Movie 1 rating 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 17

  18.  A = U  V T ‐ example: 0.18 0 1 1 1 0 0 0.36 0 2 2 2 0 0 9.64 0 0.18 0 1 1 1 0 0 = x x 0 5.29 5 5 5 0 0 0.90 0 0 0 0 2 2 0 0.53 v 1 0 0 0 3 3 0 0.80 0.58 0.58 0.58 0 0 0 0 0 1 1 0 0.27 0 0 0 0.71 0.71 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 18

  19.  A = U  V T ‐ example: variance (‘spread’) on the v 1 axis 0.18 0 1 1 1 0 0 0.36 0 2 2 2 0 0 9.64 0 0.18 0 1 1 1 0 0 = x x 0 5.29 5 5 5 0 0 0.90 0 0 0 0 2 2 0 0.53 0 0 0 3 3 0 0.80 0.58 0.58 0.58 0 0 0 0 0 1 1 0 0.27 0 0 0 0.71 0.71 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 19

  20.  A = U  V T ‐ example:  U  Gives the coordinates of the points in the projection axis 0.18 0 1 1 1 0 0 0.36 0 2 2 2 0 0 9.64 0 0.18 0 1 1 1 0 0 = x x 0 5.29 5 5 5 0 0 0.90 0 0 0 0 2 2 0 0.53 0 0 0 3 3 0 0.80 0.58 0.58 0.58 0 0 0 0 0 1 1 0 0.27 0 0 0 0.71 0.71 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 20

  21. More details  Q: How exactly is dim. reduction done? 0.18 0 1 1 1 0 0 0.36 0 2 2 2 0 0 9.64 0 0.18 0 1 1 1 0 0 = x x 0 5.29 5 5 5 0 0 0.90 0 0 0 0 2 2 0 0.53 0 0 0 3 3 0 0.80 0.58 0.58 0.58 0 0 0 0 0 1 1 0 0.27 0 0 0 0.71 0.71 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 21

  22. More details  Q: How exactly is dim. reduction done?  A: Set the smallest singular values to zero 0.18 0 1 1 1 0 0 0.36 0 2 2 2 0 0 9.64 0 0.18 0 1 1 1 0 0 = x x 0 5.29 5 5 5 0 0 0.90 0 0 0 0 2 2 0 0.53 A= 0 0 0 3 3 0 0.80 0.58 0.58 0.58 0 0 0 0 0 1 1 0 0.27 0 0 0 0.71 0.71 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 22

  23. More details  Q: How exactly is dim. reduction done?  A: Set the smallest singular values to zero 0.18 0 1 1 1 0 0 0.36 0 2 2 2 0 0 9.64 0 0.18 0 1 1 1 0 0 x x ~ 0 0 5 5 5 0 0 0.90 0 0 0 0 2 2 0 0.53 A= 0 0 0 3 3 0 0.80 0.58 0.58 0.58 0 0 0 0 0 1 1 0 0.27 0 0 0 0.71 0.71 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 23

  24. More details  Q: How exactly is dim. reduction done?  A: Set the smallest singular values to zero: 0.18 0 1 1 1 0 0 0.36 0 2 2 2 0 0 9.64 0 0.18 0 1 1 1 0 0 x x ~ 0 0 5 5 5 0 0 0.90 0 0 0 0 2 2 0 0.53 A= 0 0 0 3 3 0 0.80 0.58 0.58 0.58 0 0 0 0 0 1 1 0 0.27 0 0 0 0.71 0.71 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 24

  25. More details  Q: How exactly is dim. reduction done?  A: Set the smallest singular values to zero: 0.18 1 1 1 0 0 0.36 2 2 2 0 0 9.64 0.18 1 1 1 0 0 x x ~ 5 5 5 0 0 0.90 0 0 0 2 2 0 A= 0 0 0 3 3 0 0.58 0.58 0.58 0 0 0 0 0 1 1 0 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 25

  26. More details  Q: How exactly is dim. reduction done?  A: Set the smallest singular values to zero B= 1 1 1 0 0 1 1 1 0 0 Frobenius norm: 2 2 2 0 0 2 2 2 0 0 ǁ M ǁ F = Σ ij M ij 2 1 1 1 0 0 1 1 1 0 0 ~ 5 5 5 0 0 A= 5 5 5 0 0 0 0 0 0 0 0 0 0 2 2 ǁ A-B ǁ F = Σ ij (A ij -B ij ) 2 0 0 0 0 0 0 0 0 3 3 0 0 0 0 0 is “small” 0 0 0 1 1 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 26

  27.  Theorem: Let A = U  V T ( σ 1  σ 2  …, rank( A )= r ) then B = U S V T  S = diagonal n x n matrix where s i = σ i ( i=1…k ) else s i =0 is a best rank ‐ k approximation to A :  B is solution to min B ǁ A-B ǁ F where rank( B )= k Σ � �� � ��  We will need 2 facts: �� �  where M = P Q R is SVD of M � �  U  V T ‐ U S V T = U (  ‐ S ) V T 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 27

  28.  We will need 2 facts: �� �  where M = P Q R is SVD of M � � We apply: -- P column orthonormal -- R row orthonormal -- Q is diagonal  U  V T ‐ U S V T = U (  ‐ S ) V T 1/25/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 28

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

http://cs246.stanford.edu High-dimension == many features Find concepts/topics/genres:

http://cs246.stanford.edu High-dimension == many features Find concepts/topics/genres:

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu High-dimension == many features Find concepts/topics/genres: Documents: Features: thousands of words, millions of word pairs Surveys

928 views • 54 slides
http://cs246.stanford.edu Input features: N features: X 1 , X 2 , X N A Each X j

http://cs246.stanford.edu Input features: N features: X 1 , X 2 , X N A Each X j

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu Input features: N features: X 1 , X 2 , X N A Each X j has domain D j X 1 <v 1 Categorical: C Y= 0.42 D j = {red, blue} X 2

508 views • 40 slides
http://cs246.stanford.edu High dim. High dim. Graph Graph Infinite Infinite Machine Machine Apps

http://cs246.stanford.edu High dim. High dim. Graph Graph Infinite Infinite Machine Machine Apps

3 announcements: Thanks for filling out the HW1 poll HW2 is due today 5pm (scans must be readable) HW3 will be posted today CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu High dim. High dim.

1.07k views • 61 slides
http://cs246.stanford.edu High dim. Graph Infinite Machine Apps data data data learning

http://cs246.stanford.edu High dim. Graph Infinite Machine Apps data data data learning

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu High dim. Graph Infinite Machine Apps data data data learning Locality Filtering PageRank, Recommen sensitive data SVM SimRank der systems

784 views • 61 slides
http://cs246.stanford.edu Web advertising We discussed how to match

http://cs246.stanford.edu Web advertising We discussed how to match

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu Web advertising We discussed how to match advertisers to queries in real-time

686 views • 45 slides
http://cs246.stanford.edu Web pages are not equally important www.joe-schmoe.com vs.

http://cs246.stanford.edu Web pages are not equally important www.joe-schmoe.com vs.

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu Web pages are not equally important www.joe-schmoe.com vs. www.stanford.edu We already know: Since there is large diversity in the

668 views • 44 slides
http://cs246.stanford.edu Web advertising We discussed how to match advertisers to

http://cs246.stanford.edu Web advertising We discussed how to match advertisers to

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu Web advertising We discussed how to match advertisers to queries in real-time But we did not discuss how to estimate CTR Recommendation

674 views • 45 slides
http://cs246.stanford.edu Web advertising We discussed how to match advertisers to queries

http://cs246.stanford.edu Web advertising We discussed how to match advertisers to queries

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu Web advertising We discussed how to match advertisers to queries in real-time But we did not discuss how to estimate the CTR (Click-Through

882 views • 44 slides
http://cs246.stanford.edu Classic model of algorithms You get to see the entire input, then

http://cs246.stanford.edu Classic model of algorithms You get to see the entire input, then

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu Classic model of algorithms You get to see the entire input, then compute some function of it In this context, offline algorithm

611 views • 39 slides
http://cs246.stanford.edu Web advertising Weve learned how to match advertisers to

http://cs246.stanford.edu Web advertising Weve learned how to match advertisers to

CS246: Mining Massive Datasets Caroline Lo, Stanford University http://cs246.stanford.edu Web advertising Weve learned how to match advertisers to queries in real-time But how to estimate the CTR (Click-Through Rate)?

815 views • 32 slides
http://cs246.stanford.edu HITS (Hypertext Induced Topic Selection) Is a measure of

http://cs246.stanford.edu HITS (Hypertext Induced Topic Selection) Is a measure of

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu HITS (Hypertext Induced Topic Selection) Is a measure of importance of pages or documents, similar to PageRank Proposed at around same time

855 views • 52 slides
http://cs246.stanford.edu Goal: Given a large number (N in the millions or billions) of text

http://cs246.stanford.edu Goal: Given a large number (N in the millions or billions) of text

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu Goal: Given a large number (N in the millions or billions) of text documents, find pairs that are near duplicates Application: Detect

697 views • 68 slides
http://cs246.stanford.edu Overlaps with machine learning, statistics, artificial intelligence,

http://cs246.stanford.edu Overlaps with machine learning, statistics, artificial intelligence,

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu Overlaps with machine learning, statistics, artificial intelligence, databases, visualization but more stress on scalability of number

613 views • 46 slides
http://cs246.stanford.edu More algorithms for streams: (1) Filtering a data stream: Bloom

http://cs246.stanford.edu More algorithms for streams: (1) Filtering a data stream: Bloom

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu More algorithms for streams: (1) Filtering a data stream: Bloom filters Select elements with property x from stream (2) Counting distinct

842 views • 40 slides
http://cs246.stanford.edu TAs : Bahman Bahmani Juthika Dabholkar Pierre Kreitmann

http://cs246.stanford.edu TAs : Bahman Bahmani Juthika Dabholkar Pierre Kreitmann

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu TAs : Bahman Bahmani Juthika Dabholkar Pierre Kreitmann Lu Li Aditya Ramesh Office hours: Jure: Tuesdays 9-10am, Gates 418

890 views • 66 slides
http://cs246.stanford.edu Instructor: Jure Leskovec TAs: Aditya Parameswaran

http://cs246.stanford.edu Instructor: Jure Leskovec TAs: Aditya Parameswaran

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu Instructor: Jure Leskovec TAs: Aditya Parameswaran Bahman Bahmani Peyman Kazemian 1/3/2011 Jure Leskovec, Stanford C246: Mining

522 views • 35 slides
SIM CQM & ASSESSMENT UPDATES WEBINAR TUESDAY, FEBRUARY 28 11:00 12:00 AGENDA Welcome

SIM CQM & ASSESSMENT UPDATES WEBINAR TUESDAY, FEBRUARY 28 11:00 12:00 AGENDA Welcome

SIM CQM & ASSESSMENT UPDATES WEBINAR TUESDAY, FEBRUARY 28 11:00 12:00 AGENDA Welcome & Intros Conflict of Interest Forms SIM CQM Measure Simplification 2 Revised CQM Reporting Schedules Discussion: How are CHITAs working with

2.38k views • 14 slides
Ambedkar University Delhi Work participation rates of rural women in 15 major states of India (per

Ambedkar University Delhi Work participation rates of rural women in 15 major states of India (per

Why does the Work Participation Rate of Women Low in the Eastern-India? A case study of Bengal with reference to Tamil Nadu Deepita Chakravarty Ambedkar University Delhi Work participation rates of rural women in 15 major states of India (per

204 views • 18 slides
Non-linear interlinkages and key objectives amongst the Paris Agreement and the Sustainable

Non-linear interlinkages and key objectives amongst the Paris Agreement and the Sustainable

Non-linear interlinkages and key objectives amongst the Paris Agreement and the Sustainable Development Goals Felix Laumann*, Julius von Kgelgen, Mauricio Barahona * f.laumann18@imperial.ac.uk Climate Change AI workshop, ICLR 2020 Paris

517 views • 5 slides
Selecting and Using Views To Compute Aggregate Queries Foto Afrati (NTUA Greece) and Rada

Selecting and Using Views To Compute Aggregate Queries Foto Afrati (NTUA Greece) and Rada

Selecting and Using Views To Compute Aggregate Queries Foto Afrati (NTUA Greece) and Rada Chirkova (NC State University) F. Afrati and R. Chirkova Views to Compute Aggregate Queries January 7, 2005 What this Talk Is About

1.15k views • 51 slides
Alien terms and definitions Erik Nordmark erik.nordmark@sun.com Wassim Haddad From

Alien terms and definitions Erik Nordmark erik.nordmark@sun.com Wassim Haddad From

Alien terms and definitions Erik Nordmark erik.nordmark@sun.com Wassim Haddad From <draft-haddad-momipriv-problem-statement- 01.txt> Anonymity An entity "A" in a system has anonymity if no other entity can identify

338 views • 5 slides
Tradeoffs between Anonymity and Identifiability Bob Hinden IETF 63 Paris 3 August 2005 The

Tradeoffs between Anonymity and Identifiability Bob Hinden IETF 63 Paris 3 August 2005 The

Tradeoffs between Anonymity and Identifiability Bob Hinden IETF 63 Paris 3 August 2005 The Problem Its good to protect disclosure of the user identify and location Its bad to protect identity and location of hackers, BOTs,

245 views • 6 slides
Interfacing AliEn and ARC Interfacing AliEn and ARC for a distributed Nordic T1 for a

Interfacing AliEn and ARC Interfacing AliEn and ARC for a distributed Nordic T1 for a

Interfacing AliEn and ARC Interfacing AliEn and ARC for a distributed Nordic T1 for a distributed Nordic T1 Why such an interface Why such an interface ARC and AliEn ARC and AliEn The Interface The Interface Tests Tests

1.03k views • 36 slides
Unit 2: Probability and distributions 3. Normal and binomial distributions Sta 101 - Spring 2019

Unit 2: Probability and distributions 3. Normal and binomial distributions Sta 101 - Spring 2019

Announcements Unit 2: Probability and distributions 3. Normal and binomial distributions Sta 101 - Spring 2019 RA 3 on Tuesday PS 2 due Friday, PA 2 due Sunday Duke University, Department of Statistical Science Dr. Abrahamsen Slides

250 views • 5 slides

More recommend