clustering big data
play

Clustering Big Data Anil K. Jain (with Radha Chitta and Rong Jin) - PowerPoint PPT Presentation

May 31, 2023 •177 likes •625 views

Clustering Big Data Anil K. Jain (with Radha Chitta and Rong Jin) Department of Computer Science Michigan State University November 29, 2012 Outline Big Data How to extract information? Data clustering Clustering Big Data

  1. Clustering Big Data Anil K. Jain (with Radha Chitta and Rong Jin) Department of Computer Science Michigan State University November 29, 2012

  2. Outline • Big Data • How to extract “information”? • Data clustering • Clustering Big Data • Kernel K-means & approximation • Summary

  3. How Big is Big Data? • Big is a fast moving target: kilobytes, megabytes, gigabytes, terabytes (10 12 ), petabytes (10 15 ), exabytes (10 18 ), zettabytes (10 21 ),…… • Over 1.8 zb created in 2011; ~8 zb by 2015 D E a x t a a b y s t i e z s e Source: IDC’s Digital Universe study, sponsored by EMC, June 2011 http://idcdocserv.com/1142 As of June 2012 http://www.emc.com/leadership/programs/digital-universe.htm Nature of Big Data: Volume, Velocity and Variety

  4. Big Data on the Web Over 225 million users generating over 800 tweets per second ~900 million users, 2.5 billion content items, 105 terabytes of data each half hour, 300M photos and 4M videos posted per day http://techcrunch.com/2012/08/22/how-big-is-facebooks-data-2-5-billion-pieces-of-content-and-500-terabytes-ingested- every-day/ http://royal.pingdom.com/2012/01/17/internet-2011-in-numbers/ http://www.dataversity.net/the-growth-of-unstructured-data-what-are-we-going-to-do-with-all-those-zettabytes/

  5. Big Data on the Web Over 50 billion pages indexed and Articles from over 10,000 more than 2 million queries/min sources in real time ~4.5 million photos 48 hours of video uploaded/min; uploaded/day more than 1 trillion video views No. of mobile phones will exceed the world’s population by the end of 2012

  6. What to do with Big Data? • Extract information to make decisions • Evidence-based decision: data-driven vs. analysis based on intuition & experience • Analytics, business intelligence, data mining, machine learning, pattern recognition • Big Data computing: IBM is promoting Watson (Jeopardy champion) to tackle Big Data in healthcare, finance, drug design,.. Steve Lohr, “Amid the Flood, A Catchphrase is Born”, NY Times, August 12, 2012

  7. Decision Making • Data Representation • Features and similarity • Learning • Classification (labeled data) • Clustering (unlabeled data) Most big data problems have unlabeled objects 7 �

  8. Pattern Matrix n x d pattern matrix

  9. Similarity ¡Matrix ¡ Polynomial kernel: ¡ n x n similarity matrix

  10. Classification Dogs Cats Given a training set of labeled objects, learn a decision rule

  11. Clustering Given a collection of (unlabeled) objects, find meaningful groups

  12. Semi-supervised Clustering Supervised Unsupervised Dogs Cats Semi-supervised   Pairwise constraints improve the clustering performance

  13. What is a cluster? “A group of the same or similar elements gathered or occurring closely together” Cluster munition Birdhouse clusters Galaxy clusters Cluster lights Hongkeng Tulou cluster Cluster computing

  14. Clusters in 2D

  15. Challenges in Data Clustering • Measure of similarity • No. of clusters • Cluster validity • Outliers

  16. Data Clustering Organize a collection of n objects into a partition or a hierarchy (nested set of partitions) “Data clustering” returned ~6,100 hits for 2011 (Google Scholar)

  17. Clustering is the Key to Big Data Problem • Not feasible to “label” large collection of objects • No prior knowledge of the number and nature of groups (clusters) in data • Clusters may evolve over time • Clustering provides efficient browsing, search, recommendation and organization of data

  18. Clustering Users on Facebook • ~300,000 status updates per minute on tens of thousands of topics • Cluster users based on topic of status messages http://www.insidefacebook.com/2011/08/08/posted-about-page/ http://searchengineland.com/by-the-numbers-twitter-vs-facebook-vs-google-buzz-36709

  19. Clustering Articles on Google News Topic cluster Article Listings http://blogoscoped.com/archive/2006-07-28-n49.html

  20. Clustering Videos on Youtube Keywords • Popularity • Viewer • engagement User browsing • history http://www.strutta.com/blog/blog/six-degrees-of-youtube

  21. Clustering for Efficient Image retrieval Retrieval without clustering Retrieval with clustering Fig. 1. Upper-left image is the query. Numbers under the images on left side: image ID and cluster ID; on the right side: Image ID, matching score, number of regions. Retrieval accuracy for the “food” category (average precision): Without clustering: 47% With clustering: 61% Chen et al., “CLUE: cluster-based retrieval of images by unsupervised learning,” IEEE Tans. On Image Processing, 2005.

  22. Clustering Algorithms Hundreds of clustering algorithms are available; many are “admissible”, but no algorithm is “optimal” K-means • Gaussian mixture models • Kernel K-means • Spectral Clustering • Nearest neighbor • Latent Dirichlet Allocation • A.K. Jain, “Data Clustering: 50 Years Beyond K-Means”, PRL, 2011

  23. K-means Algorithm Re-compute centers Repeat until there is no change in the cluster labels Randomly assign cluster labels to the data points Compute the center of each cluster Assign points to the nearest cluster center

  24. K-means: Limitations Prefers “compact” and “isolated” clusters

  25. Gaussian Mixture Model Figueiredo & Jain, “Unsupervised Learning of Finite Mixture Models”, PAMI, 2002

  26. Kernel K-means Non-linear mapping to find clusters of arbitrary shapes Polynomial kernel representation

  27. Spectral Clustering Represent data using the top K eigenvectors of the kernel matrix; equivalent to Kernel K-means

  28. K-means vs. Kernel K-means K-means Kernel K-means Data Kernel clustering is able to find “complex” clusters How to choose the right kernel? RBF kernel is the default

  29. Kernel K-means is Expensive No. of operations No. of Objects Kernel (n) K-means K-means O(nKd) O(n 2 K) 1M 10 13 (6412*) 10 16 10M 10 14 10 18 100M 10 15 10 20 1B 10 16 10 22 d = 10,000; K=10 * Runtime in seconds on Intel Xeon 2.8 GHz processor using 40 GB memory A petascale supercomputer (IBM Sequoia, June 2012) with ~1 exabyte memory is needed to run kernel K-means on 1 billion points!

  30. Clustering Big Data Sampling Data Summarization Pre-processing Incremental n x n similarity Distributed Clustering matrix Approximation Cluster labels

  31. Distributed Clustering Clustering 100,000 2-D points with 2 clusters on 2.3 GHz quad-core Intel Xeon processors, with 8GB memory in intel07 cluster K-means Speedup Number of Kernel processors K-means K- means 2 1.1 1.3 3 2.4 1.5 Kernel K-means 4 3.1 1.6 5 3.0 3.8 6 3.1 1.9 7 3.3 1.5 8 1.2 1.5 Network communication cost increases with the no. of processors

  32. Approximate kernel K-means Tradeoff between clustering accuracy and running time Given n points in d-dimensional space Obtain the final cluster labels Linear runtime and memory complexity Chitta, Jin, Havens & Jain, Approximate Kernel k-means: solution to Large Scale Kernel Clustering, KDD , 2011

  33. Approximate Kernel K-Means Running time Clustering accuracy (%) (seconds) No. of Approximate objects Approximate Kernel kernel K- (n) kernel K- K- K-means Kernel K-means K-means means means (m=100) means (m=100) 10K 3.09 0.20 0.03 100 93.8 50.1 100K 320.10 1.18 0.17 100 93.7 49.9 1M - 15.06 0.72 - 95.1 50.0 10M - 234.49 12.14 - 91.6 50.0 2.8 GHz processor, 40 GB

  34. Tiny Image Data set ~80 million 32x32 images from ~75K classes (bamboo, fish, mushroom, leaf, mountain,…); image represented by 384- dim. GIST descriptors Fergus et al., 80 million tiny images: a large dataset for non-parametric object and scene recognition , PAMI 2008

  35. Tiny Image Data set 10-class subset (CIFAR-10): 60K manually annotated images Airplane Automobile Bird Cat Deer Dog Frog Horse Ship Truck Krizhevsky, Learning multiple layers of features from tiny images , 2009

  36. Clustering Tiny Images C 1 Average clustering time (100 clusters) C 2 Approximate kernel K- 8.5 means (m=1,000) hours C 3 C 4 K-means 6 hours C 5 2.3GHz, 150GB memory Example Clusters

  37. Clustering Tiny Images Best Supervised Classification Accuracy on CIFAR-10: 54.7% Clustering accuracy Kernel K-means 29.94% Approximate kernel K-means 29.76% (m = 5,000) Spectral clustering 27.09% K-means 26.70% Ranzato et. Al., Modeling pixel means and covariances using factorized third-order boltzmann machines, CVPR 2010 Fowlkes et al., Spectral grouping using the Nystrom method , PAMI 2004

  38. Distributed Approx. Kernel K-means For better scalability and faster clustering Split the remaining n - m randomly into p partitions and assign Combine the labels from each task using ensemble clustering Assign each point in task s (s ≠ t) to the closest center from Run approximate kernel K-means in each task t and find the Randomly sample m points (m << n) Given n points in d-dimensional space partition P t to task t cluster centers algorithm task t

  39. Distributed Approximate kernel K-means 2-D data set with 2 concentric circles Running time Size of Speedup data set 10K 3.8 100K 4.8 1M 3.8 10M 6.4 2.3 GHz quad-core Intel Xeon processors, with 8GB memory in the intel07 cluster

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns in data, or grouping similar groups of data-points together into clusters . Clustering algorithms for numeric data: Lloyds K-means, EM

688 views • 19 slides
Clustering Clustering What? Given some input data, partition the data in multiple groups

Clustering Clustering What? Given some input data, partition the data in multiple groups

Clustering Clustering What? Given some input data, partition the data in multiple groups Why? Approximate large/infinite/continuous set of objects with finite set of representatives Eg. Vector quantization, codebook learning,

846 views • 68 slides
Data Mining in Bioinformatics Day 7: Clustering in Bioinformatics Clustering Gene Expression Data

Data Mining in Bioinformatics Day 7: Clustering in Bioinformatics Clustering Gene Expression Data

Data Mining in Bioinformatics Day 7: Clustering in Bioinformatics Clustering Gene Expression Data Chlo-Agathe Azencott &amp; Karsten Borgwardt February 18 to March 1, 2013 Machine Learning &amp; Computational Biology Research Group Max

538 views • 51 slides
Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat Clustering 7.3 Embedding into Vector Space for Visualization 7.4 Applications Clustering: unsupervised grouping (partitioning) of objects into

763 views • 49 slides
Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as

Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as

Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as any other clustering Identification of communities in social networks Webpage clustering for better data management of web data Outline

607 views • 41 slides
k -means clustering Method to automatically separate data sets into distinct groups. Clustering

k -means clustering Method to automatically separate data sets into distinct groups. Clustering

k -means clustering Method to automatically separate data sets into distinct groups. Clustering example Clustering example k -means clustering algorithm 1. Start with k randomly chosen means 2. Color data points by the shortest distance to any

1.56k views • 19 slides
Data Mining in Bioinformatics Day 8: Clustering in Bioinformatics Clustering Gene Expression Data

Data Mining in Bioinformatics Day 8: Clustering in Bioinformatics Clustering Gene Expression Data

Data Mining in Bioinformatics Day 8: Clustering in Bioinformatics Clustering Gene Expression Data Chlo-Agathe Azencott &amp; Karsten Borgwardt February 10 to February 21, 2014 Machine Learning &amp; Computational Biology Research Group Max

522 views • 50 slides
Data-driven Clustering via Parameterized Lloyds Families Travis Dick Joint work with

Data-driven Clustering via Parameterized Lloyds Families Travis Dick Joint work with

Data-driven Clustering via Parameterized Lloyds Families Travis Dick Joint work with Maria-Florina Balcan and Colin White Carnegie Mellon University NeurIPS 2018 Data-driven Clustering Data-driven Clustering Clustering aims to divide a

675 views • 53 slides
Bi-clustering and co-clustering Going further in cluster analysis and classifjcation: publics ou

Bi-clustering and co-clustering Going further in cluster analysis and classifjcation: publics ou

HAL Id: hal-01810380 manant des tablissements denseignement et de Summer School on Clustering, Data Analysis and Visualization of Complex Data, May 2018, Catania, Going further in cluster analysis and classifjcation: Bi-clustering and

1.23k views • 67 slides
Clustering A Categorization of Major Clustering Methods Partitioning Methods

Clustering A Categorization of Major Clustering Methods Partitioning Methods

Clustering What is Clustering? Types of Data in Cluster Analysis Clustering A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods 1 2 What is Clustering? What is Clustering? Typical

647 views • 18 slides
Clustering Lecture notes Clustering is Exploratory, unsupervised method Data in cluster is

Clustering Lecture notes Clustering is Exploratory, unsupervised method Data in cluster is

Clustering Lecture notes Clustering is Exploratory, unsupervised method Data in cluster is similar to each other, and dissimilar to other cluster data Finding structure in data Hard vs. Soft/Fuzzy vs. Exclusive Understanding and/or summarise

400 views • 26 slides
DS504/CS586: Big Data Analytics Big Data Clustering II Prof. Yanhua Li Time: 6pm 8:50pm Thu

DS504/CS586: Big Data Analytics Big Data Clustering II Prof. Yanhua Li Time: 6pm 8:50pm Thu

Welcome to DS504/CS586: Big Data Analytics Big Data Clustering II Prof. Yanhua Li Time: 6pm 8:50pm Thu Location: AK 232 Fall 2016 More Discussions, Limitations v Center based clustering K-means BFR algorithm v Hierarchical clustering

873 views • 52 slides
Clustering Data Clustering with user constraints The clustering problem : Given a set of

Clustering Data Clustering with user constraints The clustering problem : Given a set of

Clustering Data Clustering with user constraints The clustering problem : Given a set of objects, find groups of similar objects Cluster: a collection of data objects Similar to one another within the same cluster

478 views • 36 slides
Clustering Clustering is an unsupervised classification method, i.e. unlabeled data is partitioned

Clustering Clustering is an unsupervised classification method, i.e. unlabeled data is partitioned

Clustering Clustering is an unsupervised classification method, i.e. unlabeled data is partitioned into subsets (clusters), according to a similarity measure, such thatsimilardata is grouped into the same cluster. Unlabeled Data

442 views • 22 slides
CHAPTER VIII VIII CHAPTER Data Clustering and Data Clustering and Self- -Organizing Feature

CHAPTER VIII VIII CHAPTER Data Clustering and Data Clustering and Self- -Organizing Feature

Ugur HALICI - METU EEE - ANKARA 11/18/2004 CHAPTER VIII VIII CHAPTER Data Clustering and Data Clustering and Self- -Organizing Feature Maps Organizing Feature Maps Self CHAPTER VI : VI : Data Clustering CHAPTER Data Clustering &amp;

337 views • 16 slides
Lecture 12: Clustering Geoffrey Hinton Clustering We assume that the data was generated from

Lecture 12: Clustering Geoffrey Hinton Clustering We assume that the data was generated from

CSC321: Neural Networks Lecture 12: Clustering Geoffrey Hinton Clustering We assume that the data was generated from a number of different classes. The aim is to cluster data from the same class together. How do we decide the number of

529 views • 21 slides
Stability of Cluster Analysis 2. Preparation of the data 3. Distance measure used S T A T I S

Stability of Cluster Analysis 2. Preparation of the data 3. Distance measure used S T A T I S

Stability of Cluster Analysis For real data sets without obvious grouping structure the stability of clusters depends on: 1. Input data - the selection of variables Stability of Cluster Analysis 2. Preparation of the data 3. Distance measure

384 views • 4 slides
INFO 1998: Introduction to Machine Learning Lecture 9: Clustering and Unsupervised Learning INFO

INFO 1998: Introduction to Machine Learning Lecture 9: Clustering and Unsupervised Learning INFO

INFO 1998: Introduction to Machine Learning Lecture 9: Clustering and Unsupervised Learning INFO 1998: Introduction to Machine Learning If intelligence was a cake, unsupervised learning would be the cake, supervised learning would be the

410 views • 27 slides
10601 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview

10601 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview

10601 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview Clustering - Hierarchical, semi-supervised learning Graphical models - Bayesian networks, HMMs, Reasoning under uncertainty Putting it

862 views • 44 slides
November 2013 Agenda AGM Forum Meeting - Decommissioning Petrol Stations Kirsten Hotchkiss,

November 2013 Agenda AGM Forum Meeting - Decommissioning Petrol Stations Kirsten Hotchkiss,

SCLF AGM and Forum Meeting November 2013 Agenda AGM Forum Meeting - Decommissioning Petrol Stations Kirsten Hotchkiss, Trading Stds, East Dunbartonshire Council - Oil Spill Preparation and Response Sam Lord, Operations Manager, Soilutions Ltd

88 views • 8 slides
Data Mining: Concepts and Techniques Cluster Analysis Li Xiong Slide credits: Jiawei Han and

Data Mining: Concepts and Techniques Cluster Analysis Li Xiong Slide credits: Jiawei Han and

Data Mining: Concepts and Techniques Cluster Analysis Li Xiong Slide credits: Jiawei Han and Micheline Kamber Tan, Steinbach, Kumar September 26, 2017 Data Mining: Concepts and Techniques 1 Cluster Analysis: Basic Concepts and Methods

1.97k views • 116 slides
Alternative Clusterings: Current Progress and Open Challenges James Bailey Department of

Alternative Clusterings: Current Progress and Open Challenges James Bailey Department of

Alternative Clusterings: Current Progress and Open Challenges James Bailey Department of Computer Science and Software Engineering The University of Melbourne, Australia 1 Introduction Cluster analysis: group similar objects into

348 views • 19 slides
Machine Learning Lecture Notes on Clustering (IV) 2016-2017 Davide Eynard davide.eynard@usi.ch

Machine Learning Lecture Notes on Clustering (IV) 2016-2017 Davide Eynard davide.eynard@usi.ch

Machine Learning Lecture Notes on Clustering (IV) 2016-2017 Davide Eynard davide.eynard@usi.ch Institute of Computational Science Universit` a della Svizzera italiana p. 1/34 Lecture outline Cluster Evaluation Internal measures

514 views • 34 slides
K -means Clustering Ke Chen Reading: [7.3, EA], [9.1, CMB] COMP24111 Machine Learning Outline

K -means Clustering Ke Chen Reading: [7.3, EA], [9.1, CMB] COMP24111 Machine Learning Outline

K -means Clustering Ke Chen Reading: [7.3, EA], [9.1, CMB] COMP24111 Machine Learning Outline Introduction K -means Algorithm Example How K -means partitions? K -means Demo Relevant Issues Application:

416 views • 22 slides

More recommend