clustering
play

Clustering Themis Palpanas University of Trento - PDF document

Jan 16, 2024 •317 likes •580 views

Data Mining for Knowledge Management Clustering Themis Palpanas University of Trento http://disi.unitn.eu/~themis 1 Data Mining for Knowledge Management Thanks for slides to: Jiawei Han Eamonn Keogh Jeff Ullman 2 Data

  1. Data Mining for Knowledge Management Clustering Themis Palpanas University of Trento http://disi.unitn.eu/~themis 1 Data Mining for Knowledge Management Thanks for slides to: Jiawei Han  Eamonn Keogh  Jeff Ullman  2 Data Mining for Knowledge Management 1

  2. Roadmap 1. What is Cluster Analysis? 2. Types of Data in Cluster Analysis 3. A Categorization of Major Clustering Methods 4. Partitioning Methods 5. Hierarchical Methods 6. Density-Based Methods 7. Grid-Based Methods 8. Model-Based Methods 9. Clustering High-Dimensional Data 10. Constraint-Based Clustering 11. Summary 3 Data Mining for Knowledge Management What is Cluster Analysis?  Cluster: a collection of data objects  Similar to one another within the same cluster  Dissimilar to the objects in other clusters  Cluster analysis  Finding similarities between data according to the characteristics found in the data and grouping similar data objects into clusters 4 Data Mining for Knowledge Management 2

  3. Example: Clusters x x xx x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 5 Data Mining for Knowledge Management Example: Clusters x x xx x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 6 Data Mining for Knowledge Management 3

  4. What is Cluster Analysis?  Cluster: a collection of data objects  Similar to one another within the same cluster  Dissimilar to the objects in other clusters  Cluster analysis  Finding similarities between data according to the characteristics found in the data and grouping similar data objects into clusters  Unsupervised learning: no predefined classes  Typical applications  As a stand-alone tool to get insight into data distribution  As a preprocessing step for other algorithms 7 Data Mining for Knowledge Management Clustering: Rich Applications and Multidisciplinary Efforts  Pattern Recognition  Spatial Data Analysis  Create thematic maps in GIS by clustering feature spaces  Detect spatial clusters or for other spatial mining tasks  Image Processing  Economic Science (especially market research)  WWW  Document classification  Cluster Weblog data to discover groups of similar access patterns 8 Data Mining for Knowledge Management 4

  5. Examples of Clustering Applications Marketing: Help marketers discover distinct groups in their customer  bases, and then use this knowledge to develop targeted marketing programs Land use: Identification of areas of similar land use in an earth  observation database Insurance: Identifying groups of motor insurance policy holders with  a high average claim cost City-planning: Identifying groups of houses according to their house  type, value, and geographical location Earth-quake studies: Observed earth quake epicenters should be  clustered along continent faults 9 Data Mining for Knowledge Management Quality: What Is Good Clustering?  A good clustering method will produce high quality clusters with  high intra-class similarity  low inter-class similarity  The quality of a clustering result depends on both the similarity measure used by the method and its implementation  The quality of a clustering method is also measured by its ability to discover some or all of the hidden patterns 10 Data Mining for Knowledge Management 5

  6. Measure the Quality of Clustering  Dissimilarity/Similarity metric: Similarity is expressed in terms of a distance function, typically metric: d ( i, j )  There is a separate “quality” function that measures the “goodness” of a cluster.  The definitions of distance functions are usually very different for interval-scaled, boolean, categorical, ordinal ratio, vector, and string variables.  Weights should be associated with different variables based on applications and data semantics.  It is hard to define “similar enough” or “good enough” the answer is typically highly subjective.  11 Data Mining for Knowledge Management Problems With Clustering  Clustering in two dimensions looks easy.  Clustering small amounts of data looks easy.  And in most cases, looks are not deceiving. 12 Data Mining for Knowledge Management 6

  7. The Curse of Dimensionality  Many applications involve not 2, but 10 or 10,000 dimensions.  High-dimensional spaces look different: almost all pairs of points are at about the same distance.  Example: assume random points within a bounding box, e.g., values between 0 and 1 in each dimension. 13 Data Mining for Knowledge Management Example: SkyCat  A catalog of 2 billion “sky objects” represents objects by their radiation in 9 dimensions (frequency bands).  Problem: cluster into similar objects, e.g., galaxies, nearby stars, quasars, etc.  Sloan Sky Survey is a newer, better version. 14 Data Mining for Knowledge Management 7

  8. Example : Clustering CD’s (Collaborative Filtering)  Intuitively: music divides into categories, and customers prefer a few categories.  But what are categories really?  Represent a CD by the customers who bought it.  Similar CD’s have similar sets of customers, and vice - versa. 15 Data Mining for Knowledge Management The Space of CD’s  Think of a space with one dimension for each customer.  Values in a dimension may be 0 or 1 only.  A CD’s point in this space is ( x 1 , x 2 ,…, x k ), where x i = 1 iff the i th customer bought the CD.  Compare with the “shingle/signature” matrix: rows = customers; cols. = CD’s.  For Amazon, the dimension count is tens of millions. 16 Data Mining for Knowledge Management 8

  9. Example: Clustering Documents  Represent a document by a vector ( x 1 , x 2 ,…, x k ), where x i = 1 iff the i th word (in some order) appears in the document.  It actually doesn’t matter if k is infinite; i.e., we don’t limit the set of words.  Documents with similar sets of words may be about the same topic. 17 Data Mining for Knowledge Management Example: Gene Sequences  Objects are sequences of {C,A,T,G}.  Distance between sequences is edit distance , the minimum number of inserts and deletes needed to turn one into the other.  Note there is a “distance,” but no convenient space in which points “live.” 18 Data Mining for Knowledge Management 9

  10. Requirements of Clustering in Data Mining  Scalability  Ability to deal with different types of attributes  Ability to handle dynamic data  Discovery of clusters with arbitrary shape  Minimal requirements for domain knowledge to determine input parameters  Able to deal with noise and outliers  Insensitive to order of input records  High dimensionality  Incorporation of user-specified constraints  Interpretability and usability 19 Data Mining for Knowledge Management Roadmap 1. What is Cluster Analysis? 2. Types of Data in Cluster Analysis 3. A Categorization of Major Clustering Methods 4. Partitioning Methods 5. Hierarchical Methods 6. Density-Based Methods 7. Grid-Based Methods 8. Model-Based Methods 9. Clustering High-Dimensional Data 10. Constraint-Based Clustering 11. Summary 20 Data Mining for Knowledge Management 10

  11. Type of data in clustering analysis  Interval-scaled variables  Binary variables  Categorical (or Nominal), ordinal, and ratio variables  Variables of mixed types 21 Data Mining for Knowledge Management Interval-valued variables  Standardize data  Calculate the mean absolute deviation: 1 (| | | | ... | |) s x m x m x m n f 1 f f 2 f f nf f 1 where m (x x x ... ) n . f 1 f 2 f nf  Calculate the standardized measurement ( z-score ) x m if f z s if f  Using mean absolute deviation is more robust than using standard deviation 22 Data Mining for Knowledge Management 11

  12. Similarity and Dissimilarity Between Objects  Distances are normally used to measure the similarity or dissimilarity between two data objects  Some popular ones include: Minkowski distance : q q q ( , ) (| | | | ... | | ) d i j x x x x x x q i j i j i j 1 1 2 2 p p where i = ( x i1 , x i2 , …, x ip ) and j = ( x j1 , x j2 , …, x jp ) are two p - dimensional data objects, and q is a positive integer  Also, one can use weighted distance, parametric Pearson product moment correlation, or other dissimilarity measures 23 Data Mining for Knowledge Management Similarity and Dissimilarity Between Objects (Cont.)  If q = 1, d is Manhattan distance ( , ) | | | | ... | | d i j x x x x x x i j i j i j 1 1 2 2 p p 24 Data Mining for Knowledge Management 12

  13. Similarity and Dissimilarity Between Objects (Cont.) 25 Data Mining for Knowledge Management Similarity and Dissimilarity Between Objects (Cont.)  If q = 1, d is Manhattan distance ( , ) | | | | ... | | d i j x x x x x x i j i j i j 1 1 2 2 p p  If q = 2 , d is Euclidean distance: 2 2 2 ( , ) (| | | | ... | | ) d i j x x x x x x i j i j i j 1 1 2 2 p p 26 Data Mining for Knowledge Management 13

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering? Aykut Erdem May 2016 Hacettepe University Last time K-Means An iterative clustering algorithm - Initialize: Pick K random points as cluster

932 views • 64 slides
Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity

772 views • 40 slides
Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns in data, or grouping similar groups of data-points together into clusters . Clustering algorithms for numeric data: Lloyds K-means, EM

688 views • 19 slides
Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity and high separation Many

768 views • 43 slides
Clustering DSE 210 Clustering in R d Two common uses of clustering: Vector quantization Find

Clustering DSE 210 Clustering in R d Two common uses of clustering: Vector quantization Find

Clustering DSE 210 Clustering in R d Two common uses of clustering: Vector quantization Find a finite set of representatives that provides good coverage of a complex, possibly infinite, high-dimensional space. Finding meaningful

396 views • 36 slides
Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine Learning 10-601B Seyoung Kim Many of these slides are derived from William Cohen,

660 views • 46 slides
Clustering and Dimensionality Reduction Preview Clustering K -means clustering

Clustering and Dimensionality Reduction Preview Clustering K -means clustering

Clustering and Dimensionality Reduction Preview Clustering K -means clustering Mixture models Hierarchical clustering Dimensionality reduction Principal component analysis Multidimensional scaling Isomap

1.61k views • 17 slides
Clustering A Categorization of Major Clustering Methods Partitioning Methods

Clustering A Categorization of Major Clustering Methods Partitioning Methods

Clustering What is Clustering? Types of Data in Cluster Analysis Clustering A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods 1 2 What is Clustering? What is Clustering? Typical

647 views • 18 slides
Subspace Clustering Ensemble Clustering Subspace Clustering, Ensemble Clustering, Alternative

Subspace Clustering Ensemble Clustering Subspace Clustering, Ensemble Clustering, Alternative

LUDWIG- MAXIMILIANS- INSTITUTE DATABASE UNIVERSITT FOR SYSTEMS MNCHEN MNCHEN INFORMATICS INFORMATICS GROUP GROUP Subspace Clustering Ensemble Clustering Subspace Clustering, Ensemble Clustering, Alternative Clustering, Multiview

513 views • 9 slides
Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering Clustering by Partitioning, e.g. k-Means Density Based Clustering, e.g. DBScan Grid Based Clustering Compendium slides for Guide to Intelligent

1.32k views • 82 slides
Introduction to Machine Learning, Clustering and EM Barnab s P czos Contents Clustering

Introduction to Machine Learning, Clustering and EM Barnab s P czos Contents Clustering

Introduction to Machine Learning, Clustering and EM Barnab s P czos Contents Clustering K-means Mixture of Gaussians Expectation Maximization Variational Methods 2 Clustering 3 K- means clustering What is clustering?

1.32k views • 66 slides
PAC-Bayesian Analysis of Co-clustering, Graph Clustering and Pairwise Clustering Yevgeny Seldin

PAC-Bayesian Analysis of Co-clustering, Graph Clustering and Pairwise Clustering Yevgeny Seldin

PAC-Bayesian Analysis of Co-clustering, Graph Clustering and Pairwise Clustering Yevgeny Seldin Motivation Example Clustering cannot be analyzed without specifying what it will be used for! be used for! Example Cluster then pack

293 views • 28 slides
Clustering kMeans, Expectation Maximization, Self-Organizing Maps Outline K-means

Clustering kMeans, Expectation Maximization, Self-Organizing Maps Outline K-means

Clustering kMeans, Expectation Maximization, Self-Organizing Maps Outline K-means clustering Hierarchical clustering Incremental clustering Probability-based clustering Self-Organising Maps Classification vs. Clustering

1k views • 67 slides
Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat Clustering 7.3 Embedding into Vector Space for Visualization 7.4 Applications Clustering: unsupervised grouping (partitioning) of objects into

763 views • 49 slides
Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as

Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as

Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as any other clustering Identification of communities in social networks Webpage clustering for better data management of web data Outline

607 views • 41 slides
Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean

Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean

Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean clustering, hierarchical clustering Adaptive learning (online learning) CL, FSCL, RPCL Gaussian Mixture Models (GMM)

805 views • 34 slides
Clustering Data Mining: Concepts and October 18, 2019 Techniques 1 Chapter 8. Cluster Analysis

Clustering Data Mining: Concepts and October 18, 2019 Techniques 1 Chapter 8. Cluster Analysis

Clustering Data Mining: Concepts and October 18, 2019 Techniques 1 Chapter 8. Cluster Analysis What is Cluster Analysis? Types of Data in Cluster Analysis A Categorization of Major Clustering Methods Partitioning Methods

893 views • 32 slides
Learning Deep Broadband Network@HOME Hongjoo LEE Who am I? Machine Learning Engineer

Learning Deep Broadband Network@HOME Hongjoo LEE Who am I? Machine Learning Engineer

Learning Deep Broadband Network@HOME Hongjoo LEE Who am I? Machine Learning Engineer Fraud Detection System Software Defect Prediction Software Engineer Email Services (40+ mil. users) High traffic server (IPC,

630 views • 59 slides
Measuring inequality - Week 9 ECON1910 - Poverty and distribution in developing countries

Measuring inequality - Week 9 ECON1910 - Poverty and distribution in developing countries

Measuring inequality - Week 9 ECON1910 - Poverty and distribution in developing countries Readings: Ray chapter 6 5. March 2010 (Readings: Ray chapter 6) Measuring inequality - Week 9 5. March 2010 1 / 30 Why care about economic inequality?

476 views • 30 slides
= x ... What is a Statistic ? What are Statistic s ? A quantity that is computed

= x ... What is a Statistic ? What are Statistic s ? A quantity that is computed

Why do we need statistics? CS533 Modeling and Performance 1. Noise, noise, noise, noise, noise! Evaluation of Network and Computer Systems Statistics for Performance Evaluation OK not really this type of noise (Chapters 12-15) Why Do

774 views • 20 slides
Where should Background Research contributions infrastructure be Supporting

Where should Background Research contributions infrastructure be Supporting

10/24/2014 Outline Introduction (VM) Virtual Machine Research goals (PM) Physical Machine Challenges Research questions Background Research contributions PHD Dissertation Defense Supporting Infrastructure Research

297 views • 13 slides
Data Mining Fundamentals Liyao Xiang http://xiangliyao.cn/ Shanghai Jiao Tong University

Data Mining Fundamentals Liyao Xiang http://xiangliyao.cn/ Shanghai Jiao Tong University

EE226 Big Data Mining Lecture 2 Data Mining Fundamentals Liyao Xiang http://xiangliyao.cn/ Shanghai Jiao Tong University http://jhc.sjtu.edu.cn/public/courses/EE226/ Please check https://oc.sjtu.edu.cn/login/ canvas for slides, announcement,

796 views • 61 slides
) ( (6-1) ( = P X ) B f ( x ) dx . X B Note that represents

) ( (6-1) ( = P X ) B f ( x ) dx . X B Note that represents

6. Mean, Variance, Moments and Characteristic Functions For a r.v X , its p.d.f represents complete information f X ( x ) about it, and for any Borel set B on the x -axis ) ( (6-1) ( = P X ) B f ( x ) dx . X B

731 views • 50 slides
Data Analysis and Approximate Models Laurie Davies Fakult at Mathematik, Universit at

Data Analysis and Approximate Models Laurie Davies Fakult at Mathematik, Universit at

Data Analysis and Approximate Models Laurie Davies Fakult at Mathematik, Universit at Duisburg-Essen CRiSM Workshop: Non-likelihood Based Statistical Modelling, University of Warwick, 7-9 September 2015 Is statistics too difficult?

1.27k views • 69 slides

More recommend