hierarchical clustering
play

Hierarchical Clustering MAT 6480W / STT 6705V Guy Wolf - PowerPoint PPT Presentation

Oct 06, 2023 •139 likes •709 views

Geometric Data Analysis Hierarchical Clustering MAT 6480W / STT 6705V Guy Wolf guy.wolf@umontreal.ca Universit e de Montr eal Fall 2019 MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 1 / 17 Outline Hierarchical

  1. Geometric Data Analysis Hierarchical Clustering MAT 6480W / STT 6705V Guy Wolf guy.wolf@umontreal.ca Universit´ e de Montr´ eal Fall 2019 MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 1 / 17

  2. Outline Hierarchical clustering 1 Divisive & agglomerative approaches Dendrogram visualization Bisecting k -means Agglomerative clustering 2 Single linkage Complete linkage Average linkage Ward’s method Large-scale clustering 3 CURE BIRCH Chameleon MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 2 / 17

  3. Hierarchical clustering Question: how many cluster should we find in the data? MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 3 / 17

  4. Hierarchical clustering Question: how many cluster should we find in the data? MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 3 / 17

  5. Hierarchical clustering Question: how many cluster should we find in the data? MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 3 / 17

  6. Hierarchical clustering Question: how many cluster should we find in the data? MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 3 / 17

  7. Hierarchical clustering Question: how many cluster should we find in the data? Suggestion: why not consider all options in a single hierarchy? MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 3 / 17

  8. Hierarchical clustering A hierarchical approach can be useful when considering versatile cluster shapes: 2-means By first detecting many small clusters, and then merging them, we can uncover patterns that are challenging for partitional methods. MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 3 / 17

  9. Hierarchical clustering A hierarchical approach can be useful when considering versatile cluster shapes: 10-means By first detecting many small clusters, and then merging them, we can uncover patterns that are challenging for partitional methods. MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 3 / 17

  10. Hierarchical clustering Divisive & agglomerative approaches Hierarchical clustering methods produce a set of nested clusters organized in a hierarchy tree. The cluster hierarchy is typically visualized using dendrograms Such approaches are applied either to provide multiresolution data organization, or to alleviate computational challenges when clustering big datasets In general, two approaches are applied to build nested clusters: divisive clustering and agglomerative clustering. Divisive approaches start with the entire data as one cluster, and then iteratively split “loose” clusters until a stopping criterion (e.g., k clusters or tight enough clusters) is satisfied. Agglomerative approaches start with small tight clusters, or even with single-point clusters, and then iteratively merge close clusters until only a single one remains. MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 4 / 17

  11. Hierarchical clustering Dendrogram visualization A dendrogram is a tree graph that visualizes a sequence of cluster merges or divisions: Divisive methods take a top-down root → leaves approach. Agglomerative ones take a bottom-up leaves → root approach. MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 5 / 17

  12. Hierarchical clustering Bisecting k-means Bisecting k -means is a divisive algorithm that utilized the k -means iteratively to bisect the data into clusters. Bisecting k -means Use 2 -means to split the data into two clusters 1 While there are less than k clusters: Select C as the cluster with the highest SSE Use 2 -means to split C into two clusters 1 Replace C with the two new clusters The hierarchical approach in this case is used to stabilize some of the weaknesses of the original k -means algorithm, and not for data organization purposes. 1 Choose best SSE out of t attempts MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 6 / 17

  13. Hierarchical clustering Bisecting k-means Example MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 6 / 17

  14. Hierarchical clustering Bisecting k-means Example MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 6 / 17

  15. Hierarchical clustering Bisecting k-means Example MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 6 / 17

  16. Hierarchical clustering Bisecting k-means Example MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 6 / 17

  17. Hierarchical clustering Bisecting k-means Example MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 6 / 17

  18. Hierarchical clustering Bisecting k-means Example MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 6 / 17

  19. Hierarchical clustering Bisecting k-means Example MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 6 / 17

  20. Hierarchical clustering Bisecting k-means Example MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 6 / 17

  21. Hierarchical clustering Bisecting k-means Example MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 6 / 17

  22. Hierarchical clustering Bisecting k-means Example MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 6 / 17

  23. Agglomerative clustering Agglomerative clustering approaches are more popular than divisive ones. They all use variations of the following simple algorithm: Agglomerative clustering paradigm Build a singleton cluster for each data point Repeat the following steps: Find the two closest clusters Merge these two clusters together Until there is only a single cluster Two main choices distinguish agglomerative clustering algorithms: How to quantify proximity between clusters 1 How to merge clusters and efficiently update this proximity 2 MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 7 / 17

  24. Agglomerative clustering Agglomerative clustering approaches are more popular than divisive ones. They all use variations of the following simple algorithm: Agglomerative clustering paradigm Build a singleton cluster for each data point Repeat the following steps: Find the two closest clusters Merge these two clusters together Until there is only a single cluster With proper implementation, this approach is also helpful for Big Data processing, since each iteration considers a smaller coarse-grained ver- sion of the dataset. MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 7 / 17

  25. Agglomerative clustering MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 7 / 17

  26. Agglomerative clustering MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 7 / 17

  27. Agglomerative clustering MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 7 / 17

  28. Agglomerative clustering MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 7 / 17

  29. Agglomerative clustering Linkage How to quantify distance or similarity between clusters? MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 8 / 17

  30. Agglomerative clustering Linkage How to quantify distance or similarity between clusters? Suggestion #1: represent clusters by centroids and use dis- tance/similarity between them. MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 8 / 17

  31. Agglomerative clustering Linkage How to quantify distance or similarity between clusters? Suggestion #1: represent clusters by centroids and use dis- tance/similarity between them. Problem: this approach ignores the shapes of the clusters. MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 8 / 17

  32. Agglomerative clustering Linkage How to quantify distance or similarity between clusters? Suggestion #1: represent clusters by centroids and use dis- tance/similarity between them. Problem: this approach ignores the shapes of the clusters. Suggestion #2: combine pairwise distances between each point in one cluster and each point in the other cluster. This approach is called linkage . MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 8 / 17

  33. Agglomerative clustering Single linkage Single linkage uses minimal distance (or maximum similarity) between a point in one cluster and a point in the other cluster. Only one inter-cluster link determines the distance, while many other links can be significantly weaker. MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 9 / 17

  34. Agglomerative clustering Single linkage Example MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 9 / 17

  35. Agglomerative clustering Complete linkage Complete linkage uses maximal distance (or minimal similarity) be- tween a point in one cluster and a point in the other cluster. In some sense, all inter-cluster links are considered since they must all be strong to have a small distance. MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 10 / 17

  36. Agglomerative clustering Complete linkage Example MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 10 / 17

  37. Agglomerative clustering Average linkage Average linkage uses mean distance (or similarity) between points in one cluster and points in the other cluster. Less susceptible than single- and complete-linkage to noise an outliers, but biased toward globular clusters. MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 11 / 17

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering? Aykut Erdem May 2016 Hacettepe University Last time K-Means An iterative clustering algorithm - Initialize: Pick K random points as cluster

932 views • 64 slides
Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity

772 views • 40 slides
Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning no labels/output, only x/input Clustering Group similar points together Machine learning taxonomy Supervised Semi-Supervised Unsupervised

366 views • 15 slides
Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine Learning 10-601B Seyoung Kim Many of these slides are derived from William Cohen,

660 views • 46 slides
Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity and high separation Many

768 views • 43 slides
Unsupervised Learning and Clustering Owen Roberts, Zach Busser, Ganesh Sugunan Hierarchical

Unsupervised Learning and Clustering Owen Roberts, Zach Busser, Ganesh Sugunan Hierarchical

Unsupervised Learning and Clustering Owen Roberts, Zach Busser, Ganesh Sugunan Hierarchical Clustering Hierarchical Clustering Hierarchical Representations Hierarchical Algorithms Agglomeration "Nearest" Clusters Complexity

769 views • 49 slides
RECSM Summer School: Machine Learning for Social Sciences Session 3.4: Hierarchical Clustering

RECSM Summer School: Machine Learning for Social Sciences Session 3.4: Hierarchical Clustering

RECSM Summer School: Machine Learning for Social Sciences Session 3.4: Hierarchical Clustering Reto West Department of Political Science and International Relations University of Geneva 1 Clustering Clustering Hierarchical Clustering

237 views • 22 slides
Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering Clustering by Partitioning, e.g. k-Means Density Based Clustering, e.g. DBScan Grid Based Clustering Compendium slides for Guide to Intelligent

1.32k views • 82 slides
Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat Clustering 7.3 Embedding into Vector Space for Visualization 7.4 Applications Clustering: unsupervised grouping (partitioning) of objects into

763 views • 49 slides
Hierarchical clustering David M. Blei COS424 Princeton University February 28, 2008 D. Blei

Hierarchical clustering David M. Blei COS424 Princeton University February 28, 2008 D. Blei

Hierarchical clustering David M. Blei COS424 Princeton University February 28, 2008 D. Blei Clustering 02 1 / 21 Hierarchical clustering Hierarchical clustering is a widely used data analysis tool. D. Blei Clustering 02 2 / 21

1.1k views • 83 slides
LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

DATA MINING LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm Clustering Evaluation What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or

1.39k views • 110 slides
Hierarchical Clustering Lecture 15 David Sontag New York

Hierarchical Clustering Lecture 15 David Sontag New York

Hierarchical Clustering Lecture 15 David Sontag New York University Agglomerative Clustering Agglomerative clustering: First merge very similar instances Incrementally build larger clusters out

465 views • 7 slides
Separation and convexity properties of hierarchical and non hierarchical clustering Patrice

Separation and convexity properties of hierarchical and non hierarchical clustering Patrice

Plan Separation and convexity properties of hierarchical and non hierarchical clustering Patrice Bertrand 1 1 CEREMADE, Universit e Paris-Dauphine, Paris, France Joint work with Jean Diatta 2 2 LIM, Universit e de La R eunion,

401 views • 29 slides
Cl Clustering t i A Categorization of Major Clustering Methods Partitioning Methods

Cl Clustering t i A Categorization of Major Clustering Methods Partitioning Methods

Clustering What is Clustering? Types of Data in Cluster Analysis Cl Clustering t i A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods Hierarchical Methods 1 2 What is Clustering?

617 views • 24 slides
10601 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview

10601 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview

10601 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview Clustering - Hierarchical, semi-supervised learning Graphical models - Bayesian networks, HMMs, Reasoning under uncertainty Putting it

862 views • 44 slides
Hierarchical)&)Spectral)clustering) Lecture)13) David&Sontag&

Hierarchical)&)Spectral)clustering) Lecture)13) David&Sontag&

Hierarchical)&)Spectral)clustering) Lecture)13) David&Sontag& New&York&University& Slides adapted from Luke Zettlemoyer, Vibhav Gogate, Carlos Guestrin, Andrew Moore, Dan Klein Agglomerative Clustering Agglomerative

496 views • 25 slides
Tropospheric Water Vapor Variability and Linkage to Tropospheric Water Vapor Variability and

Tropospheric Water Vapor Variability and Linkage to Tropospheric Water Vapor Variability and

Tropospheric Water Vapor Variability and Linkage to Tropospheric Water Vapor Variability and Linkage to Large- -Scale Circulation Scale Circulation Large Hui Su and Jonathan H. Jiang Hui Su and Jonathan H. Jiang The AIRS and MLS Teams The

297 views • 12 slides
B Street / Broadway Piers, Downtown Anchorage, and Switzer Creek TMDLs Public Workshop &

B Street / Broadway Piers, Downtown Anchorage, and Switzer Creek TMDLs Public Workshop &

B Street / Broadway Piers, Downtown Anchorage, and Switzer Creek TMDLs Public Workshop & CEQA Scoping Meeting April 21, 2003 Presented by Brennan Ott Workshop Outline Introduction to Downtown Anchorage, Switzer Creek, and B Street /

347 views • 20 slides
Shape Co-analysis and constrained clustering Daniel Cohen-Or Tel-Aviv University 1 High-level

Shape Co-analysis and constrained clustering Daniel Cohen-Or Tel-Aviv University 1 High-level

Shape Co-analysis and constrained clustering Daniel Cohen-Or Tel-Aviv University 1 High-level Shape analysis [Mehra et al. 08] [Fu et al. 08] [Kalograkis et al. 10] Shape abstraction Upright orientation Learning segmentation [Kim et al.

1.03k views • 81 slides
Introduction to Microarray Data Analysis and Gene Networks Lecture 5 Alvis Brazma European

Introduction to Microarray Data Analysis and Gene Networks Lecture 5 Alvis Brazma European

Introduction to Microarray Data Analysis and Gene Networks Lecture 5 Alvis Brazma European Bioinformatics Institute Lecture 5 Clustering Hierarchical K-means A few minutes about representing experimental designs

558 views • 21 slides
, Phase 1: Purpose and Need Pre-Solicitation Informational Meeting DEPARTMENT OF TRANSPORTATION

, Phase 1: Purpose and Need Pre-Solicitation Informational Meeting DEPARTMENT OF TRANSPORTATION

SP 7001-128 MN 47 and MN 65 Planning and Environmental Linkage (PEL) Study MN 65 MN 47 , Phase 1: Purpose and Need Pre-Solicitation Informational Meeting DEPARTMENT OF TRANSPORTATION Metro District -- Dec 4, 2019 1 ~ SP 7001-128 TH 47

602 views • 30 slides
Marker Based Infinitesimal Model for Quantitative Trait Analysis Shizhong Xu Department of

Marker Based Infinitesimal Model for Quantitative Trait Analysis Shizhong Xu Department of

Marker Based Infinitesimal Model for Quantitative Trait Analysis Shizhong Xu Department of Botany and Plant Sciences University of California Riverside, CA 92521 Outline Quantitative trait and the infinitesimal model Infinitesimal

749 views • 48 slides
Oblivious Coopetitive Analytics Using Hardware Enclaves Ankur Dave , Chester Leung, Raluca Ada

Oblivious Coopetitive Analytics Using Hardware Enclaves Ankur Dave , Chester Leung, Raluca Ada

Oblivious Coopetitive Analytics Using Hardware Enclaves Ankur Dave , Chester Leung, Raluca Ada Popa, Joseph E. Gonzalez, Ion Stoica (UC Berkeley) EuroSys 2020 April 28, 2020 The need for coopetitive analytics Analytics can extract value

733 views • 20 slides
Shado w price sensiti v it y anal y sis SU P P LY C H AIN AN ALYTIC S IN P YTH ON Aaren St u

Shado w price sensiti v it y anal y sis SU P P LY C H AIN AN ALYTIC S IN P YTH ON Aaren St u

Shado w price sensiti v it y anal y sis SU P P LY C H AIN AN ALYTIC S IN P YTH ON Aaren St u bber eld S u ppl y Chain Anal y tics Mgr . Define shado w price Modeling in iss u es : Inp u t for model constraints are o en estimates Will

744 views • 51 slides

More recommend