clustering algorithms
play

Clustering Algorithms Dalya Baron (Tel Aviv University) XXX Winter - PowerPoint PPT Presentation

Sep 05, 2022 •267 likes •815 views

Clustering Algorithms Dalya Baron (Tel Aviv University) XXX Winter School, November 2018 Clustering Feature 2 Feature 1 Clustering cluster #1 Feature 2 cluster #2 Feature 1 Clustering Why should we look for clusters? cluster #1 Feature

  1. Clustering Algorithms Dalya Baron (Tel Aviv University) XXX Winter School, November 2018

  2. Clustering Feature 2 Feature 1

  3. Clustering cluster #1 Feature 2 cluster #2 Feature 1

  4. Clustering Why should we look for clusters? cluster #1 Feature 2 cluster #2 Feature 1

  5. Clustering

  6. K-means Input: measured features, and the number of clusters, k . The algorithm will classify all the objects in the sample into k clusters. Feature 2 Feature 1

  7. K-means The algorithm places randomly k points that represent the centroids of the clusters. (I) The algorithm performs several iterations, in each of them: (II) The algorithm associates each object with a single cluster, according to its distance from the cluster centroid. (III) The algorithm recalculates the cluster centroid according to the objects that are associated with it. Feature 2 Feature 1

  8. K-means The algorithm places randomly k points that represent the centroids of the clusters. (I) The algorithm performs several iterations, in each of them: (II) The algorithm associates each object with a single cluster, according to its distance from the cluster centroid. (III) The algorithm recalculates the cluster centroid according to the objects that are associated with it. Feature 2 Two centroids are randomly placed Feature 1

  9. K-means The algorithm places randomly k points that represent the centroids of the clusters. (I) The algorithm performs several iterations, in each of them: (II) The algorithm associates each object with a single cluster, according to its distance from the cluster centroid. (III) The algorithm recalculates the cluster centroid according to the objects that are associated with it. Feature 2 The objects are associated to the closest cluster centroid (Euclidean distance). Feature 1

  10. K-means The algorithm places randomly k points that represent the centroids of the clusters. (I) The algorithm performs several iterations, in each of them: (II) The algorithm associates each object with a single cluster, according to its distance from the cluster centroid. (III) The algorithm recalculates the cluster centroid according to the objects that are associated with it. Feature 2 New cluster centroids are computed using the average location of the cluster members. Feature 1

  11. K-means The algorithm places randomly k points that represent the centroids of the clusters. (I) The algorithm performs several iterations, in each of them: (II) The algorithm associates each object with a single cluster, according to its distance from the cluster centroid. (III) The algorithm recalculates the cluster centroid according to the objects that are associated with it. Feature 2 The objects are associated to the closest cluster centroid (Euclidean distance). Feature 1

  12. K-means The algorithm places randomly k points that represent the centroids of the clusters. (I) The algorithm performs several iterations, in each of them: (II) The algorithm associates each object with a single cluster, according to its distance from the cluster centroid. (III) The algorithm recalculates the cluster centroid according to the objects that are associated with it. Feature 2 The process stops when the objects that are associated with a given class do not change. Feature 1

  13. The anatomy of K-means cluster Internal choices and/or internal cost function: centroids (I) Initial centroids are randomly selected from the set of examples. (II) The global cost function that is minimized by K-means: cluster Euclidean members distance

  14. The anatomy of K-means cluster Internal choices and/or internal cost function: centroids (I) Initial centroids are randomly selected from the set of examples. (II) The global cost function that is minimized by K-means: cluster Euclidean members distance k=3, and two di ff erent random placements of centroids

  15. The anatomy of K-means Input dataset: a list of objects with measured features. For which datasets should we use K-means? Feature 2 Feature 2 Feature 1 Feature 1

  17. The anatomy of K-means Input dataset: a list of objects with measured features. What happens when we have an outlier in the dataset? Feature 2 outlier! Feature 1

  18. The anatomy of K-means Input dataset: a list of objects with measured features. What happens when we have an outlier in the dataset? Feature 2 outlier! Feature 1

  19. The anatomy of K-means Input dataset: a list of objects with measured features. What happens when the features have di ff erent physical units? input dataset K-means output

  20. The anatomy of K-means Input dataset: a list of objects with measured features. What happens when the features have di ff erent physical units? How can we avoid this? input dataset K-means output

  21. The anatomy of K-means Hyper-parameters: the number of clusters, k. Can we find the optimal k using the cost function? k=2 k=3 k=5

  22. The anatomy of K-means Hyper-parameters: the number of clusters, k. Can we find the optimal k using the cost function? k=2 k=3 k=5 Minimal cost function Elbow Number of clusters

  23. Questions?

  24. Hierarchal Clustering or, how to visualize complicated similarity measures Correa-Gallego+ 2016

  25. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Feature 2 Feature 1

  26. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 Feature 1

  27. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 distance Feature 1 Dendrogram

  28. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 distance Feature 1 Dendrogram

  29. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 distance Feature 1 Dendrogram

  30. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 distance Feature 1 Dendrogram

  31. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 distance Feature 1 Dendrogram

  32. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 distance Feature 1 Dendrogram

  33. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 distance Feature 1 Dendrogram

  34. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. The process stops when all the objects are merged into a single cluster Feature 2 distance Feature 1 Dendrogram

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Week 7 Video 3 Advanced Clustering Algorithms Today Multiple advanced algorithms for

Week 7 Video 3 Advanced Clustering Algorithms Today Multiple advanced algorithms for

Week 7 Video 3 Advanced Clustering Algorithms Today Multiple advanced algorithms for clustering Gaussian Mixture Models Often called EM-based clustering Kind of a misnomer in my opinion What distinguishes this algorithm

681 views • 21 slides
Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean

Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean

Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean clustering, hierarchical clustering Adaptive learning (online learning) CL, FSCL, RPCL Gaussian Mixture Models (GMM)

805 views • 34 slides
Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns in data, or grouping similar groups of data-points together into clusters . Clustering algorithms for numeric data: Lloyds K-means, EM

688 views • 19 slides
Clustering Algorithms Johannes Bl omer WS 2015/16 1 / 20 Introduction Clustering techniques

Clustering Algorithms Johannes Bl omer WS 2015/16 1 / 20 Introduction Clustering techniques

Clustering Algorithms Johannes Bl omer WS 2015/16 1 / 20 Introduction Clustering techniques for data management and analysis that classify/group given set of objects into categories/subgroups or clusters 2 / 20 Introduction Clustering

1.01k views • 51 slides
INF4820: Algorithms for AI and NLP Clustering Milen Kouylekov & Stephan Oepen Language

INF4820: Algorithms for AI and NLP Clustering Milen Kouylekov & Stephan Oepen Language

INF4820: Algorithms for AI and NLP Clustering Milen Kouylekov & Stephan Oepen Language Technology Group University of Oslo Oct. 2, 2014 Agenda Yesterday Flat clustering k -Means Today Bottom-up hierarchical clustering.

976 views • 44 slides
Web Information Retrieval Lecture 15 Clustering Todays Topic: Clustering Document

Web Information Retrieval Lecture 15 Clustering Todays Topic: Clustering Document

Web Information Retrieval Lecture 15 Clustering Todays Topic: Clustering Document clustering Motivations Document representations Success criteria Clustering algorithms Partitional Hierarchical Ch. 16 What is

766 views • 61 slides
Outline Linkage-based Clustering Motivation Definitions Clustering with Semantic

Outline Linkage-based Clustering Motivation Definitions Clustering with Semantic

Outline Linkage-based Clustering Motivation Definitions Clustering with Semantic Links Applications Algorithms SimRank Yin X., Han J. and Yu P. S., 2006, LinkClus: Efficient ReCoM Clustering via Heterogeneous

221 views • 8 slides
Streaming algorithms for k -center clustering with outliers and with anonymity Richard Matthew

Streaming algorithms for k -center clustering with outliers and with anonymity Richard Matthew

Streaming algorithms for k -center clustering with outliers and with anonymity Richard Matthew McCutchen and Samir Khuller University of Maryland {rmccutch,samir}@cs.umd.edu k -center clustering problem Input: n points in an arbitrary

751 views • 63 slides
3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature

3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature

3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature descriptors LAMBDA Workshop on Retrieval and Shape analysis Alexandros Georgiou M.Sc. in Algorithms, Logic and Discrete Mathematics 1/12 Problem

570 views • 53 slides
Improved Clustering Algorithms for the Random Cluster Graph Model Ron Shamir Dekel Tsur Tel

Improved Clustering Algorithms for the Random Cluster Graph Model Ron Shamir Dekel Tsur Tel

Improved Clustering Algorithms for the Random Cluster Graph Model Ron Shamir Dekel Tsur Tel Aviv University 1/18 The Clustering Problem Input: A graph G . (edges in G represent similarity between the vertices) Output: A partition of the

485 views • 34 slides
Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering? Aykut Erdem May 2016 Hacettepe University Last time K-Means An iterative clustering algorithm - Initialize: Pick K random points as cluster

932 views • 64 slides
Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity

772 views • 40 slides
Clustering Reference:http://www.elet.polimi.it/upload/matteucc/Clustering/tutorial_html/ Dr Ahmed

Clustering Reference:http://www.elet.polimi.it/upload/matteucc/Clustering/tutorial_html/ Dr Ahmed

Clustering Reference:http://www.elet.polimi.it/upload/matteucc/Clustering/tutorial_html/ Dr Ahmed Rafea Outline Introduction Clustering Algorithms K-means Fuzzy C-means Hierarchical clustering Introduction(1)

394 views • 23 slides
INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Erik Velldal &

INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Erik Velldal &

INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Erik Velldal & Stephan Oepen Language Technology Group (LTG) September 23, 2015 Agenda Last week Supervised vs unsupervised learning. Vectors space

1.15k views • 54 slides
INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Erik Velldal &

INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Erik Velldal &

INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Erik Velldal & Stephan Oepen Language Technology Group (LTG) September 23, 2015 Agenda Last week Supervised vs unsupervised learning. Vectors space

496 views • 14 slides
INF4820 Algorithms for AI and NLP Hierarchical Clustering Erik Velldal & Stephan

INF4820 Algorithms for AI and NLP Hierarchical Clustering Erik Velldal & Stephan

INF4820 Algorithms for AI and NLP Hierarchical Clustering Erik Velldal & Stephan Oepen Language Technology Group (LTG) October 7, 2015 Agenda Last week Evaluation of classifiers Machine learning for class discovery:

892 views • 38 slides
LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

DATA MINING LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm Clustering Evaluation What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or

1.39k views • 110 slides
Projects Chandrasekar, Arun Kumar, Group 17 Nearly all group have submitted a proposal

Projects Chandrasekar, Arun Kumar, Group 17 Nearly all group have submitted a proposal

Projects Chandrasekar, Arun Kumar, Group 17 Nearly all group have submitted a proposal May 21: Each person gives one slide, 15 min/group. First principles vs Data driven Small data Big data to train Data High reliance on domain

823 views • 44 slides
Machine Learning Lecture Notes on Clustering (II) 2017-2018 Davide Eynard davide.eynard@usi.ch

Machine Learning Lecture Notes on Clustering (II) 2017-2018 Davide Eynard davide.eynard@usi.ch

Machine Learning Lecture Notes on Clustering (II) 2017-2018 Davide Eynard davide.eynard@usi.ch Institute of Computational Science Universit` a della Svizzera italiana p. 1/39 Todays Outline K-Means limits K-Means extensions:

701 views • 39 slides
Clustering Duen Horng (Polo) Chau Georgia Tech Partly based on materials by Professors

Clustering Duen Horng (Polo) Chau Georgia Tech Partly based on materials by Professors

CSE 6242 / CX 4242 Clustering Duen Horng (Polo) Chau Georgia Tech Partly based on materials by Professors Guy Lebanon, Jeffrey Heer, John Stasko, Christos Faloutsos, Le Song Clustering in Google Image Search How would you build this?

418 views • 14 slides
Data Clustering: A Very Brief Overview Serhan Cosar INRIA-STARS Outline Introduction

Data Clustering: A Very Brief Overview Serhan Cosar INRIA-STARS Outline Introduction

Data Clustering: A Very Brief Overview Serhan Cosar INRIA-STARS Outline Introduction Five Ws of Clustering Who, What, When, Where, Why? One H of Clustering How? Algorithms Conclusion Introduction Unsupervised Learning:

475 views • 22 slides
EM and GMM Jia-Bin Huang Virginia Tech Spring 2019 ECE-5424G / CS-5824 Administrative HW 3

EM and GMM Jia-Bin Huang Virginia Tech Spring 2019 ECE-5424G / CS-5824 Administrative HW 3

EM and GMM Jia-Bin Huang Virginia Tech Spring 2019 ECE-5424G / CS-5824 Administrative HW 3 due March 27. Final project discussion: Link Final exam date/time Exam Section: 14M

1.09k views • 51 slides
Unsupervised Learning and Clustering Selim Aksoy Department of Computer Engineering Bilkent

Unsupervised Learning and Clustering Selim Aksoy Department of Computer Engineering Bilkent

Unsupervised Learning and Clustering Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2008 CS 551, Spring 2008 2008, Selim Aksoy (Bilkent University) c 1 / 50 Introduction

535 views • 50 slides
Design of Experiments The linear regression model relates the expected value of a dependent

Design of Experiments The linear regression model relates the expected value of a dependent

ST 430/514 Introduction to Regression Analysis/Statistics for Management and the Social Sciences II Design of Experiments The linear regression model relates the expected value of a dependent variable, or response , E ( Y ), to some independent

306 views • 7 slides

More recommend