clustering unsupervised learning
play

Clustering / Unsupervised Learning The target features are not given - PowerPoint PPT Presentation

Nov 13, 2023 •286 likes •608 views

Clustering / Unsupervised Learning The target features are not given in the training examples The aim is to construct a natural classification that can be used to predict features of the data. D. Poole and A. Mackworth 2019 c Artificial

  1. Clustering / Unsupervised Learning The target features are not given in the training examples The aim is to construct a natural classification that can be used to predict features of the data. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 1 / 19

  2. Clustering / Unsupervised Learning The target features are not given in the training examples The aim is to construct a natural classification that can be used to predict features of the data. The examples are partitioned in into clusters or classes. Each class predicts feature values for the examples in the class. ◮ In hard clustering each example is placed definitively in a class. ◮ In soft clustering each example has a probability distribution over its class. Each clustering has a prediction error on the examples. The best clustering is the one that minimizes the error. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 1 / 19

  3. k -means algorithm The k -means algorithm is used for hard clustering. Inputs: training examples the number of classes, k Outputs: a prediction of a value for each feature for each class an assignment of examples to classes � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 2 / 19

  4. k -means algorithm formalized E is the set of all examples the input features are X 1 , . . . , X n X j ( e ) is the value of feature X j for example e . there is a class for each integer i ∈ { 1 , . . . , k } . � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 3 / 19

  5. k -means algorithm formalized E is the set of all examples the input features are X 1 , . . . , X n X j ( e ) is the value of feature X j for example e . there is a class for each integer i ∈ { 1 , . . . , k } . The k -means algorithm outputs function class : E → { 1 , . . . , k } . class ( e ) = i means e is in class i . prediction � X j ( i ) for each feature X j and class i . � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 3 / 19

  6. k -means algorithm formalized E is the set of all examples the input features are X 1 , . . . , X n X j ( e ) is the value of feature X j for example e . there is a class for each integer i ∈ { 1 , . . . , k } . The k -means algorithm outputs function class : E → { 1 , . . . , k } . class ( e ) = i means e is in class i . prediction � X j ( i ) for each feature X j and class i . The sum-of-squares error for class and � X j ( i ) is � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 3 / 19

  7. k -means algorithm formalized E is the set of all examples the input features are X 1 , . . . , X n X j ( e ) is the value of feature X j for example e . there is a class for each integer i ∈ { 1 , . . . , k } . The k -means algorithm outputs function class : E → { 1 , . . . , k } . class ( e ) = i means e is in class i . prediction � X j ( i ) for each feature X j and class i . The sum-of-squares error for class and � X j ( i ) is � � 2 n � � � X j ( class ( e )) − X j ( e ) . e ∈ E j =1 Aim: find class and prediction function that minimize sum-of-squares error. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 3 / 19

  8. Minimizing the error The sum-of-squares error for class and � X j ( i ) is � � 2 � � n � X j ( class ( e )) − X j ( e ) . j =1 e ∈ E Given class , the � X j that minimizes the sum-of-squares error is � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 4 / 19

  9. Minimizing the error The sum-of-squares error for class and � X j ( i ) is � � 2 � � n � X j ( class ( e )) − X j ( e ) . j =1 e ∈ E Given class , the � X j that minimizes the sum-of-squares error is the mean value of X j for that class. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 4 / 19

  10. Minimizing the error The sum-of-squares error for class and � X j ( i ) is � � 2 � � n � X j ( class ( e )) − X j ( e ) . j =1 e ∈ E Given class , the � X j that minimizes the sum-of-squares error is the mean value of X j for that class. Given � X j for each j , each example can be assigned to the class that � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 4 / 19

  11. Minimizing the error The sum-of-squares error for class and � X j ( i ) is � � 2 � � n � X j ( class ( e )) − X j ( e ) . j =1 e ∈ E Given class , the � X j that minimizes the sum-of-squares error is the mean value of X j for that class. Given � X j for each j , each example can be assigned to the class that minimizes the error for that example. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 4 / 19

  12. k -means algorithm Initially, randomly assign the examples to the classes. Repeat the following two steps: For each class i and feature X j , � e : class ( e )= i X j ( e ) � X j ( i ) ← |{ e : class ( e ) = i }| , For each example e , assign e to the class i that minimizes � � 2 � n � X j ( i ) − X j ( e ) . j =1 until the second step does not change the assignment of any example. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 5 / 19

  13. k -means algorithm Sufficient statistics: cc [ c ] is the number of examples in class c , fs [ j , c ] is the sum of the values for X j ( e ) for examples in class c . then define pn ( j , c ), current estimate of � X j ( c ) pn ( j , c ) = � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 6 / 19

  14. k -means algorithm Sufficient statistics: cc [ c ] is the number of examples in class c , fs [ j , c ] is the sum of the values for X j ( e ) for examples in class c . then define pn ( j , c ), current estimate of � X j ( c ) pn ( j , c ) = fs [ j , c ] / cc [ c ] � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 6 / 19

  15. k -means algorithm Sufficient statistics: cc [ c ] is the number of examples in class c , fs [ j , c ] is the sum of the values for X j ( e ) for examples in class c . then define pn ( j , c ), current estimate of � X j ( c ) pn ( j , c ) = fs [ j , c ] / cc [ c ] class ( e ) = � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 6 / 19

  16. k -means algorithm Sufficient statistics: cc [ c ] is the number of examples in class c , fs [ j , c ] is the sum of the values for X j ( e ) for examples in class c . then define pn ( j , c ), current estimate of � X j ( c ) pn ( j , c ) = fs [ j , c ] / cc [ c ] n � ( pn ( j , c ) − X j ( e )) 2 class ( e ) = arg min c j =1 These can be updated in one pass through the training data. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 6 / 19

  17. 1: procedure k-means ( Xs , Es , k ) Initialize fs and cc randomly (based on data) 2: def pn ( j , c ) = fs [ j , c ] / cc [ c ] 3: � n j =1 ( pn ( j , c ) − X j ( e )) 2 def class ( e ) = arg min c 4: repeat 5: fsn and ccn initialized to be all zero 6: for each example e ∈ Es do 7: c := class ( e ) 8: ccn [ c ]+ = 1 9: for each feature X j ∈ Xs do 10: fsn [ j , c ]+ = X j ( e ) 11: stable := ( fsn = fs ) and ( ccn = cc ) 12: fs := fsn 13: cc := ccn 14: until stable 15: return class , pn 16: � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 7 / 19

  18. Example Data 10 8 6 4 2 0 0 2 4 6 8 10 � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 8 / 19

  19. Random Assignment to Classes 10 8 6 4 2 0 0 2 4 6 8 10 � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 9 / 19

  20. Assign Each Example to Closest Mean 10 8 6 4 2 0 0 2 4 6 8 10 � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 10 / 19

  21. Ressign Each Example to Closest Mean 10 8 6 4 2 0 0 2 4 6 8 10 � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 11 / 19

  22. Properties of k -means An assignment of examples to classes is stable if running both the M step and the E step does not change the assignment. This algorithm will eventually converge to a stable local minimum. Any permutation of the labels of a stable assignment is also a stable assignment. It is not guaranteed to converge to a global minimum. It is sensitive to the relative scale of the dimensions. Increasing k can always decrease error until k is the number of different examples. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 12 / 19

  23. EM Algorithm Used for soft clustering — examples are probabilistically in classes. k -valued random variable C Model Data Probabilities ➪ X 1 X 2 X 3 X 4 P ( C ) t f t t P ( X 1 | C ) C f t t f P ( X 2 | C ) f f t t P ( X 3 | C ) X 1 X 2 X 3 X 4 · · · P ( X 4 | C ) � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 13 / 19

  24. EM Algorithm M-step X 1 X 2 X 3 X 4 C count . . . . . . P ( C ) . . . . . . . . . . . . P ( X 1 | C ) t f t t 1 0.4 P ( X 2 | C ) t f t t 2 0.1 P ( X 3 | C ) t f t t 3 0.5 P ( X 4 | C ) . . . . . . . . . . . . . . . . . . E-step � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 14 / 19

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised learning: X - y pairs, f(x) function approximation Unsupervised learning: only X, no y Exploring the space of X measurements,

440 views • 14 slides
Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no

Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no

Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no output classifications (labels) l Clustering is an important type of unsupervised learning PCA was another type of unsupervised learning l The

858 views • 38 slides
Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning no labels/output, only x/input Clustering Group similar points together Machine learning taxonomy Supervised Semi-Supervised Unsupervised

366 views • 15 slides
Lecture 11 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Lecture 11 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Unsupervised Machine Learning and Data Mining DS 5230 / DS 4420 - Fall 2018 Lecture 11 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels for training) Group data into similar classes that Maximize

833 views • 26 slides
Lecture 10 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Lecture 10 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Unsupervised Machine Learning and Data Mining DS 5230 / DS 4420 - Fall 2018 Lecture 10 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels for training) Group data into similar classes that Maximize

1.29k views • 69 slides
Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of

Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of

UNSUPERVISED LEARNING IN R Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of clustering finding groups of homogeneous items Next up, dimensionality reduction Find structure in features

904 views • 29 slides
3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature

3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature

3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature descriptors LAMBDA Workshop on Retrieval and Shape analysis Alexandros Georgiou M.Sc. in Algorithms, Logic and Discrete Mathematics 1/12 Problem

570 views • 53 slides
Clustering ! Unsupervised learning: learning without a teacher ! Only features and no labels

Clustering ! Unsupervised learning: learning without a teacher ! Only features and no labels

So far in the course Supervised learning: learning with a teacher ! You had training data which was (feature, label) pairs and the goal was to learn a mapping from features to labels Clustering ! Unsupervised learning: learning without a

164 views • 12 slides
9.54 Class 13 Unsupervised learning Clustering Shimon Ullman + Tomaso Poggio Danny Harari +

9.54 Class 13 Unsupervised learning Clustering Shimon Ullman + Tomaso Poggio Danny Harari +

9.54 Class 13 Unsupervised learning Clustering Shimon Ullman + Tomaso Poggio Danny Harari + Daneil Zysman + Darren Seibert Outline Introduction to clustering K-means Bag of words (dictionary learning) Hierarchical clustering

895 views • 54 slides
Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering

Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering

Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering and nearest neighbour classifiers. Justin Pearson 1 2020 1 http://user.it.uu.se/~justin/Teaching/MachineLearning/index.html 1 / 27 Supervised

813 views • 27 slides
UNSUPERVISED LEARNING AND CLUSTERING Jeff Robble, Brian Renzenbrink, Doug Roberts Unsupervised

UNSUPERVISED LEARNING AND CLUSTERING Jeff Robble, Brian Renzenbrink, Doug Roberts Unsupervised

UNSUPERVISED LEARNING AND CLUSTERING Jeff Robble, Brian Renzenbrink, Doug Roberts Unsupervised Procedures A procedure that uses unlabeled data in its classification process. Why would we use these? Collecting and labeling large data sets can

544 views • 28 slides
Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat Clustering 7.3 Embedding into Vector Space for Visualization 7.4 Applications Clustering: unsupervised grouping (partitioning) of objects into

763 views • 49 slides
Machine Learning and Data Mining Clustering (adapted from) Prof. Alexander Ihler Unsupervised

Machine Learning and Data Mining Clustering (adapted from) Prof. Alexander Ihler Unsupervised

+ Machine Learning and Data Mining Clustering (adapted from) Prof. Alexander Ihler Unsupervised learning Supervised learning Predict target value (y) given features (x) Unsupervised learning Understand patterns of

559 views • 27 slides
Learning From Data Lecture 19 A Peek At Unsupervised Learning k -Means Clustering Probability

Learning From Data Lecture 19 A Peek At Unsupervised Learning k -Means Clustering Probability

Learning From Data Lecture 19 A Peek At Unsupervised Learning k -Means Clustering Probability Density Estimation Gaussian Mixture Models M. Magdon-Ismail CSCI 4100/6100 recap: Radial Basis Functions Nonparametric RBF Parametric k -RBF-Network

253 views • 23 slides
Machine Learning 2 DS 4420 - Spring 2020 Clustering I Byron C. Wallace Unsupervised learning

Machine Learning 2 DS 4420 - Spring 2020 Clustering I Byron C. Wallace Unsupervised learning

Machine Learning 2 DS 4420 - Spring 2020 Clustering I Byron C. Wallace Unsupervised learning So far we have reviewed some fundamentals, discussed Maximum Likelihood Estimation (MLE) for probabilistic models, and neural networks/backprop

931 views • 75 slides
Machine Learning Fall 2017 Unsupervised Learning (Clustering: k -means, EM, mixture models)

Machine Learning Fall 2017 Unsupervised Learning (Clustering: k -means, EM, mixture models)

Machine Learning Fall 2017 Unsupervised Learning (Clustering: k -means, EM, mixture models) Professor Liang Huang (Chaps. 15-16 of CIML) Roadmap CIML Chaps. 3, 4,5,7,11,17 ,18 so far: (large-margin) supervised learning online learning:

815 views • 54 slides
Positive Logic Is 2-Exptime Hard Aleksy Schubert Pawe Urzyczyn Daria Walukiewicz-Chrzszcz

Positive Logic Is 2-Exptime Hard Aleksy Schubert Pawe Urzyczyn Daria Walukiewicz-Chrzszcz

Positive Logic Is 2-Exptime Hard Aleksy Schubert Pawe Urzyczyn Daria Walukiewicz-Chrzszcz University of Warsaw TYPES, Toulouse, April 26, 2013 Work in Progress. . . Positive Logic Is 2-UExptime Hard Aleksy Schubert Pawe Urzyczyn Daria

877 views • 70 slides
Monomial Bases for NBC Complexes Jason I. Brown Department of Mathematics and Statistics

Monomial Bases for NBC Complexes Jason I. Brown Department of Mathematics and Statistics

Monomial Bases for NBC Complexes Jason I. Brown Department of Mathematics and Statistics Dalhousie University Halifax, NS B3H 3J5, CANADA Bruce E. Sagan Department of Mathematics Michigan State University East Lansing, MI 48824-1027, USA

328 views • 12 slides
Constructing English Reading Courseware Masao Utiyama (NICT) Midori Tanimura (Kinki Univ.)

Constructing English Reading Courseware Masao Utiyama (NICT) Midori Tanimura (Kinki Univ.)

Constructing English Reading Courseware Masao Utiyama (NICT) Midori Tanimura (Kinki Univ.) Hitoshi Isahara (NICT) Contents Goal and motivation Courseware constructed Construction algorithm Experiment Conclustion 1 Goal

432 views • 19 slides
Local, Unconstrained Function Optimization COMPSCI 527 Computer Vision COMPSCI 527

Local, Unconstrained Function Optimization COMPSCI 527 Computer Vision COMPSCI 527

Local, Unconstrained Function Optimization COMPSCI 527 Computer Vision COMPSCI 527 Computer Vision Local, Unconstrained Function Optimization 1 / 27 Outline 1 Gradient, Hessian, and Convexity 2 A Local, Unconstrained Optimization

455 views • 12 slides
A motion planner for nonholonomic mobile robots Miguel Vargas Material taken form: J. P.

A motion planner for nonholonomic mobile robots Miguel Vargas Material taken form: J. P.

A motion planner for nonholonomic mobile robots Miguel Vargas Material taken form: J. P. Laumond, P. E. Jacobs, M. Taix, R. M. Murray. A motion planner for nonholonomic mobile robots. IEEE Transactions on Robotics and Automation. Vol. 10-5.

186 views • 18 slides
k -means++ seeding Have seen that the k -means algorithm can output arbitrarily poor solutions, if

k -means++ seeding Have seen that the k -means algorithm can output arbitrarily poor solutions, if

k -means++ seeding Have seen that the k -means algorithm can output arbitrarily poor solutions, if started with a bad set of initial centroids k -means++ is a simple, probabilistic algorithm to compute initial centroids These centroids are

745 views • 8 slides
Notes on Support Vector Machines, COMP24111 Tingting Mu tingtingmu@manchester.ac.uk School of

Notes on Support Vector Machines, COMP24111 Tingting Mu tingtingmu@manchester.ac.uk School of

Notes on Support Vector Machines, COMP24111 Tingting Mu tingtingmu@manchester.ac.uk School of Computer Science University of Manchester Manchester M13 9PL, UK Editor: NA We start from introducing the notations to be used in this notes. We

308 views • 13 slides
Self-Advocacy Coffee Break May 15 th , 3:00 pm EST Coffee Break Agenda Introduction In the

Self-Advocacy Coffee Break May 15 th , 3:00 pm EST Coffee Break Agenda Introduction In the

Self-Advocacy Coffee Break May 15 th , 3:00 pm EST Coffee Break Agenda Introduction In the large group we will be discussing the question What is your dream job and why? Word of the week Self-Advocacy Exit question,

375 views • 6 slides

More recommend