clustering unsupervised learning
play

Clustering / Unsupervised Learning The target features are not given - PowerPoint PPT Presentation

Feb 21, 2023 •539 likes •741 views

Clustering / Unsupervised Learning The target features are not given in the training examples The aim is to construct a natural classification that can be used to predict features of the data. D. Poole and A. Mackworth 2010 c Artificial

  1. Clustering / Unsupervised Learning The target features are not given in the training examples The aim is to construct a natural classification that can be used to predict features of the data. � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 1

  2. Clustering / Unsupervised Learning The target features are not given in the training examples The aim is to construct a natural classification that can be used to predict features of the data. The examples are partitioned in into clusters or classes. Each class predicts feature values for the examples in the class. ◮ In hard clustering each example is placed definitively in a class. ◮ In soft clustering each example has a probability distribution over its class. Each clustering has a prediction error on the examples. The best clustering is the one that minimizes the error. � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 2

  3. k -means algorithm The k -means algorithm is used for hard clustering. Inputs: training examples the number of classes, k Outputs: a prediction of a value for each feature for each class an assignment of examples to classes � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 3

  4. k -means algorithm formalized E is the set of all examples the input features are X 1 , . . . , X n val ( e , X j ) is the value of feature X j for example e . there is a class for each integer i ∈ { 1 , . . . , k } . The k -means algorithm outputs a function class : E → { 1 , . . . , k } . class ( e ) = i means e is in class i . a pval function where pval ( i , X j ) is the prediction for each example in class i for feature X j . The sum-of-squares error for class and pval is n � � ( pval ( class ( e ) , X j ) − val ( e , X j )) 2 . e ∈ E j =1 Aim: find class and pval that minimize sum-of-squares error. � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 4

  5. Minimizing the error The sum-of-squares error for class and pval is n � � ( pval ( class ( e ) , X j ) − val ( e , X j )) 2 . e ∈ E j =1 Given class , the pval that minimizes the sum-of-squares error is � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 5

  6. Minimizing the error The sum-of-squares error for class and pval is n � � ( pval ( class ( e ) , X j ) − val ( e , X j )) 2 . e ∈ E j =1 Given class , the pval that minimizes the sum-of-squares error is the mean value for that class. Given pval , each example can be assigned to the class that � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 6

  7. Minimizing the error The sum-of-squares error for class and pval is n � � ( pval ( class ( e ) , X j ) − val ( e , X j )) 2 . e ∈ E j =1 Given class , the pval that minimizes the sum-of-squares error is the mean value for that class. Given pval , each example can be assigned to the class that minimizes the error for that example. � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 7

  8. k -means algorithm Initially, randomly assign the examples to the classes. Repeat the following two steps: For each class i and feature X j , � e : class ( e )= i val ( e , X j ) pval ( i , X j ) ← |{ e : class ( e ) = i }| , For each example e , assign e to the class i that minimizes n � ( pval ( i , X j ) − val ( e , X j )) 2 . j =1 until the second step does not change the assignment of any example. � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 8

  9. Example Data 10 8 6 4 2 0 0 2 4 6 8 10 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 9

  10. Random Assignment to Classes 10 8 6 4 2 0 0 2 4 6 8 10 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 10

  11. Assign Each Example to Closest Mean 10 8 6 4 2 0 0 2 4 6 8 10 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 11

  12. Ressign Each Example to Closest Mean 10 8 6 4 2 0 0 2 4 6 8 10 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 12

  13. Properties of k -means An assignment of examples to classes is stable if running both the M step and the E step does not change the assignment. This algorithm will eventually converge to a stable local minimum. Any permutation of the labels of a stable assignment is also a stable assignment. It is not guaranteed to converge to a global minimum. It is sensitive to the relative scale of the dimensions. Increasing k can always decrease error until k is the number of different examples. � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 13

  14. EM Algorithm Used for soft clustering — examples are probabilistically in classes. k -valued random variable C Model Data Probabilities ➪ X 1 X 2 X 3 X 4 P ( C ) t f t t P ( X 1 | C ) C f t t f P ( X 2 | C ) f f t t P ( X 3 | C ) X1 X2 X3 X4 · · · P ( X 4 | C ) � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 14

  15. EM Algorithm M-step X 1 X 2 X 3 X 4 C count . . . . . . P ( C ) . . . . . . . . . . . . P ( X 1 | C ) t f t t 1 0.4 P ( X 2 | C ) t f t t 2 0.1 P ( X 3 | C ) t f t t 3 0.5 P ( X 4 | C ) . . . . . . . . . . . . . . . . . . E-step � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 15

  16. EM Algorithm Overview Repeat the following two steps: E-step give the expected number of data points for the ◮ unobserved variables based on the given probability distribution. M-step infer the (maximum likelihood or maximum ◮ aposteriori probability) probabilities from the data. Start either with made-up data or made-up probabilities. EM will converge to a local maxima. � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 16

  17. Augmented Data — E step Suppose k = 3, and dom ( C ) = { 1 , 2 , 3 } . P ( C = 1 | X 1 = t , X 2 = f , X 3 = t , X 4 = t ) = 0 . 407 P ( C = 2 | X 1 = t , X 2 = f , X 3 = t , X 4 = t ) = 0 . 121 P ( C = 3 | X 1 = t , X 2 = f , X 3 = t , X 4 = t ) = 0 . 472: A [ X 1 , . . . , X 4 , C ] � �� � X 1 X 2 X 3 X 4 C Count . . . . . . X 1 X 2 X 3 X 4 Count . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 40 . 7 t f t t − → t f t t 100 t f t t 2 12 . 1 . . . . . 3 47 . 2 . . . . . t f t t . . . . . . . . . . . . . . . . . . . . . . . � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 17

  18. M step C X 1 X 2 X 3 X 4 C Count . . . . . . . . . . . . . . . . . . X1 X2 X3 X4 1 40 . 7 t f t t − → t f t t 2 12 . 1 3 47 . 2 t f t t . . . . . . . . . . . . . . . . . . � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 18

  19. M step C X 1 X 2 X 3 X 4 C Count . . . . . . . . . . . . . . . . . . X1 X2 X3 X4 1 40 . 7 t f t t − → t f t t 2 12 . 1 3 47 . 2 t f t t . . . . . . . . . . . . . . . . . . � = C = v i Count ( t ) t | P ( C = v i ) = � t Count ( t ) � = C = v i ∧ X k = v j Count ( t ) t | P ( X k = v j | C = v i ) = � = C = v i Count ( t ) t | ...perhaps including pseudo-counts � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 11.1, Page 19

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised learning: X - y pairs, f(x) function approximation Unsupervised learning: only X, no y Exploring the space of X measurements,

440 views • 14 slides
Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no

Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no

Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no output classifications (labels) l Clustering is an important type of unsupervised learning PCA was another type of unsupervised learning l The

858 views • 38 slides
Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning no labels/output, only x/input Clustering Group similar points together Machine learning taxonomy Supervised Semi-Supervised Unsupervised

366 views • 15 slides
Lecture 11 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Lecture 11 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Unsupervised Machine Learning and Data Mining DS 5230 / DS 4420 - Fall 2018 Lecture 11 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels for training) Group data into similar classes that Maximize

833 views • 26 slides
Lecture 10 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Lecture 10 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Unsupervised Machine Learning and Data Mining DS 5230 / DS 4420 - Fall 2018 Lecture 10 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels for training) Group data into similar classes that Maximize

1.29k views • 69 slides
Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of

Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of

UNSUPERVISED LEARNING IN R Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of clustering finding groups of homogeneous items Next up, dimensionality reduction Find structure in features

904 views • 29 slides
3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature

3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature

3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature descriptors LAMBDA Workshop on Retrieval and Shape analysis Alexandros Georgiou M.Sc. in Algorithms, Logic and Discrete Mathematics 1/12 Problem

570 views • 53 slides
Clustering ! Unsupervised learning: learning without a teacher ! Only features and no labels

Clustering ! Unsupervised learning: learning without a teacher ! Only features and no labels

So far in the course Supervised learning: learning with a teacher ! You had training data which was (feature, label) pairs and the goal was to learn a mapping from features to labels Clustering ! Unsupervised learning: learning without a

164 views • 12 slides
9.54 Class 13 Unsupervised learning Clustering Shimon Ullman + Tomaso Poggio Danny Harari +

9.54 Class 13 Unsupervised learning Clustering Shimon Ullman + Tomaso Poggio Danny Harari +

9.54 Class 13 Unsupervised learning Clustering Shimon Ullman + Tomaso Poggio Danny Harari + Daneil Zysman + Darren Seibert Outline Introduction to clustering K-means Bag of words (dictionary learning) Hierarchical clustering

895 views • 54 slides
Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering

Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering

Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering and nearest neighbour classifiers. Justin Pearson 1 2020 1 http://user.it.uu.se/~justin/Teaching/MachineLearning/index.html 1 / 27 Supervised

813 views • 27 slides
UNSUPERVISED LEARNING AND CLUSTERING Jeff Robble, Brian Renzenbrink, Doug Roberts Unsupervised

UNSUPERVISED LEARNING AND CLUSTERING Jeff Robble, Brian Renzenbrink, Doug Roberts Unsupervised

UNSUPERVISED LEARNING AND CLUSTERING Jeff Robble, Brian Renzenbrink, Doug Roberts Unsupervised Procedures A procedure that uses unlabeled data in its classification process. Why would we use these? Collecting and labeling large data sets can

544 views • 28 slides
Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat Clustering 7.3 Embedding into Vector Space for Visualization 7.4 Applications Clustering: unsupervised grouping (partitioning) of objects into

763 views • 49 slides
Machine Learning and Data Mining Clustering (adapted from) Prof. Alexander Ihler Unsupervised

Machine Learning and Data Mining Clustering (adapted from) Prof. Alexander Ihler Unsupervised

+ Machine Learning and Data Mining Clustering (adapted from) Prof. Alexander Ihler Unsupervised learning Supervised learning Predict target value (y) given features (x) Unsupervised learning Understand patterns of

559 views • 27 slides
Learning From Data Lecture 19 A Peek At Unsupervised Learning k -Means Clustering Probability

Learning From Data Lecture 19 A Peek At Unsupervised Learning k -Means Clustering Probability

Learning From Data Lecture 19 A Peek At Unsupervised Learning k -Means Clustering Probability Density Estimation Gaussian Mixture Models M. Magdon-Ismail CSCI 4100/6100 recap: Radial Basis Functions Nonparametric RBF Parametric k -RBF-Network

253 views • 23 slides
Machine Learning 2 DS 4420 - Spring 2020 Clustering I Byron C. Wallace Unsupervised learning

Machine Learning 2 DS 4420 - Spring 2020 Clustering I Byron C. Wallace Unsupervised learning

Machine Learning 2 DS 4420 - Spring 2020 Clustering I Byron C. Wallace Unsupervised learning So far we have reviewed some fundamentals, discussed Maximum Likelihood Estimation (MLE) for probabilistic models, and neural networks/backprop

931 views • 75 slides
Machine Learning Fall 2017 Unsupervised Learning (Clustering: k -means, EM, mixture models)

Machine Learning Fall 2017 Unsupervised Learning (Clustering: k -means, EM, mixture models)

Machine Learning Fall 2017 Unsupervised Learning (Clustering: k -means, EM, mixture models) Professor Liang Huang (Chaps. 15-16 of CIML) Roadmap CIML Chaps. 3, 4,5,7,11,17 ,18 so far: (large-margin) supervised learning online learning:

815 views • 54 slides
BBM406 Fundamentals of Machine Learning Lecture 5: ML Methodology Aykut Erdem // Hacettepe

BBM406 Fundamentals of Machine Learning Lecture 5: ML Methodology Aykut Erdem // Hacettepe

Illustration: detail from The Alchemist Discovering Phosphorus by Joseph Wright (1771) BBM406 Fundamentals of Machine Learning Lecture 5: ML Methodology Aykut Erdem // Hacettepe University // Fall 2019 About class projects This semester

1.42k views • 46 slides
Linear Regression - Estimating Parameters Bernd Schr oder logo1 Bernd Schr oder

Linear Regression - Estimating Parameters Bernd Schr oder logo1 Bernd Schr oder

Coefficients Examples Error Sum of Squares Coefficient of Determination Linear Regression - Estimating Parameters Bernd Schr oder logo1 Bernd Schr oder Louisiana Tech University, College of Engineering and Science Linear Regression -

1.16k views • 102 slides
Error Handling Marco Chiarandini Department of Mathematics & Computer Science University of

Error Handling Marco Chiarandini Department of Mathematics & Computer Science University of

DM560 Introduction to Programming in C++ Error Handling Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark [ Based on slides by Bjarne Stroustrup ] Error Handling Outline 1. Error Handling 2

573 views • 36 slides
CS7015 (Deep Learning) : Lecture 3 Sigmoid Neurons, Gradient Descent, Feedforward Neural Networks,

CS7015 (Deep Learning) : Lecture 3 Sigmoid Neurons, Gradient Descent, Feedforward Neural Networks,

CS7015 (Deep Learning) : Lecture 3 Sigmoid Neurons, Gradient Descent, Feedforward Neural Networks, Representation Power of Feedforward Neural Networks Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute of

869 views • 70 slides
Certifying Non-negativity with Lasserres Hierarchy and Semidefinite Programming Victor Magron ,

Certifying Non-negativity with Lasserres Hierarchy and Semidefinite Programming Victor Magron ,

Certifying Non-negativity with Lasserres Hierarchy and Semidefinite Programming Victor Magron , LAASCNRS 5 March 2019 Faculty of Mechanical Engineering University of Ljubljana Victor Magron Certifying Non-negativity with Lasserres

1.73k views • 85 slides
Lecture 10. Simple linear regression 2020 (1) Using one r.v. to predict another X and Y are

Lecture 10. Simple linear regression 2020 (1) Using one r.v. to predict another X and Y are

Lecture 10. Simple linear regression 2020 (1) Using one r.v. to predict another X and Y are random variables. What is the best linear predictor b 0 + b 1 X of Y ? Prediction error is e = Y ` b 0 ` b 1 X . For the best predictor, there is

523 views • 40 slides
Earthmover resilience & testing in ordered structures Eldar Fischer Omri Ben-Eliezer

Earthmover resilience & testing in ordered structures Eldar Fischer Omri Ben-Eliezer

Earthmover resilience & testing in ordered structures Eldar Fischer Omri Ben-Eliezer Technion Tel-Aviv University Computational Complexity Conference 2018 UCSD, San Diago Property testing (RS96, GGR98) Meta problem: Given property P,

377 views • 34 slides
Software Engineering I cs361 Announcements Office hours canceled on Tuesday Jan 19th

Software Engineering I cs361 Announcements Office hours canceled on Tuesday Jan 19th

Software Engineering I cs361 Announcements Office hours canceled on Tuesday Jan 19th Shanes Tue Office Hours moved to Thurs 11:30-12:30 How to show you are using TDD Sources for Writing Assignment Question for Ciera Japan

505 views • 32 slides

More recommend