machine learning 2
play

Machine Learning 2 DS 4420 - Spring 2018 From clustering to EM - PowerPoint PPT Presentation

Jun 25, 2023 •29 likes •799 views

Machine Learning 2 DS 4420 - Spring 2018 From clustering to EM Byron C. Wallace Clustering Four Types of Clustering 1. Centroid-based (K-means, K-medoids) Notion of Clusters: Voronoi tesselation Four Types of Clustering 2. Density-based

  1. Machine Learning 2 DS 4420 - Spring 2018 From clustering to EM Byron C. Wallace

  2. Clustering

  3. Four Types of Clustering 1. Centroid-based (K-means, K-medoids) Notion of Clusters: Voronoi tesselation

  4. Four Types of Clustering 2. Density-based (DBSCAN, OPTICS) Notion of Clusters: Connected regions of high density

  5. Four Types of Clustering 3. Connectivity-based (Hierarchical) Notion of Clusters: Cut off dendrogram at some depth

  6. Four Types of Clustering 4. Distribution-based (Mixture Models) Notion of Clusters: Distributions on features

  7. Hierarchical Clustering

  8. Dendrogram ( a.k.a. a similarity tree ) Terminal Branch Terminal Branch Root Root Similarity of A and B is Internal Branch Internal Branch represented as height 
 Internal Node Internal Node of lowest shared 
 Leaf Leaf internal node

  9. Dendrogram ( a.k.a. a similarity tree ) Similarity of A and B is D(A,B) represented as height 
 of lowest shared 
 internal node (Bovine: 0.69395, (Spider Monkey: 0.390, (Gibbon:0.36079,(Orang: 0.33636, (Gorilla: 0.17147, 
 (Chimp: 0.19268, Human: 0.11927): 0.08386): 0.06124): 0.15057): 0.54939);

  10. Dendrogram ( a.k.a. a similarity tree ) D(A,B) Natural when measuring 
 genetic similarity, distance 
 to common ancestor (Bovine: 0.69395, (Spider Monkey: 0.390, (Gibbon:0.36079,(Orang: 0.33636, (Gorilla: 0.17147, 
 (Chimp: 0.19268, Human: 0.11927): 0.08386): 0.06124): 0.15057): 0.54939);

  11. Example: Iris data Iris Setosa Iris versicolor Iris virginica https://en.wikipedia.org/wiki/Iris_flower_data_set

  12. Hierarchical Clustering ( Euclidian Distance ) https://en.wikipedia.org/wiki/Iris_flower_data_set

  13. Edit Distance Distance Patty and Selma Change dress color, 1 point Change earring shape, 1 point Change hair part, 1 point D(Patty, Selma) = 3 Distance Marge and Selma Change dress color, 1 point Add earrings, 1 point Decrease height, 1 point Take up smoking, 1 point Lose weight, 1 point D(Marge,Selma) = 5 Can be defined for any set of discrete features

  14. Edit Distance for Strings • Transform string Q into string C , using only Similarity “Peter” and “Piotr”? Substitution , Insertion and Deletion . Substitution 1 Unit • Assume that each of these operators has a Insertion 1 Unit cost associated with it. Deletion 1 Unit • The similarity between two strings can be D ( Peter , Piotr ) is 3 defined as the cost of the cheapest transformation from Q to C. Peter Substitution (i for e) Piter Insertion (o) Pioter Deletion (e) Pedro Piotr Peter Piotr Piero Pyotr Petros Pietro Pierre

  15. Hierarchical Clustering ( Edit Distance ) Pedro (Portuguese) Petros (Greek), Peter (English), Piotr (Polish), Peadar (Irish), Pierre (French), Peder (Danish), Peka (Hawaiian), Pietro (Italian), Piero (Italian Alternative), Petr (Czech), Pyotr (Russian) Cristovao (Portuguese) Christoph (German), Christophe (French), Cristobal (Spanish), Cristoforo (Italian), Kristoffer (Scandinavian), Krystof (Czech), Christopher (English) Miguel (Portuguese) Michalis (Greek), Michael (English), Mick (Irish) Cristovao Pedro Miguel Christoph n Piotr r Petros o Pierre o Peter Peka r Michalis Michael Mick Christopher e Cristobal Cristoforo Kristoffer f r o t a e r r h a o t t e d d p e s y e i a e y P P o d i P e r P t s K s P i r i r C h C

  16. Meaningful Patterns Edit distance yields clustering according to geography Slide from Eamonn Keogh Pedro ( Portuguese/Spanish ) Petros ( Greek ), Peter ( English ), Piotr ( Polish ), Peadar (Irish), Pierre ( French ), Peder ( Danish ), Peka (Hawaiian), Pietro ( Italian ), Piero ( Italian Alternative ), Petr (Czech), Pyotr ( Russian )

  17. Spurious Patterns In general clusterings will only be as meaningful as your distance metric spurious; there is no connection between the two South Georgia & Serbia & St. Helena & U.K. AUSTRALIA ANGUILLA FRANCE NIGER INDIA IRELAND BRAZIL South Sandwich Montenegro Dependencies Islands (Yugoslavia)

  18. Spurious Patterns In general clusterings will only be as meaningful as your distance metric spurious; there is no connection between the two South Georgia & Serbia & St. Helena & U.K. AUSTRALIA ANGUILLA FRANCE NIGER INDIA IRELAND BRAZIL South Sandwich Montenegro Dependencies Islands (Yugoslavia) Former UK colonies No relation

  19. “Correct” Number of Clusters to determine the “correct”

  20. “Correct” Number of Clusters to determine the “correct” Determine number of clusters by looking at distance

  21. Detecting Outliers The single isolated branch is suggestive of a data point that is very different to all others Outlier

  22. Bottom up vs. Top down Bottom-up ( agglomerative): Each item starts as its own cluster; greedily merge

  23. Bottom up vs. Top down Bottom-up ( agglomerative): Each item starts as its own cluster; greedily merge Top-down ( divisive ): Start with one big cluster (all data); recursively split

  24. Distance Matrix We begin with a distance matrix which contains the distances between every pair of objects in our 0 8 8 7 7 database. 0 2 4 4 0 3 3 D( , ) = 8 0 1 D( , ) = 1 0

  25. Bottom-up (Agglomerative Clustering) … merges… … merges… Consider Choose … all possible the best merges… 25

  26. Bottom-up (Agglomerative Clustering) … merges… Consider Choose all possible … the best merges… Consider Choose … all possible the best merges… 25

  27. Bottom-up (Agglomerative Clustering) Consider Choose all possible … the best merges… Consider Choose all possible … the best merges… Consider Choose … all possible the best merges… 25

  28. Bottom-up (Agglomerative Clustering) Consider Choose all possible … … the best merges… merges… Consider Choose all possible … the best … merges… merges… Consider Choose … all possible the best … merges… merges… 25

  29. Bottom-up (Agglomerative Clustering) Can you now implement this? Consider Choose all possible … … the best merges… merges… Consider Choose all possible … the best … merges… merges… Consider Choose … all possible the best … merges… merges… 25

  30. Bottom-up (Agglomerative Clustering) Distances between examples (can calculate using metric) Consider Choose all possible … … the best merges… merges… Consider Choose all possible … the best … merges… merges… Consider Choose … all possible the best … merges… merges… 25

  31. Bottom-up (Agglomerative Clustering) How do we calculate the 
 distance to a cluster? Consider Choose all possible … … the best merges… merges… Consider Choose all possible … the best … merges… merges… Consider Choose … all possible the best … merges… merges… 25

  32. Clustering Criteria Single link: 
 d ( A , B ) = min a ∈ A , b ∈ B d ( a , b ) (Closest point) Complete link: 
 d ( A , B ) = max a ∈ A , b ∈ B d ( a , b ) (Furthest point) 1 X Group average: 
 d ( A , B ) = d ( a , b ) | A || B | (Average distance) a ∈ A , b ∈ B µ X = 1 X Centroid: 
 d ( A , B ) = d ( µ A , µ B ) x | X | (Distance of average) x ∈ X

  33. Hierarchical Clustering Summary + No need to specify number of clusters + Hierarchical structure maps nicely onto 
 human intuition in some domains - Scaling: Time complexity at least O ( n 2 ) 
 in number of examples - Heuristic search method : 
 Local optima are a problem - Interpretation of results is (very) subjective

  34. Hierarchical Clustering Summary + No need to specify number of clusters + Hierarchical structure maps nicely onto 
 human intuition in some domains - Scaling: Time complexity at least O ( n 2 ) 
 in number of examples - Heuristic search method : 
 Local optima are a problem - Interpretation of results is (very) subjective

  35. Hierarchical Clustering Summary + No need to specify number of clusters + Hierarchical structure maps nicely onto 
 human intuition in some domains - Scaling: Time complexity at least O ( n 2 ) 
 in number of examples - Heuristic search method : 
 Local optima are a problem - Interpretation of results is (very) subjective

  36. Hierarchical Clustering Summary + No need to specify number of clusters + Hierarchical structure maps nicely onto 
 human intuition in some domains - Scaling: Time complexity at least O ( n 2 ) 
 in number of examples - Heuristic search method : 
 Local optima are a problem - Interpretation of results is (very) subjective

  37. Hierarchical Clustering Summary + No need to specify number of clusters + Hierarchical structure maps nicely onto 
 human intuition in some domains - Scaling: Time complexity at least O ( n 2 ) 
 in number of examples - Heuristic search method : 
 Local optima are a problem - Interpretation of results is (very) subjective

  38. � � � � Evaluation? 1 1 0.9 0.9 0.8 0.8 0.7 0.7 Random DBSCAN 0.6 0.6 Points 0.5 0.5 y y 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 x x 1 1 0.9 0.9 K-means Complete 0.8 0.8 Link 0.7 0.7 0.6 0.6 0.5 0.5 y y 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 x x

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

An Exercise in An Exercise in Machine Learning Machine Learning

An Exercise in An Exercise in Machine Learning Machine Learning

An Exercise in An Exercise in Machine Learning Machine Learning http://www.cs.iastate.edu/~cs573x/bbsilab.html Machine Learning Software Preparing Data Building Classifiers Interpreting Results Machine Learning Software

496 views • 26 slides
Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by

Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by

Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by which computers can be trained through observation, rather than being explicitly programmed. Basically, a program is given input and adjusts its

556 views • 17 slides
Machine Learning: Study of algorithms that improve their performance P at some task T

Machine Learning: Study of algorithms that improve their performance P at some task T

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 12, 2015 Today: Readings: What is machine learning? The Discipline of ML Decision tree learning Mitchell, Chapter 3

872 views • 29 slides
Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning

Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning Pipeline in Classification Tasks A set of hand-designed audio features are selected

398 views • 26 slides
CS 335 Machine Learning What is Machine Learning? Dan Sheldon Spring 2019 What is Machine

CS 335 Machine Learning What is Machine Learning? Dan Sheldon Spring 2019 What is Machine

1/22/19 CS 335 Machine Learning What is Machine Learning? Dan Sheldon Spring 2019 What is Machine Learning? A Simple Task: Recognize Obama How do you program a computer to Recognize faces? Recommend movies? Decide which web

581 views • 5 slides
Machine Learning Machine Learning: algorithms that use experience to improve their

Machine Learning Machine Learning: algorithms that use experience to improve their

Machine Learning Machine Learning: algorithms that use experience to improve their performance We use machine learning in situations where it is very challenging (or impossible) to define the rules by hand: e.g. face detection

1.04k views • 28 slides
Introduction to Machine Learning COMPSCI 371D Machine Learning COMPSCI 371D Machine

Introduction to Machine Learning COMPSCI 371D Machine Learning COMPSCI 371D Machine

Introduction to Machine Learning COMPSCI 371D Machine Learning COMPSCI 371D Machine Learning Introduction to Machine Learning 1 / 12 Outline 1 Nearest Neighbor Prediction 2 Complexity Considerations 3 The Voronoi Diagram 4 Overfitting

564 views • 12 slides
1 Why Study Machine Learning? Why Study Machine Learning? Cognitive Science The Time is Ripe

1 Why Study Machine Learning? Why Study Machine Learning? Cognitive Science The Time is Ripe

What is Learning? Herbert Simon: Learning is any process by which a system improves performance from CS 391L: Machine Learning experience. Introduction What is the task? Classification Problem solving / planning /

603 views • 6 slides
MACHINE LEARNING, STATISTICAL LEARNING AND PARALLEL COMPUTING INTRODUCTION VS MACHINE LEARNING

MACHINE LEARNING, STATISTICAL LEARNING AND PARALLEL COMPUTING INTRODUCTION VS MACHINE LEARNING

JACOPO DI IORIO, MATTEO FONTANA, DANIELE OCCHIUTO, CLAUDIA VOLPETTI MACHINE LEARNING, STATISTICAL LEARNING AND PARALLEL COMPUTING INTRODUCTION VS MACHINE LEARNING STATISTICAL LEARNING Separate evolutions, but with shared methodologies MERGE

565 views • 55 slides
Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine

Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine

Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine Learning? What is Machine Learning? Examples of machine learning applications: Facial recognition software E-mail spam detectors Automated

302 views • 17 slides
Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning

Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning

Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning 11.1 Learning agents 11.2 Inductive learning 11.3 Deep learning 11.4 Statistical learning 11.5 Reinforcement learning 11.6 Transfer learning

1.7k views • 159 slides
Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine

Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine learning pipeline Temporal Frame-level Unsupervised Classifier Summary

613 views • 27 slides
Machine Learning for Auto Optimization What is Machine Learning? Definition: Machine

Machine Learning for Auto Optimization What is Machine Learning? Definition: Machine

Machine Learning for Auto Optimization What is Machine Learning? Definition: Machine learning refers to any system where the performance of a machine in performing a task improves by gaining more experience in performing that task .

394 views • 27 slides
MACHINE LEARNING kernels 1 MACHINE LEARNING 2012 MACHINE LEARNING Kernels: Intuition How

MACHINE LEARNING kernels 1 MACHINE LEARNING 2012 MACHINE LEARNING Kernels: Intuition How

MACHINE LEARNING 2012 MACHINE LEARNING MACHINE LEARNING kernels 1 MACHINE LEARNING 2012 MACHINE LEARNING Kernels: Intuition How to separate the red class from the grey class? x 2 360 r x 1 Polar coordinates Data

1.04k views • 44 slides
BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest

BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest

photo:@rewardyfahmi // Unsplash BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest Neighbor Classifier Aykut Erdem // Hacettepe University // Fall 2019 When Do We Use Machine Learning? ML is

1.33k views • 110 slides
Machine Learning Modeling and Learning 15-110 Monday 4/13 Learning Goals Given a

Machine Learning Modeling and Learning 15-110 Monday 4/13 Learning Goals Given a

Machine Learning Modeling and Learning 15-110 Monday 4/13 Learning Goals Given a dataset, identify features which may help predict information about the data Identify common types of machine learning algorithms 2 Machine

553 views • 27 slides
1 7 8 Types of simulation models Dynamic simulation models Stochastic vs. deterministic.

1 7 8 Types of simulation models Dynamic simulation models Stochastic vs. deterministic.

2 Outlines Simulation Modeling in Animal Health Management: Introduction: What is simulation modeling? A Stochastic Bio-economic Model of Bovine Intramammary Why simulation modeling is used? Infections (IMI) What are the

484 views • 14 slides
Udder Dimensions Dairy produc.on traits Genome Wide Associa.on

Udder Dimensions Dairy produc.on traits Genome Wide Associa.on

David Riley, Texas A&M University 6/2/17 Udder Dimensions Dairy produc.on traits Genome Wide Associa.on for Weaning weights in beef caDle

490 views • 4 slides
CFAP Assistance CFAP provides direct assistance to agricultural producers impacted by the

CFAP Assistance CFAP provides direct assistance to agricultural producers impacted by the

CFAP Assistance CFAP provides direct assistance to agricultural producers impacted by the effects of the COVID-19 outbreak. CFAP 4 CFAP Funding Two Funding Sources: CARES Act provides $ 9.5 Billion CCC Charter Act provides

824 views • 53 slides
Making a successful RfPB application Tony Akobeng Royal Manchester Childrens Hospital The

Making a successful RfPB application Tony Akobeng Royal Manchester Childrens Hospital The

Making a successful RfPB application Tony Akobeng Royal Manchester Childrens Hospital The project Telephone consultation as a substitute for routine out-patient face-to-face consultation for children with inflammatory bowel disease:

636 views • 19 slides
Can beef cattle intensification reduce environmental pressure on Brazilian areas? A case

Can beef cattle intensification reduce environmental pressure on Brazilian areas? A case

Can beef cattle intensification reduce environmental pressure on Brazilian areas? A case study for Corumb city(Pantanal biome) and Mato Grosso state(Legal Amazon) using a mathematical model model Adriano Gomes Garcia Magda Peixoto

476 views • 21 slides
Negative incidents are very rare Usually only occurs if: East Bay Regional Parks District 18

Negative incidents are very rare Usually only occurs if: East Bay Regional Parks District 18

Negative incidents are very rare Usually only occurs if: East Bay Regional Parks District 18 (reported) incidents / 4 years very ill 0.000225% feels threatened A few out of millions is an chased or harassed unacceptable risk?

419 views • 12 slides
Semantics for Natural Languages Compositionality Desiderata for Meaning Representation

Semantics for Natural Languages Compositionality Desiderata for Meaning Representation

Introduction Introduction Syntax and Semantics Logical Representations Logical Representations Compositionality Semantic Composition Semantic Composition Desiderata for Meaning Representation Introduction 1 Syntax and Semantics Semantics

280 views • 8 slides
11: Catchup II Machine Learning and Real-world Data (MLRD) Ann Copestake Lent 2019 Last

11: Catchup II Machine Learning and Real-world Data (MLRD) Ann Copestake Lent 2019 Last

11: Catchup II Machine Learning and Real-world Data (MLRD) Ann Copestake Lent 2019 Last session: HMM in a biological application In the last session, we used an HMM as a way of approximating some aspects of protein structure. Today: catchup

380 views • 17 slides

More recommend