conceptual clustering using lingo algorithm evaluation on
play

Conceptual Clustering Using Lingo Algorithm: Evaluation on Open - PowerPoint PPT Presentation

Mar 19, 2023 •478 likes •722 views

Conceptual Clustering Using Lingo Algorithm: Evaluation on Open Directory Project Data Stanis law Osi nski Dawid Weiss Institute of Computing Science Pozna n University of Technology May 20th, 2004 Some background: how to evaluate

  1. Conceptual Clustering Using Lingo Algorithm: Evaluation on Open Directory Project Data Stanis� law Osi´ nski Dawid Weiss Institute of Computing Science Pozna´ n University of Technology May 20th, 2004

  2. Some background: how to evaluate an SRC algorithm? About various goals of evaluation. . . Reconstruction of a predefined structure Test data: merge-then-cluster, manual labeling Measures: precision-recall, entropy measures. . . Labels “quality”, descriptiveness User surveys, click-distance methods

  3. Some background: how to evaluate an SRC algorithm? What types of “errors” can an algorithm make structure-wise? Misassignment errors (document → cluster) Missing documents in a cluster Incorrect clusters (unexplainable) Missing clusters (undetected) Granularity level confusion (subcluster domination problems)

  4. Evaluation of Lingo’s performance We tried to answer the following questions: Clusters’ structure: 1 Is Lingo able to cluster similar documents? 2 Is Lingo able to highlight outliers and “minorities”? 3 Is Lingo able to capture generalizations of closely-related subjects? 4 How does Lingo compare to Suffix Tree Clustering? Quality of cluster labels Are clusters labelled appropriately? Are they informative?

  5. Data set for the experiment Data set: a subset of the Open Directory Project Rationale: Human-created and maintained structure Human-created and maintained labels Descriptions resemble search results (snippets) Free availability

  6. ODP Categories chosen for the experiment MOVIES HEALTH CARE Ortho LRings BRunner PHOTOGRAPHY Infra COMPUTER SCIENCE DATABASES MISC. DWare MySQL JavaTut XMLDB Vi Postgr

  7. Test sets for the experiement Test sets Test sets were combinations of categories designed to help in answering the set of questions. Identifier Merged categories Test set rationale G1 LRings , MySQL Separation of two unrelated categories. G2 LRings , MySQL , Ortho Separation of three unrelated categories. G3 LRings , MySQL , Ortho , Separation of four unrelated categories, highligting Infra small topics ( Infra ). G4 MySQL , XMLDB , DWare , Separation of four conceptually close categories, all Postgr connected to database. G5 MySQL , XMLDB , DWare , Four conceptually very close categories (database) Postgr , JavaTut , Vi plus two distinct, but within the same abstract topic (computer science). G6 MySQL , XMLDB , DWare , Outlier highlight test – four dominating conceptually Postgr , Ortho close categories (databases) and one outlier ( Ortho ) G7 All categories All categories mixed together. Cross-topic cluster de- tection test (movies, databases).

  8. The experiment Lingo’s implementation → Carrot 2 framework The algorithm’s thresholds: Fixed at “good guess” values (same as those used in the on-line demo) Stemming and stop-word detection applied to the input data

  9. The results Method of analysis Manual investigation of document-to-cluster assignment charts. Helps understand the internal structure of results Prevents compensations inherent in aggregative measures

  10. → Is Lingo able to cluster similar documents? Categories-in-clusters view, input test: G3 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 30 25 20 15 10 5 0 MySQL News Information on Infrared Images Galleries Foot Orthotics Lord of the Rings Movie Orthopedic Products Humor Lotr Site Shoes Links Medical Database Support Stockings Middle Earth Manager lord of the rings mysql orthopedic infrared photography G1–G3: clear separation of topics, but with some extra clusters G1: granularity problem

  11. → Is Lingo able to cluster similar documents? Categories-in-clusters view, input test: G5 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 25 20 15 10 5 0 Java Tutorial Vim Page Federated Data Warehouse Native Xml Database Web Postgresql Database Mysql Server Free Links Development Tool Quick Reference Data Warehousing Mysql Client Object Oriented Postgres Driver java tutorials vi mysql data warehouses articles native xml databases postgres G5: misassignment problem

  12. → Is Lingo able to highlight outliers and “minorities”? Categories-in-clusters view, input test: G6 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 18 16 14 12 10 8 6 4 2 0 Mysql Database Federated Data Warehouse Foot Orthotics Orthopedic Products Access Postgresql Web Mysql Server Medical Shoes Designed Orthopaedic Postgres Mysql Client Data Warehousing Offers Software Development Tool Innovative mysql data warehouses articles native xml databases postgres orthopedic Ortho category (outlier), XMLDB consumed by MySQL!

  13. → Is Lingo able to highlight outliers and “minorities”? Categories-in-clusters view, input test: G7 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 50 45 40 35 30 25 20 15 10 5 0 Blade Runner Mysql Database Java Tutorial Lord of the Rings News Movie Review Information on Infrared Data Warehouse Images Galleries BBC Film Vim Macro Web Site Fan Fiction Fan Art Custom Orthotics Layout Management DBMS Online blade runner infrared photography java tutorials lord of the rings mysql orthopedic vi data warehouses articles native xml databases postgres Infra category (outlier)

  14. → Is Lingo able to highlight outliers and “minorities”? Categories-in-clusters view, input test: G5 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 25 20 15 10 5 0 Java Tutorial Vim Page Federated Data Warehouse Native Xml Database Web Postgresql Database Mysql Server Free Links Development Tool Quick Reference Data Warehousing Mysql Client Object Oriented Postgres Driver java tutorials vi mysql data warehouses articles native xml databases postgres XMLDB category (outlier)

  15. → Is Lingo able to capture generalizations? Categories-in-clusters view, input test: G7 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 50 45 40 35 30 25 20 15 10 5 0 Blade Runner Mysql Database Java Tutorial Lord of the Rings News Movie Review Information on Infrared Data Warehouse Images Galleries BBC Film Vim Macro Web Site Fan Fiction Fan Art Custom Orthotics Layout Management DBMS Online blade runner infrared photography java tutorials lord of the rings mysql orthopedic vi data warehouses articles native xml databases postgres “movie review” cluster is a generalization, but. . .

  16. → Is Lingo able to capture generalizations? Categories-in-clusters view, input test: G7 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 50 45 40 35 30 25 20 15 10 5 0 Blade Runner Mysql Database Java Tutorial Lord of the Rings News Movie Review Information on Infrared Data Warehouse Images Galleries BBC Film Vim Macro Web Site Fan Fiction Fan Art Custom Orthotics Layout Management DBMS Online blade runner infrared photography java tutorials lord of the rings mysql orthopedic vi data warehouses articles native xml databases postgres Clusters are usually orthogonal with SVD, so no good results should be expected in this area.

  17. → How does Lingo compare to Suffix Tree Clustering? Categories-in-clusters view, input test: G7 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 50 45 40 35 30 25 20 15 10 5 0 Blade Runner Mysql Database Java Tutorial Lord of the Rings News Movie Review Information on Infrared Data Warehouse Images Galleries BBC Film Vim Macro Web Site Fan Fiction Fan Art Custom Orthotics Layout Management DBMS Online blade runner infrared photography java tutorials lord of the rings mysql orthopedic vi data warehouses articles native xml databases postgres

  18. → How does Lingo compare to Suffix Tree Clustering? Categories-in-clusters view, input test: G7 STC algorithm, Top 16 clusters 35 30 25 20 15 10 5 0 xml,native,native xml database includes blade runner,blade,runner information dm,dm review article,dm review used database ralph,article by ralph kimball,ralp [...] mysql articles written site cast dm review article by douglas hackne [...] review characters blade runner infrared photography java tutorials lord of the rings mysql orthopedic vi data warehouses articles native xml databases postgres

  19. Key differences between Lingo and STC Size-dominated clusters in STC Cluster labels much less informative Common-term clusters in STC

  20. Cluster labels quality Performed manually Problems: Single term labels usually ambiguous or too broad (“news”, “free”) Level of granularity usually unclear (need for hierarchical methods?)

  21. A word about analytical comparison methods. . . Can these conclusions be derived using formulas? We think so: cluster contamination measures might help.

  22. Online demo A nice form of evaluation (although scientifically doubtful), is the online demo’s popularity and feedback we get from users.

  24. http://carrot.cs.put.poznan.pl Thank you. Questions?

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

DATA MINING LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm Clustering Evaluation What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or

1.39k views • 110 slides
DATA MINING LECTURE 9 The EM Algorithm Clustering Evaluation Sequence segmentation CLUSTERING

DATA MINING LECTURE 9 The EM Algorithm Clustering Evaluation Sequence segmentation CLUSTERING

DATA MINING LECTURE 9 The EM Algorithm Clustering Evaluation Sequence segmentation CLUSTERING What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or related) to one another and

905 views • 79 slides
1 K-means clustering The K-means clustering algorithm can be seen as applying the EM algorithm to

1 K-means clustering The K-means clustering algorithm can be seen as applying the EM algorithm to

Statistical Modeling and Analysis of Neural Data (NEU 560) Princeton University, Spring 2018 Jonathan Pillow Lecture 18 notes: K-means and Factor Analysis Tues, 4.17 1 K-means clustering The K-means clustering algorithm can be seen as

547 views • 3 slides
Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering? Aykut Erdem May 2016 Hacettepe University Last time K-Means An iterative clustering algorithm - Initialize: Pick K random points as cluster

932 views • 64 slides
LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a

LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a

DATA MINING LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or related) to one another and

998 views • 78 slides
Week 7 Video 1 Clustering Clustering A type of Structure Discovery algorithm This type of

Week 7 Video 1 Clustering Clustering A type of Structure Discovery algorithm This type of

Week 7 Video 1 Clustering Clustering A type of Structure Discovery algorithm This type of method is also referred to as Dimensionality Reduction , based on a common application Clustering You have a large number of data points You

974 views • 63 slides
A fuzzy clustering method using Genetic Algorithm and Fuzzy Subtractive Clustering Thanh Le, Tom

A fuzzy clustering method using Genetic Algorithm and Fuzzy Subtractive Clustering Thanh Le, Tom

A fuzzy clustering method using Genetic Algorithm and Fuzzy Subtractive Clustering Thanh Le, Tom Altman and Katheleen Gardiner University of Colorado Denver July 18, 2012 Overview Introduction Fuzzy clustering using Fuzzy C-Means

345 views • 22 slides
Clustering 2 Clustering 2 Nov 3 2008 HAC Algorithm HAC Algorithm St t Start with all objects in

Clustering 2 Clustering 2 Nov 3 2008 HAC Algorithm HAC Algorithm St t Start with all objects in

Clustering 2 Clustering 2 Nov 3 2008 HAC Algorithm HAC Algorithm St t Start with all objects in their own cluster. ith ll bj t i th i l t Until there is only one cluster: Among the current clusters determine the two Among the current

345 views • 23 slides
CB-DBSCAN: A Novel Clustering Algorithm for Adjacent Clusters with Different Densities Gashin

CB-DBSCAN: A Novel Clustering Algorithm for Adjacent Clusters with Different Densities Gashin

CB-DBSCAN: A Novel Clustering Algorithm for Adjacent Clusters with Different Densities Gashin Ghazizadeh Outline Introduction Multi-density Clustering CB-DBSCAN Results CB-DBSCAN: A Novel Clustering Algorithm for Adjacent

454 views • 33 slides
K-MEANS++ OPTIMAL INITIALIZATION ALGORITHM An Improved K-means Clustering Method OVERVIEW

K-MEANS++ OPTIMAL INITIALIZATION ALGORITHM An Improved K-means Clustering Method OVERVIEW

K-MEANS++ OPTIMAL INITIALIZATION ALGORITHM An Improved K-means Clustering Method OVERVIEW K-means Clustering Algorithm K-means++ Initialization Algorithm Experiment Datasets Conclusion K-MEANS CLUSTERING ALGORITHM A

313 views • 14 slides
A history of the k-means algorithm Hans-Hermann Bock, RWTH Aachen, Allemagne 1. Clustering with

A history of the k-means algorithm Hans-Hermann Bock, RWTH Aachen, Allemagne 1. Clustering with

A history of the k-means algorithm Hans-Hermann Bock, RWTH Aachen, Allemagne 1. Clustering with SSQ and the basic k-means algorithm 1.1 Discrete case 1.2 Continuous version 2. SSQ clustering for stratified survey sampling Dalenius (1950/51) 3.

587 views • 24 slides
Clustering on Graphs: The Markov Cluster Algorithm (MCL) CS 595D Presentation By Kathy Macropol

Clustering on Graphs: The Markov Cluster Algorithm (MCL) CS 595D Presentation By Kathy Macropol

Clustering on Graphs: The Markov Cluster Algorithm (MCL) CS 595D Presentation By Kathy Macropol MCL Algorithm Based on the PhD thesis by Stijn van Dongen Van Dongen, S. (2000) Graph Clustering by Flow Simulation . PhD Thesis, University of

1.03k views • 46 slides
Clustering Genetic Algorithm Petra Kudov Department of Theoretical Computer Science Institute

Clustering Genetic Algorithm Petra Kudov Department of Theoretical Computer Science Institute

Introduction Clustering Genetic Algorithm Experimental results Conclusion Clustering Genetic Algorithm Petra Kudov Department of Theoretical Computer Science Institute of Computer Science Academy of Sciences of the Czech Republic ETID

400 views • 18 slides
Clustering in Swedish The Impact of some Properties of the Swedish Language on Document

Clustering in Swedish The Impact of some Properties of the Swedish Language on Document

Introduction Clustering Evaluation Clustering in Swedish Clustering in Swedish The Impact of some Properties of the Swedish Language on Document Clustering and an Evaluation Method Magnus Rosell 2006-03-15 Magnus Rosell Clustering in

388 views • 28 slides
EM, K-Means, GNG 4-6-16 Reading Quiz Which of the following can be considered an instance of the

EM, K-Means, GNG 4-6-16 Reading Quiz Which of the following can be considered an instance of the

EM, K-Means, GNG 4-6-16 Reading Quiz Which of the following can be considered an instance of the EM algorithm? a) Agglomerative clustering b) Divisive clustering c) K-means clustering d) Growing neural gas EM algorithm E step:

445 views • 17 slides
k -means clustering Method to automatically separate data sets into distinct groups. Clustering

k -means clustering Method to automatically separate data sets into distinct groups. Clustering

k -means clustering Method to automatically separate data sets into distinct groups. Clustering example Clustering example k -means clustering algorithm 1. Start with k randomly chosen means 2. Color data points by the shortest distance to any

1.56k views • 19 slides
1 ACCESS TO TRAUMA CARE ACCESS TO TRAUMA CARE * The Swiss Government funded Sudan by 30,000.00USD

1 ACCESS TO TRAUMA CARE ACCESS TO TRAUMA CARE * The Swiss Government funded Sudan by 30,000.00USD

DATA COLLECTION DATA COLLECTION * Through the UNMAS IMSMA system which registers all the informat * Through the UNMAS IMSMA system which registers all the information ion gathered by the NGOs, survey teams, mine action sub offices. The

413 views • 5 slides
Presentation, diagnosis and management of SMA: Multidisciplinary perspectives SMA, spinal

Presentation, diagnosis and management of SMA: Multidisciplinary perspectives SMA, spinal

Presentation, diagnosis and management of SMA: Multidisciplinary perspectives SMA, spinal muscular atrophy. What are the key roles of the multidisciplinary teams that manage SMA and what challenges do they face? Dr Oscar Henry Mayer

323 views • 8 slides
Durable Medical Equipment, Orthotics, Prosthetics and Supplies Provider Outreach and Education

Durable Medical Equipment, Orthotics, Prosthetics and Supplies Provider Outreach and Education

Durable Medical Equipment, Orthotics, Prosthetics and Supplies Provider Outreach and Education Cindy White, Education Consultant Supplier Requirements Supplier Standard number 11 A supplier is prohibited from direct solicitation to

379 views • 3 slides
Morphisms of Reaction Networks Luca Cardelli, Microsoft Research & Oxford University with:

Morphisms of Reaction Networks Luca Cardelli, Microsoft Research & Oxford University with:

Morphisms of Reaction Networks Luca Cardelli, Microsoft Research & Oxford University with: Mirco T ribastone, Max Tschaikowski, Andrea Vandin IMT Institute for Advanced Studies, Lucca Attila Csiksz-Nagy Kings College London Neil

915 views • 55 slides
FOURTH QUARTER 2011 EARNINGS CALL February 23, 2012 1 FORWARD-LOOKING STATEMENTS Some of the

FOURTH QUARTER 2011 EARNINGS CALL February 23, 2012 1 FORWARD-LOOKING STATEMENTS Some of the

FOURTH QUARTER 2011 EARNINGS CALL February 23, 2012 1 FORWARD-LOOKING STATEMENTS Some of the statements made in the presentation whether written or oral may be forward-looking statements within the meaning of Section 27A of the

286 views • 17 slides
Investor Presentation August 2020 Disclaimer This presentation has been prepared by Osteopore

Investor Presentation August 2020 Disclaimer This presentation has been prepared by Osteopore

Breakthrough 3D printed bioresorbable implants to assist with the natural stages of bone healing. Osteopore Limited (ASX: OSX) Investor Presentation August 2020 Disclaimer This presentation has been prepared by Osteopore Limited and its

457 views • 13 slides
ASSOCIATION NEWS "How Does a Village Communicate Fluently?" Iris will cover a lot of

ASSOCIATION NEWS "How Does a Village Communicate Fluently?" Iris will cover a lot of

February 2020 ASSOCIATION NEWS "How Does a Village Communicate Fluently?" Iris will cover a lot of territory in this presentation and answer a lot of question such as "Why Are We Here?" "Who Is Our Customer?"

372 views • 20 slides
Opioid Abuse: What it means to Medicine Bruce Bonanno, MD, FACEP No declarations

Opioid Abuse: What it means to Medicine Bruce Bonanno, MD, FACEP No declarations

Opioid Abuse: What it means to Medicine Bruce Bonanno, MD, FACEP No declarations NJACEP President 201011 NJ PMP instituted Prescription Drug Abuse Campaign: ACEP March 20122015 NJACEP Opiate Abuse Task Force April

1.27k views • 88 slides

More recommend