a better k means algorithm via local search
play

A better k-means++ Algorithm via Local Search Silvio Lattanzi - PowerPoint PPT Presentation

Sep 20, 2022 •567 likes •670 views

A better k-means++ Algorithm via Local Search Silvio Lattanzi Christian Sohler Google Research Google Research ICML 2019 k-means Find a set of k centers X c C d 2 ( x, c ) ( X, C ) = min x X Constant approximation algorithms

  1. A better k-means++ Algorithm via Local Search Silvio Lattanzi Christian Sohler 
 Google Research Google Research ICML 2019

  2. k-means Find a set of k centers X c ∈ C d 2 ( x, c ) φ ( X, C ) = min x ∈ X Constant approximation algorithms are known. Goal is to design a constant approximation algorithm that is efficient, easy to implement and has good experimental results. A better k-means++ Algorithm via Local Search

  3. <latexit sha1_base64="Qtv+Xu/NZDL6vUHV4Ro0OEjlzA=">AB8HicdVDLSgMxFM3UV62vqks3wSLUzZDpWNtl0Y07K9iHtEPJpJk2NDMZkoxQhn6FGxeKuPVz3Pk3ptMKnrgwuGce7n3Hj/mTGmEPqzcyura+kZ+s7C1vbO7V9w/aCuRSEJbRHAhuz5WlLOItjTnHZjSXHoc9rxJ5dzv3NPpWIiutXTmHohHkUsYARrI91dl/tcjODkdFAsIdtFbrWOYEbcWiUjteq5Cx0bZSiBJZqD4nt/KEgS0kgTjpXqOSjWXoqlZoTWaGfKBpjMsEj2jM0wiFVXpodPIMnRhnCQEhTkYaZ+n0ixaFS09A3nSHWY/Xbm4t/eb1EB3UvZVGcaBqRxaIg4VALOP8eDpmkRPOpIZhIZm6FZIwlJtpkVDAhfH0K/yftiu0g27k5KzUulnHkwRE4BmXgBpogCvQBC1AQAgewBN4tqT1aL1Yr4vWnLWcOQ/YL19AvQWj9o=</latexit> <latexit sha1_base64="Qtv+Xu/NZDL6vUHV4Ro0OEjlzA=">AB8HicdVDLSgMxFM3UV62vqks3wSLUzZDpWNtl0Y07K9iHtEPJpJk2NDMZkoxQhn6FGxeKuPVz3Pk3ptMKnrgwuGce7n3Hj/mTGmEPqzcyura+kZ+s7C1vbO7V9w/aCuRSEJbRHAhuz5WlLOItjTnHZjSXHoc9rxJ5dzv3NPpWIiutXTmHohHkUsYARrI91dl/tcjODkdFAsIdtFbrWOYEbcWiUjteq5Cx0bZSiBJZqD4nt/KEgS0kgTjpXqOSjWXoqlZoTWaGfKBpjMsEj2jM0wiFVXpodPIMnRhnCQEhTkYaZ+n0ixaFS09A3nSHWY/Xbm4t/eb1EB3UvZVGcaBqRxaIg4VALOP8eDpmkRPOpIZhIZm6FZIwlJtpkVDAhfH0K/yftiu0g27k5KzUulnHkwRE4BmXgBpogCvQBC1AQAgewBN4tqT1aL1Yr4vWnLWcOQ/YL19AvQWj9o=</latexit> <latexit sha1_base64="Qtv+Xu/NZDL6vUHV4Ro0OEjlzA=">AB8HicdVDLSgMxFM3UV62vqks3wSLUzZDpWNtl0Y07K9iHtEPJpJk2NDMZkoxQhn6FGxeKuPVz3Pk3ptMKnrgwuGce7n3Hj/mTGmEPqzcyura+kZ+s7C1vbO7V9w/aCuRSEJbRHAhuz5WlLOItjTnHZjSXHoc9rxJ5dzv3NPpWIiutXTmHohHkUsYARrI91dl/tcjODkdFAsIdtFbrWOYEbcWiUjteq5Cx0bZSiBJZqD4nt/KEgS0kgTjpXqOSjWXoqlZoTWaGfKBpjMsEj2jM0wiFVXpodPIMnRhnCQEhTkYaZ+n0ixaFS09A3nSHWY/Xbm4t/eb1EB3UvZVGcaBqRxaIg4VALOP8eDpmkRPOpIZhIZm6FZIwlJtpkVDAhfH0K/yftiu0g27k5KzUulnHkwRE4BmXgBpogCvQBC1AQAgewBN4tqT1aL1Yr4vWnLWcOQ/YL19AvQWj9o=</latexit> <latexit sha1_base64="Qtv+Xu/NZDL6vUHV4Ro0OEjlzA=">AB8HicdVDLSgMxFM3UV62vqks3wSLUzZDpWNtl0Y07K9iHtEPJpJk2NDMZkoxQhn6FGxeKuPVz3Pk3ptMKnrgwuGce7n3Hj/mTGmEPqzcyura+kZ+s7C1vbO7V9w/aCuRSEJbRHAhuz5WlLOItjTnHZjSXHoc9rxJ5dzv3NPpWIiutXTmHohHkUsYARrI91dl/tcjODkdFAsIdtFbrWOYEbcWiUjteq5Cx0bZSiBJZqD4nt/KEgS0kgTjpXqOSjWXoqlZoTWaGfKBpjMsEj2jM0wiFVXpodPIMnRhnCQEhTkYaZ+n0ixaFS09A3nSHWY/Xbm4t/eb1EB3UvZVGcaBqRxaIg4VALOP8eDpmkRPOpIZhIZm6FZIwlJtpkVDAhfH0K/yftiu0g27k5KzUulnHkwRE4BmXgBpogCvQBC1AQAgewBN4tqT1aL1Yr4vWnLWcOQ/YL19AvQWj9o=</latexit> k-means++ seeding Elegant and simple algorithm Experimentally gives good results when combined with Lloyd’s algorithm. The solution is a approximation in expectation. O (log k ) David Arthur, Sergei Vassilvitskii: k-means++: the advantages of careful seeding. SODA 2007: 1027-1035 A better k-means++ Algorithm via Local Search

  4. Local search Elegant and simple algorithm It returns a constant approximation and nice experimental results. The algorithm is a bit slow. Tapas Kanungo, David M. Mount, Nathan S. Netanyahu, Christine D. Piatko, Ruth Silverman, Angela Y. Wu: 
 A local search approximation algorithm for k-means clustering. Comput. Geom. 28(2-3): 89-112 (2004) A better k-means++ Algorithm via Local Search

  5. Combining the two algorithms Elegant and simple algorithm It returns a constant approximation, it is slightly slower than k-means++ and has better experimental results. A better k-means++ Algorithm via Local Search

  6. <latexit sha1_base64="CuakJu/HyZqru1PUc0cdh0lQwu4=">ACBnicbVBNS8NAEN34WetX1KMIwSLUgyUrgh6LXjxWsB/QhLZbtqlm03YnQgl5OTFv+LFgyJe/Q3e/Ddu2xy09cHA470ZuYFieAaXPfbWlpeWV1bL2UN7e2d3btvf2WjlNFWZPGIladgGgmuGRN4CBYJ1GMRIFg7WB0M/HbD0xpHst7GCfMj8hA8pBTAkbq2UeYCFU8ZkXKkIznGfYdUe5p/hgCKc9u+LW3CmcRYILUkEFGj37y+vHNI2YBCqI1l3sJuBnRAGnguVlL9UsIXREBqxrqCQR0342fSN3TozSd8JYmZLgTNXfExmJtB5HgemMCAz1vDcR/O6KYRXfsZlkgKTdLYoTIUDsTPJxOlzxSiIsSGEKm5udeiQmDzAJFc2IeD5lxdJ67yG3Rq+u6jUr4s4SugQHaMqwugS1dEtaqAmougRPaNX9GY9WS/Wu/Uxa12yipkD9AfW5w/Vw5gS</latexit> <latexit sha1_base64="CuakJu/HyZqru1PUc0cdh0lQwu4=">ACBnicbVBNS8NAEN34WetX1KMIwSLUgyUrgh6LXjxWsB/QhLZbtqlm03YnQgl5OTFv+LFgyJe/Q3e/Ddu2xy09cHA470ZuYFieAaXPfbWlpeWV1bL2UN7e2d3btvf2WjlNFWZPGIladgGgmuGRN4CBYJ1GMRIFg7WB0M/HbD0xpHst7GCfMj8hA8pBTAkbq2UeYCFU8ZkXKkIznGfYdUe5p/hgCKc9u+LW3CmcRYILUkEFGj37y+vHNI2YBCqI1l3sJuBnRAGnguVlL9UsIXREBqxrqCQR0342fSN3TozSd8JYmZLgTNXfExmJtB5HgemMCAz1vDcR/O6KYRXfsZlkgKTdLYoTIUDsTPJxOlzxSiIsSGEKm5udeiQmDzAJFc2IeD5lxdJ67yG3Rq+u6jUr4s4SugQHaMqwugS1dEtaqAmougRPaNX9GY9WS/Wu/Uxa12yipkD9AfW5w/Vw5gS</latexit> <latexit sha1_base64="CuakJu/HyZqru1PUc0cdh0lQwu4=">ACBnicbVBNS8NAEN34WetX1KMIwSLUgyUrgh6LXjxWsB/QhLZbtqlm03YnQgl5OTFv+LFgyJe/Q3e/Ddu2xy09cHA470ZuYFieAaXPfbWlpeWV1bL2UN7e2d3btvf2WjlNFWZPGIladgGgmuGRN4CBYJ1GMRIFg7WB0M/HbD0xpHst7GCfMj8hA8pBTAkbq2UeYCFU8ZkXKkIznGfYdUe5p/hgCKc9u+LW3CmcRYILUkEFGj37y+vHNI2YBCqI1l3sJuBnRAGnguVlL9UsIXREBqxrqCQR0342fSN3TozSd8JYmZLgTNXfExmJtB5HgemMCAz1vDcR/O6KYRXfsZlkgKTdLYoTIUDsTPJxOlzxSiIsSGEKm5udeiQmDzAJFc2IeD5lxdJ67yG3Rq+u6jUr4s4SugQHaMqwugS1dEtaqAmougRPaNX9GY9WS/Wu/Uxa12yipkD9AfW5w/Vw5gS</latexit> <latexit sha1_base64="CuakJu/HyZqru1PUc0cdh0lQwu4=">ACBnicbVBNS8NAEN34WetX1KMIwSLUgyUrgh6LXjxWsB/QhLZbtqlm03YnQgl5OTFv+LFgyJe/Q3e/Ddu2xy09cHA470ZuYFieAaXPfbWlpeWV1bL2UN7e2d3btvf2WjlNFWZPGIladgGgmuGRN4CBYJ1GMRIFg7WB0M/HbD0xpHst7GCfMj8hA8pBTAkbq2UeYCFU8ZkXKkIznGfYdUe5p/hgCKc9u+LW3CmcRYILUkEFGj37y+vHNI2YBCqI1l3sJuBnRAGnguVlL9UsIXREBqxrqCQR0342fSN3TozSd8JYmZLgTNXfExmJtB5HgemMCAz1vDcR/O6KYRXfsZlkgKTdLYoTIUDsTPJxOlzxSiIsSGEKm5udeiQmDzAJFc2IeD5lxdJ67yG3Rq+u6jUr4s4SugQHaMqwugS1dEtaqAmougRPaNX9GY9WS/Wu/Uxa12yipkD9AfW5w/Vw5gS</latexit> Main theoretical result Main idea is to adapt local search analysis to show that in every step with constant probability we reduce the cost of the solution by a ✓ ◆ 1 multiplicative factor 1 − 100 k A better k-means++ Algorithm via Local Search

  7. Experimental results Datasets: - RNA : 8 features from 488565 RNA input sequence pairs (Uzilov et al., 2006) - KDD-BIO : 145751 samples with 74 features measuring the match between a protein and a native sequence (KDD) - KDD-PHY : 100000 samples with 78 features representing a quantum physic task (KDD) A better k-means++ Algorithm via Local Search

  8. Experimental results KDD-BIO RNA KDD-PHY A better k-means++ Algorithm via Local Search

  9. Thanks A better k-means++ Algorithm via Local Search

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

1 K-means clustering The K-means clustering algorithm can be seen as applying the EM algorithm to

1 K-means clustering The K-means clustering algorithm can be seen as applying the EM algorithm to

Statistical Modeling and Analysis of Neural Data (NEU 560) Princeton University, Spring 2018 Jonathan Pillow Lecture 18 notes: K-means and Factor Analysis Tues, 4.17 1 K-means clustering The K-means clustering algorithm can be seen as

547 views • 3 slides
Stochastic Local Search Methods Dynamic Local Search Iterated Local Search Tabu Search Marco

Stochastic Local Search Methods Dynamic Local Search Iterated Local Search Tabu Search Marco

Outline DM811 HEURISTICS AND LOCAL SEARCH ALGORITHMS FOR COMBINATORIAL OPTIMZATION 1. Stochastic Local Search Methods (Metaheuristics) Randomized Iterative Improvement Lecture 12 Attribute Based Hill Climber Stochastic Local Search Methods

590 views • 9 slides
Accelerating a local search algorithm for large instances of the independent task scheduling

Accelerating a local search algorithm for large instances of the independent task scheduling

Motivation Initial algorithm Adaptation Large instances Conclusion Accelerating a local search algorithm for large instances of the independent task scheduling problem with the GPU Frederic Pinel Johnatan Pecero Pascal Bouvry Computer

367 views • 18 slides
Local Search Distances Landscape Characteristics Fitness-Distance Correlation Ruggedness Marco

Local Search Distances Landscape Characteristics Fitness-Distance Correlation Ruggedness Marco

Outline DM811 1. Local Search, Basic Elements HEURISTICS AND LOCAL SEARCH ALGORITHMS Components and Algorithms FOR COMBINATORIAL OPTIMZATION Beyond Local Optima Computational Complexity 2. Fundamental Search Space Properties Introduction

698 views • 20 slides
K-MEANS++ OPTIMAL INITIALIZATION ALGORITHM An Improved K-means Clustering Method OVERVIEW

K-MEANS++ OPTIMAL INITIALIZATION ALGORITHM An Improved K-means Clustering Method OVERVIEW

K-MEANS++ OPTIMAL INITIALIZATION ALGORITHM An Improved K-means Clustering Method OVERVIEW K-means Clustering Algorithm K-means++ Initialization Algorithm Experiment Datasets Conclusion K-MEANS CLUSTERING ALGORITHM A

313 views • 14 slides
A-Star Algorithm &amp; Heaps/Priority Queues Mark Redekopp 2 A* Search Algorithm ALGORITHM

A-Star Algorithm &amp; Heaps/Priority Queues Mark Redekopp 2 A* Search Algorithm ALGORITHM

1 EE 355 Unit 14 A-Star Algorithm &amp; Heaps/Priority Queues Mark Redekopp 2 A* Search Algorithm ALGORITHM HIGHLIGHT 3 Search Methods Many systems require searching for goal states Path Planning Roomba Vacuum

887 views • 40 slides
Heuristic (Informed) search strategy Search Algorithm #2 Search SEARCH#2 1.

Heuristic (Informed) search strategy Search Algorithm #2 Search SEARCH#2 1.

Recall that the ordering of FRINGE defines the Heuristic (Informed) search strategy Search Algorithm #2 Search SEARCH#2 1. INSERT(initial-node,FRINGE) (Where we try to choose smartly) (Wh t t h tl ) 2. Repeat: a. If empty(FRINGE) then

393 views • 16 slides
Automatic Algorithm Configuration based on Local Search Frank Hutter 1 Holger Hoos 1 utzle 2

Automatic Algorithm Configuration based on Local Search Frank Hutter 1 Holger Hoos 1 utzle 2

Automatic Algorithm Configuration based on Local Search Frank Hutter 1 Holger Hoos 1 utzle 2 Thomas St 1 Department of Computer Science University of British Columbia Canada 2 IRIDIA Universit e Libre de Bruxelles Belgium Real-world

835 views • 59 slides
A Decomposition-based Local Search Algorithm for Multi-objective Sequence Dependent Setup Times

A Decomposition-based Local Search Algorithm for Multi-objective Sequence Dependent Setup Times

Basic concepts MOLS/D Experimental Study Final Considerations IEEE CEC 2018 A Decomposition-based Local Search Algorithm for Multi-objective Sequence Dependent Setup Times Permutation Flowshop Scheduling Murilo Zangari, Ademir Constantino,

297 views • 17 slides
Metaheuristics 2.3 Local Search 2.4 Simulated annealing Adrian Horga 1 2.3 Local Search 2

Metaheuristics 2.3 Local Search 2.4 Simulated annealing Adrian Horga 1 2.3 Local Search 2

Metaheuristics 2.3 Local Search 2.4 Simulated annealing Adrian Horga 1 2.3 Local Search 2 Local Search Other names: Hill climbing Descent Iterative improvement General S-Metaheuristics Old and simple method at each

620 views • 25 slides
Local Search Toolbox so far Uninformed search BFS, DFS,

Local Search Toolbox so far Uninformed search BFS, DFS,

Local Search Toolbox so far Uninformed search BFS, DFS, Itera;ve deepening DFS, Uniform cost search Heuris;c search A* Review Search:

440 views • 19 slides
k -means++ seeding Have seen that the k -means algorithm can output arbitrarily poor solutions, if

k -means++ seeding Have seen that the k -means algorithm can output arbitrarily poor solutions, if

k -means++ seeding Have seen that the k -means algorithm can output arbitrarily poor solutions, if started with a bad set of initial centroids k -means++ is a simple, probabilistic algorithm to compute initial centroids These centroids are

745 views • 8 slides
IDWalk : a local search algorithm combining intensification and diversification Bertrand Neveu 1 ,

IDWalk : a local search algorithm combining intensification and diversification Bertrand Neveu 1 ,

1 IDWalk : a local search algorithm combining intensification and diversification Bertrand Neveu 1 , Gilles Trombettoni 1 , Fred Glover 2 1 Equipe COPRIN CERTIS-I3S-INRIA, Sophia Antipolis, France neveu@sophia.inria.fr trombe@sophia.inria.fr 2

259 views • 21 slides
Uninformed Search (Ch. 3-3.4) 2 Announcements HW due tonight Writing 1 due next week 3 Search

Uninformed Search (Ch. 3-3.4) 2 Announcements HW due tonight Writing 1 due next week 3 Search

1 Uninformed Search (Ch. 3-3.4) 2 Announcements HW due tonight Writing 1 due next week 3 Search algorithm To search, we will build a tree with the root as the initial state Any problems with this? 6 Search algorithm We can remove

400 views • 22 slides
Local search algorithms AIMA sections 4.1,4.2 Summary Local search algorithms Hill-climbing

Local search algorithms AIMA sections 4.1,4.2 Summary Local search algorithms Hill-climbing

Local search algorithms Local search algorithms AIMA sections 4.1,4.2 Summary Local search algorithms Hill-climbing Simulated annealing Genetic algorithms (briefly) Local search in continuous spaces (very briefly) Iterative

300 views • 16 slides
X Example? In such cases, we can use local search algorithms Keep a single

X Example? In such cases, we can use local search algorithms Keep a single

Todays Class Local Search Iterative improvement methods AI Class 6 (Ch. 4.1-4.2) Hill climbing If the path to the goal Simulated annealing does not matter [we Local beam search can use] a single current node and move to

236 views • 5 slides
Lecture 4: RNA folding Chapter 6 Problem 6.51 in Jones and Pevzner and the Turner model

Lecture 4: RNA folding Chapter 6 Problem 6.51 in Jones and Pevzner and the Turner model

Lecture 4: RNA folding Chapter 6 Problem 6.51 in Jones and Pevzner and the Turner model Spring 2020 February 4, 2020 RNA Basics RNA bases A,C,G,U Canonical Base Pairs A-U G-C G-U wobble pairing Bases can only pair

637 views • 24 slides
Faster folds, Better folds: Genetic Improvement of RNAfold W. B. Langdon Computer Science,

Faster folds, Better folds: Genetic Improvement of RNAfold W. B. Langdon Computer Science,

CREST Open Workshop 25 th September 2017 Faster folds, Better folds: Genetic Improvement of RNAfold W. B. Langdon Computer Science, University College London GI 2018, Gteborg, ICSE-2018 proposed workshop 23.9.2017 Genetic Improvement of

259 views • 24 slides
RM Synthesis &amp; SKA CMF George Heald SKA Science working group meeting 22 January 2014 1

RM Synthesis &amp; SKA CMF George Heald SKA Science working group meeting 22 January 2014 1

Netherlands Institute for Radio Astronomy RM Synthesis &amp; SKA CMF George Heald SKA Science working group meeting 22 January 2014 1 ASTRON is part of the Netherlands Organisation for Scientific Research (NWO) Overview

520 views • 32 slides
( reset ) September 6, 2013 1 / 19 A temporal semantics for Nilpotent Minimum logic Matteo

( reset ) September 6, 2013 1 / 19 A temporal semantics for Nilpotent Minimum logic Matteo

( reset ) September 6, 2013 1 / 19 A temporal semantics for Nilpotent Minimum logic Matteo Bianchi Universit` a degli Studi di Milano matteo.bianchi@unimi.it ( reset ) September 6, 2013 1 / 19 Outline Nilpotent Minimum logic (NM) was

1.11k views • 77 slides
On the Combinatorics of RNA Secondary Structures in a Polymer-Zeta Model Markus E. Nebel based on

On the Combinatorics of RNA Secondary Structures in a Polymer-Zeta Model Markus E. Nebel based on

On the Combinatorics of RNA Secondary Structures in a Polymer-Zeta Model Markus E. Nebel based on joint work with Emma Yu Jin CanaDAM 2013 Newfoundland, Canada Markus E. Nebel RNA in Polymer-Zeta Model 2013/18/5 1 / 17 Plan of Talk RNA

614 views • 32 slides
Approximation of RNA Multiple Structural Alignment Marcin Kubica 1 , Romeo Rizzi 2 , Stphane

Approximation of RNA Multiple Structural Alignment Marcin Kubica 1 , Romeo Rizzi 2 , Stphane

Approximation of RNA Multiple Structural Alignment Marcin Kubica 1 , Romeo Rizzi 2 , Stphane Vialette 3 and Tomasz Wale 1 1 Faculty of Mathematics, Informatics and Applied Mathematics Warsaw University, Poland 2 Dipartimento di Matematica ed

562 views • 22 slides
Pairwise RNA Edit Distance In the following: Sequences S 1 and S 2 associated

Pairwise RNA Edit Distance In the following: Sequences S 1 and S 2 associated

Pairwise RNA Edit Distance In the following: Sequences S 1 and S 2 associated structures P 1 and P 2 scoring of alignment: different edit operations arc altering arc removing ..(((....)))..... 1)

390 views • 18 slides
Truly Subcubic Algorithms for Language Edit Distance and RNA Folding via Fast Bounded-Difference

Truly Subcubic Algorithms for Language Edit Distance and RNA Folding via Fast Bounded-Difference

Truly Subcubic Algorithms for Language Edit Distance and RNA Folding via Fast Bounded-Difference Min-Plus Product Karl Bringmann , Fabrizio Grandoni, Barna Saha, Virginia Vassilevska Williams June 11, 2017 Bounded Differences (BD) Matrices

639 views • 22 slides

More recommend