An adaptive nearest neighbor rule for classification Akshay - PowerPoint PPT Presentation

Dec 23, 2022 •352 likes •411 views

An adaptive nearest neighbor rule for classification Akshay Balsubramani, Stanford Sanjoy Dasgupta, UCSD Yoav Freund, UCSD Shay Moran, Google AI Princeton Main Idea: Modify k -NN Algorithm by Choosing k Adaptively for Each Query Classical

An adaptive nearest neighbor rule for classification Akshay Balsubramani, Stanford Sanjoy Dasgupta, UCSD Yoav Freund, UCSD Shay Moran, Google AI Princeton
Main Idea: Modify k -NN Algorithm by Choosing k Adaptively for Each Query • Classical k -NN: classify x by the majority vote of its k nearest in the training set. x is the green point in the middle. The label assigned to x is determined by its k nearest neighbors (inside the big circle, in this example k =13+12=25 )
Main Idea: Modify k -NN Algorithm by Choosing k Adaptively for Each Query • Adaptive k -NN: • Iterate over the neighbors of x from nearest to furthest and query their labels. • If one of the label-classes obtains a significant majority then exit the loop and use this label to classify x . Points x that are far from the boundary observe a significant Points x that are close to the boundary require querying a large number of neighbors advantage after querying a small number of neighbors
Main Result s Theoretical Results 1. Adaptive k-NN rule is consistent (i.e. achieves Bayes optimality in the limit). 2. Instance-dependent generalization bounds Number of examples required to classify x correctly depends on its “local- • margin” (a formal notion introduced in the paper). Points far from the boundary are correctly classified fast. • Practical Results 1. Adaptive k -NN rule is competitive with Classical k -NN with the best choice of k Thus, this method circumvents the need to tune the meta-parameter k. •

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Nearest Neighbor Classification Machine Learning 1 This lecture K-nearest neighbor

Nearest Neighbor Classification Machine Learning 1 This lecture K-nearest neighbor classification The basic algorithm Different distance measures Some practical aspects Voronoi Diagrams and Decision Boundaries What is the

893 views • 57 slides

NEAREST NEIGHBOR RULE Jeff Robble, Brian Renzenbrink, Doug Roberts Nearest Neighbor Rule

NEAREST NEIGHBOR RULE Jeff Robble, Brian Renzenbrink, Doug Roberts Nearest Neighbor Rule Consider a test point x. x is the closest point to x out of the rest of the test points. Nearest Neighbor Rule selects the class for x with the

573 views • 33 slides

Nearest Neighbor Classification Seed classification by area and What should we compactness

Nearest Neighbor Classification Seed classification by area and What should we compactness predict for unlabeled Lecture 6 KNN and Decision Trees test points (stars)? Nearest neighbor classification: predict CS 335 label of

203 views • 3 slides

Nearest neighbor classification in metric spaces: universal consistency and rates of convergence

Nearest neighbor classification in metric spaces: universal consistency and rates of convergence Sanjoy Dasgupta University of California, San Diego Nearest neighbor The primeval approach to classification. Given: a training set { ( x 1 , y

926 views • 20 slides

Classification James H. Steiger Department of Psychology and Human Development Vanderbilt

Introduction The Linear Classification Function Quadratic Classification Functions Estimating Misclassification Rates Bias in Error Rate Estimation Error Rates in Variable Selection Classification via the k Nearest Neighbor Rule Classification

417 views • 31 slides

CSCI 447/547 MACHINE LEARNING Outline Nearest Neighbor K-Nearest Neighbor Algorithm

Nearest Neighbor CSCI 447/547 MACHINE LEARNING Outline Nearest Neighbor K-Nearest Neighbor Algorithm Note: Slides were adapted from David Sontag, New York University (who adapted them from Vibhav Gogate, Carlos Questrin, Mehryar

200 views • 16 slides

DATA MINING LECTURE 10 Classification Nearest Neighbor Classification Support Vector Machines

DATA MINING LECTURE 10 Classification Nearest Neighbor Classification Support Vector Machines Logistic Regression Nave Bayes Classifier Supervised Learning Illustrating Classification Task Learning Tid Attrib1 Attrib2 Attrib3 Class

954 views • 79 slides

DATA MINING LECTURE 11 Classification Nearest Neighbor Classification Support Vector Machines

DATA MINING LECTURE 11 Classification Nearest Neighbor Classification Support Vector Machines Logistic Regression Nave Bayes Classifier Supervised Learning Illustrating Classification Task Learning Tid Attrib1 Attrib2 Attrib3 Class

1.03k views • 75 slides

Data Mining Techniques: Statistical Decision Theory Nearest Neighbor Classification and

Classification and Prediction Overview Introduction Decision Trees Data Mining Techniques: Statistical Decision Theory Nearest Neighbor Classification and Prediction Bayesian Classification Artificial Neural Networks Mirek

701 views • 40 slides

Data Mining Techniques: Statistical Decision Theory Nearest Neighbor Classification and

548 views • 42 slides

Learning From Data Lecture 16 Similarity and Nearest Neighbor Similarity Nearest Neighbor M.

Learning From Data Lecture 16 Similarity and Nearest Neighbor Similarity Nearest Neighbor M. Magdon-Ismail CSCI 4100/6100 My 5-Year-Old Called It A ManoHorse The simplest method of learning that we know. Classify according to similar

331 views • 16 slides

Classification K-nearest neighbor classification D istance functions Choice of k Choice of k

Classification K-nearest neighbor classification D istance functions Choice of k Choice of k Leave-one-out cross validation K-fold cross validation Classification Error = Average classification error on K folds Linear Classification Linear

395 views • 22 slides

Nearest Neighbor Classifiers CSE 4308/5360: Artificial Intelligence I University of Texas at

Nearest Neighbor Classifiers CSE 4308/5360: Artificial Intelligence I University of Texas at Arlington 1 The Nearest Neighbor Classifier Let X be the space of all possible patterns for some classification problem. Let F be a distance

512 views • 24 slides

Nearest-Neighbor Methods Store all training examples Given a new test example, find the k that are

Nearest-Neighbor Methods Store all training examples Given a new test example, find the k that are closest to it in feature space (distance: Eu- About this class clidean/Mahalanobis?) k -Nearest-Neighbors Return majority classification among

368 views • 6 slides

Finding Predictors: Nearest Neighbor Modern Motivations: Be Lazy! Classification Regression

Finding Predictors: Nearest Neighbor Modern Motivations: Be Lazy! Classification Regression Choosing the right number of neighbors Some Optimizations Other types of lazy algorithms Compendium slides for Guide to Intelligent Data

1.53k views • 109 slides

Proximity in the Age of Distraction: Robust Approximate Nearest Neighbor Search Sariel Har-Peled

Proximity in the Age of Distraction: Robust Approximate Nearest Neighbor Search Sariel Har-Peled Sepideh Mahabadi UIUC MIT Nearest Neighbor Problem Nearest Neighbor Dataset of points in a metric space (, ) , e.g.

967 views • 81 slides

1 Greedy Sequential Covering Example Greedy Sequential Covering Example Y Y + + + + + +

Learning Rules If-then rules in logic are a standard representation of knowledge that have proven useful in expert-systems and other AI systems CS 391L: Machine Learning: In propositional logic a set of rules for a concept is equivalent

396 views • 14 slides

Majority Rule in the Absence of a Majority Klaus Nehring and Marcus Pivato ESSLLI August 13,

Majority Rule in the Absence of a Majority Klaus Nehring and Marcus Pivato ESSLLI August 13, 2013 Klaus Nehring and Marcus Pivato () Majority Rule ESSLLI August 13, 2013 1 / 34 Majoritarianism To fix ideas, cursory definition of

861 views • 36 slides

All A is B All A is B This is A This is B This is B This is A good form bad form Use to

The golden rule, like My perspective a diamond, reflects a is logic, which is different beauty from about forms of different perspectives. valid reasoning. All A is B All A is B This is A This is B This is B This is A good form

238 views • 9 slides

? S E L U R T L M 3 4 What Are Core Values? CORE VALUES Core values are

EVADING THE MLT POLICE? What MLT rulescan you think of? MAKING COMPROMISES WITHOUT COMPROMISING CORE VALUES Respond at PollEv.com/heathershoul495 OR Text HEATHERSHOUL495 to 37607 once to join, then text your response **Limit of 40

329 views • 5 slides

ADJUST TO FINAL RULE REVISIONS FOR MORE INFORMATION www.ae2s.com/rlcr.php Presentation

WRAP UP STAY INFORMED PREPARE ADJUST TO FINAL RULE REVISIONS FOR MORE INFORMATION www.ae2s.com/rlcr.php Presentation Slides Links to Resources Sign up for The Update QUESTIONS & ANSWER SESSION Nate.Weisenburger@ae2s.com

511 views • 3 slides

Examples of SAAS on cloud: dynamically scaling R, Galaxy and Matlab tom.visser@sara.nl

Examples of SAAS on cloud: dynamically scaling R, Galaxy and Matlab tom.visser@sara.nl linkedin.com/in/visservisser Our vision Cloud computing is not about new technology, it is about new uses of technology Self Service Dynamically Scalable

667 views • 18 slides

The Agile Advantage: How Agile Integration Improves Outcomes for Software as a Service (SaaS) in

The Agile Advantage: How Agile Integration Improves Outcomes for Software as a Service (SaaS) in the Corporate Environment By Greg Spehar MBA, PMP, PMI-ACP 1 The Need Acceleration of Migration of Applications to Cloud Based Solutions

475 views • 19 slides

Outline What is it? Why now? Cloud killer apps Economics for users Economics

UC Berkeley Above the Clouds A Berkeley View of Cloud Computing UC Berkeley RAD Lab Presentation at RPI, September 2011 1 Outline What is it? Why now? Cloud killer apps Economics for users Economics for providers

760 views • 29 slides