learning from data lecture 14 three learning principles
play

Learning From Data Lecture 14 Three Learning Principles Occams - PowerPoint PPT Presentation

Aug 17, 2023 •420 likes •1.01k views

Learning From Data Lecture 14 Three Learning Principles Occams Razor Sampling Bias Data Snooping M. Magdon-Ismail CSCI 4100/6100 recap: Validation and Cross Validation Validation Cross Validation D ( N ) D 1 D 2 D N D train D

  1. Learning From Data Lecture 14 Three Learning Principles Occam’s Razor Sampling Bias Data Snooping M. Magdon-Ismail CSCI 4100/6100

  2. recap: Validation and Cross Validation Validation Cross Validation D ( N ) D 1 D 2 · · · D N D train D ( N − K ) · · · g 1 g 2 g N ( x 1 , y 1 ) ( x 2 , y 2 ) ( x N , y N ) g D val · · · ( K ) e 1 e 2 e N � �� � take average g E val ( g ) g E cv · · · H 1 H 2 H 3 H M Model Selection − − − − − − − − − − − − → → → → · · · g 1 g 2 g 3 g M M Three Learning Principles : 2 /58 � A c L Creator: Malik Magdon-Ismail Occam, bias, snooping − →

  3. We Will Discuss . . . • Occam’s Razor : pick a model carefully • Sampling Bias : generate the data carefuly • Data Snooping : handle the data carefully M Three Learning Principles : 3 /58 � A c L Creator: Malik Magdon-Ismail Occam’s Razor − →

  4. Occam’s Razor M Three Learning Principles : 4 /58 � A c L Creator: Malik Magdon-Ismail Occam − →

  5. Occam’s Razor use a ‘ razor ’ to ‘trim down’ “an explanation of the data to make it as simple as possible but no simpler.” attributed to William of Occam (14th Century) and often mistakenly to Einstein M Three Learning Principles : 5 /58 � A c L Creator: Malik Magdon-Ismail Simpler is Better − →

  6. Simpler is Better The simplest model that fits the data is also the most plausible. . . . or, beware of using complex models to fit data M Three Learning Principles : 6 /58 � A c L Creator: Malik Magdon-Ismail What is Simpler? − →

  7. What is Simpler? simple hypothesis h simple hypothesis set H Ω( h ) Ω( H ) low order polynomial H with small d vc hypothesis with small weights small number of hypotheses easily described hypothesis low entropy set . . . . . . The equivalence: A hypothesis set with simple hypotheses must be small We had a glimpse of this: λ soft order constraint (smaller H ) → minimize E aug (favors simpler h ) . ← − − − − M Three Learning Principles : 7 /58 � A c L Creator: Malik Magdon-Ismail What is Simpler? − →

  8. What is Simpler? simple hypothesis h simple hypothesis set H Ω( h ) Ω( H ) low order polynomial H with small d vc hypothesis with small weights small number of hypotheses easily described hypothesis low entropy set . . . . . . The equivalence: A hypothesis set with simple hypotheses must be small We had a glimpse of this: λ soft order constraint (smaller H ) → minimize E aug (favors simpler h ) . ← − − − − M Three Learning Principles : 8 /58 � A c L Creator: Malik Magdon-Ismail What is Simpler? − →

  9. What is Simpler? simple hypothesis h simple hypothesis set H Ω( h ) Ω( H ) low order polynomial H with small d vc hypothesis with small weights small number of hypotheses easily described hypothesis low entropy set . . . . . . The equivalence: A hypothesis set with simple hypotheses must be small We had a glimpse of this: λ soft order constraint (smaller H ) ← − − − − → minimize E aug (favors simpler h ) . M Three Learning Principles : 9 /58 � A c L Creator: Malik Magdon-Ismail Why is Simpler Better − →

  10. Why is Simpler Better Mathematically: simple curtails ability to fit noise, VC-dimension is small, and blah and blah . . . simpler is better because you will be more “surprised” when you fit the data. If something unlikely happens, it is very significant when it happens. . . . “Is there any other point to which you would wish to draw my attention?” Detective Gregory: Sherlock Holmes: “To the curious incident of the dog in the night-time.” “The dog did nothing in the night-time.” Detective Gregory: “That was the curious incident.” Sherlock Holmes: . . . – Silver Blaze , Sir Arthur Conan Doyle M Three Learning Principles : 10 /58 � A c L Creator: Malik Magdon-Ismail Scientific Experiment − →

  11. A Scientific Experiment Axiom. If an experiment has no chance of falsifying a hypothesis, then the result of that experiment provides no evidence one way or the other for the hypothesis. Scientist 3 resistivity ρ temperature T no evidence very convincing some evidence? Who provides most evidence for the hypothesis “ ρ is linear in T ”? M Three Learning Principles : 11 /58 � A c L Creator: Malik Magdon-Ismail Scientific Experiment − →

  12. A Scientific Experiment Axiom. If an experiment has no chance of falsifying a hypothesis, then the result of that experiment provides no evidence one way or the other for the hypothesis. Scientist 2 Scientist 3 resistivity ρ resistivity ρ temperature T temperature T no evidence very convincing some evidence? Who provides most evidence for the hypothesis “ ρ is linear in T ”? M Three Learning Principles : 12 /58 � A c L Creator: Malik Magdon-Ismail Scientific Experiment − →

  13. A Scientific Experiment Axiom. If an experiment has no chance of falsifying a hypothesis, then the result of that experiment provides no evidence one way or the other for the hypothesis. Scientist 1 Scientist 2 Scientist 3 resistivity ρ resistivity ρ resistivity ρ temperature T temperature T temperature T no evidence very convincing some evidence? Who provides most evidence for the hypothesis “ ρ is linear in T ”? M Three Learning Principles : 13 /58 � A c L Creator: Malik Magdon-Ismail Scientific Experiment − →

  14. A Scientific Experiment Axiom. If an experiment has no chance of falsifying a hypothesis, then the result of that experiment provides no evidence one way or the other for the hypothesis. Scientist 1 Scientist 2 Scientist 3 resistivity ρ resistivity ρ resistivity ρ temperature T temperature T temperature T no evidence very convincing some evidence? Who provides most evidence for the hypothesis “ ρ is linear in T ”? M Three Learning Principles : 14 /58 � A c L Creator: Malik Magdon-Ismail Scientist 2 vs. 3 − →

  15. Scientist 2 Versus Scientist 3 Axiom. If an experiment has no chance of falsifying a hypothesis, then the result of that experiment provides no evidence one way or the other for the hypothesis. Scientist 1 Scientist 2 Scientist 3 resistivity ρ resistivity ρ resistivity ρ temperature T temperature T temperature T no evidence very convincing some evidence? Who provides most evidence? M Three Learning Principles : 15 /58 � A c L Creator: Malik Magdon-Ismail Scientist 1 vs. 3 − →

  16. Scientist 1 versus Scientist 3 Axiom. If an experiment has no chance of falsifying a hypothesis, then the result of that experiment provides no evidence one way or the other for the hypothesis. Scientist 1 Scientist 2 Scientist 3 resistivity ρ resistivity ρ resistivity ρ temperature T temperature T temperature T no evidence very convincing some evidence? Who provides most evidence? M Three Learning Principles : 16 /58 � A c L Creator: Malik Magdon-Ismail Non-Falsifiability − →

  17. Axiom of Non-Falsifiability Axiom. If an experiment has no chance of falsifying a hypothesis, then the result of that experiment provides no evidence one way or the other for the hypothesis. Scientist 1 Scientist 2 Scientist 3 resistivity ρ resistivity ρ temperature T temperature T no evidence very convincing some evidence? Who provides most evidence? M Three Learning Principles : 17 /58 � A c L Creator: Malik Magdon-Ismail Falsification and m H ( N ) − →

  18. Falsification and m H ( N ) If H shatters x 1 , · · · , x N , – Don’t be surprised if you fit the data. – Can’t falsify “ H is a good set of candidate hypotheses for f ”. If H doesn’t shatter x 1 , · · · , x N , and the target values are uniformly distributed, P [falsification] ≥ 1 − m H ( N ) . 2 N A good fit is surprising with simple H , hence significant. You can, but didn’t falsify “ H is a good set of candidate hypotheses for f ” The data must have a chance to win. M Three Learning Principles : 18 /58 � A c L Creator: Malik Magdon-Ismail Falsification and m H ( N ) − →

  19. Falsification and m H ( N ) If H shatters x 1 , · · · , x N , – Don’t be surprised if you fit the data. – Can’t falsify “ H is a good set of candidate hypotheses for f ”. If H doesn’t shatter x 1 , · · · , x N , and the target values are uniformly distributed, P [falsification] ≥ 1 − m H ( N ) . 2 N A good fit is surprising with simple H , hence significant. You can, but didn’t falsify “ H is a good set of candidate hypotheses for f ” The data must have a chance to win. M Three Learning Principles : 19 /58 � A c L Creator: Malik Magdon-Ismail Falsification and m H ( N ) − →

  20. Falsification and m H ( N ) If H shatters x 1 , · · · , x N , – Don’t be surprised if you fit the data. – Can’t falsify “ H is a good set of candidate hypotheses for f ”. If H doesn’t shatter x 1 , · · · , x N , and the target values are uniformly distributed, P [falsification] ≥ 1 − m H ( N ) . 2 N A good fit is surprising with simple H , hence significant. You can, but didn’t falsify “ H is a good set of candidate hypotheses for f ” The data must have a chance to win. M Three Learning Principles : 20 /58 � A c L Creator: Malik Magdon-Ismail Beyond Occam − →

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Learning From Data Lecture 2 The Perceptron The Learning Setup A Simple Learning Algorithm: PLA

Learning From Data Lecture 2 The Perceptron The Learning Setup A Simple Learning Algorithm: PLA

Learning From Data Lecture 2 The Perceptron The Learning Setup A Simple Learning Algorithm: PLA Other Views of Learning Is Learning Feasible: A Puzzle M. Magdon-Ismail CSCI 4100/6100 recap: The Plan 1. What is Learning? 2. Can We do it?

613 views • 25 slides
Learning From Data Lecture 3 Is Learning Feasible? Outside the Data Probability to the Rescue

Learning From Data Lecture 3 Is Learning Feasible? Outside the Data Probability to the Rescue

Learning From Data Lecture 3 Is Learning Feasible? Outside the Data Probability to the Rescue Learning vs. Verification Selection Bias - A Cartoon M. Magdon-Ismail CSCI 4100/6100 recap: The Perceptron Learning Algorithm y = +1 y x

538 views • 27 slides
Learning From Data Lecture 15 Reflecting on Our Path - Epilogue to Part I What We Did The

Learning From Data Lecture 15 Reflecting on Our Path - Epilogue to Part I What We Did The

Learning From Data Lecture 15 Reflecting on Our Path - Epilogue to Part I What We Did The Machine Learning Zoo Moving Forward M. Magdon-Ismail CSCI 4100/6100 recap: Three Learning Principles Scientist 1 Scientist 2 Scientist 3 resistivity

392 views • 11 slides
CSE217 INTRODUCTION TO DATA SCIENCE LECTURE 6: LEARNING PRINCIPLES Spring 2019 Marion Neumann

CSE217 INTRODUCTION TO DATA SCIENCE LECTURE 6: LEARNING PRINCIPLES Spring 2019 Marion Neumann

CSE217 INTRODUCTION TO DATA SCIENCE LECTURE 6: LEARNING PRINCIPLES Spring 2019 Marion Neumann RECAP: MACHINE LEARNING Workflow 2 NOISE noisy samples from true function 3 WHY IS NOISE A PROBLEM? small random sample from the noisy

746 views • 19 slides
Occams Razor Sampling Bias : generate the data carefuly Data Snooping : handle the data

Occams Razor Sampling Bias : generate the data carefuly Data Snooping : handle the data

recap: Validation and Cross Validation Validation Cross Validation D ( N ) D 1 D 2 D N D train Learning From Data D ( N K ) g 1 g 2 g N Lecture 14 ( x 1 , y 1 ) ( x 2 , y 2 ) ( x N , y N ) Three Learning Principles D

486 views • 15 slides
Learning Aides Additional tools that can be applied to all techniques Learning From Data Lecture

Learning Aides Additional tools that can be applied to all techniques Learning From Data Lecture

Learning Aides Additional tools that can be applied to all techniques Learning From Data Lecture 27 Learning Aides Input Preprocessing Dimensionality Reduction and Feature Selection Preprocess data to account for arbitrary choices during data

430 views • 4 slides
Learning From Data Lecture 27 Learning Aides Input Preprocessing Dimensionality Reduction and

Learning From Data Lecture 27 Learning Aides Input Preprocessing Dimensionality Reduction and

Learning From Data Lecture 27 Learning Aides Input Preprocessing Dimensionality Reduction and Feature Selection Principal Components Analysis (PCA) Hints, Data Cleaning, Validation, . . . M. Magdon-Ismail CSCI 4100/6100 Learning Aides

271 views • 16 slides
Learning From Data Lecture 25 The Kernel Trick Learning with only inner products The Kernel M.

Learning From Data Lecture 25 The Kernel Trick Learning with only inner products The Kernel M.

Learning From Data Lecture 25 The Kernel Trick Learning with only inner products The Kernel M. Magdon-Ismail CSCI 4100/6100 recap: Large Margin is Better Controling Overfitting Non-Separable Data 0.08 random hyperplane 2 w t w + C N E

576 views • 18 slides
Learning From Data Lecture 19 A Peek At Unsupervised Learning k -Means Clustering Probability

Learning From Data Lecture 19 A Peek At Unsupervised Learning k -Means Clustering Probability

Learning From Data Lecture 19 A Peek At Unsupervised Learning k -Means Clustering Probability Density Estimation Gaussian Mixture Models M. Magdon-Ismail CSCI 4100/6100 recap: Radial Basis Functions Nonparametric RBF Parametric k -RBF-Network

253 views • 23 slides
Learning From Data Lecture 1 The Learning Problem Introduction Motivation Credit Default - A

Learning From Data Lecture 1 The Learning Problem Introduction Motivation Credit Default - A

Learning From Data Lecture 1 The Learning Problem Introduction Motivation Credit Default - A Running Example Summary of the Learning Problem M. Magdon-Ismail CSCI 4100/6100 Resources 1. Web Page: www.cs.rpi.edu/ magdon/courses/learn.php

499 views • 16 slides
Learning of concepts and principles of IT Aim Determine the level of mastery of the

Learning of concepts and principles of IT Aim Determine the level of mastery of the

INF3280 7 February 2014 Learning of concepts and principles of IT Aim Determine the level of mastery of the learning goals for Assignment 2 according to the learning model Core literature: Chapter 5. Learning IT

260 views • 12 slides
Learning From Data Lecture 5 Training Versus Testing The Two Questions of Learning Theory of

Learning From Data Lecture 5 Training Versus Testing The Two Questions of Learning Theory of

Learning From Data Lecture 5 Training Versus Testing The Two Questions of Learning Theory of Generalization ( E in E out ) An Effective Number of Hypotheses A Combinatorial Puzzle M. Magdon-Ismail CSCI 4100/6100 recap: The Two Questions

700 views • 18 slides
CSE 255 Lecture 2 Data Mining and Predictive Analytics Supervised learning Regression

CSE 255 Lecture 2 Data Mining and Predictive Analytics Supervised learning Regression

CSE 255 Lecture 2 Data Mining and Predictive Analytics Supervised learning Regression Supervised versus unsupervised learning Learning approaches attempt to model data in order to solve a problem Unsupervised learning approaches find

535 views • 50 slides
CSE 190 Lecture 2 Data Mining and Predictive Analytics Supervised learning Regression

CSE 190 Lecture 2 Data Mining and Predictive Analytics Supervised learning Regression

CSE 190 Lecture 2 Data Mining and Predictive Analytics Supervised learning Regression Supervised versus unsupervised learning Learning approaches attempt to model data in order to solve a problem Unsupervised learning approaches find

822 views • 50 slides
Cognitive Principles in Tutor & Cognitive Tutor Principles e-Learning Design

Cognitive Principles in Tutor & Cognitive Tutor Principles e-Learning Design

Lots of Lists of Principles 1 Cognitive Principles in Tutor & Cognitive Tutor Principles e-Learning Design Koedinger, K. R. & Corbett, A. T. (2006). Cognitive Tutors: Technology bringing learning science to the classroom.

325 views • 13 slides
Supervised Learning: The Setup Machine Learning 1 Last lecture We saw What is learning?

Supervised Learning: The Setup Machine Learning 1 Last lecture We saw What is learning?

Supervised Learning: The Setup Machine Learning 1 Last lecture We saw What is learning? Learning as generalization The badges game 2 This lecture More badges Formalizing supervised learning Instance space and features

1.04k views • 80 slides
Behavioral Health Health Information Technology Learning Collaborative We will start the event

Behavioral Health Health Information Technology Learning Collaborative We will start the event

Behavioral Health Health Information Technology Learning Collaborative We will start the event momentarily. While you wait, please respond to our icebreaker poll. Learning Collaborative Audience 184 registrants 102 organizations

771 views • 75 slides
General Track: Telehealth in Physical Therapy From the computer screen to the Clinic (Part I

General Track: Telehealth in Physical Therapy From the computer screen to the Clinic (Part I

General Track: Telehealth in Physical Therapy From the computer screen to the Clinic (Part I and II) Sarah Gallagher, PT, DPT, NCS Alan Chong W. Lee, PT, PhD, DPT, GCS Date and Time of Presentation: October 27, 2018: 9 6 pm Course

577 views • 35 slides
Webinar Recording: COVID-19 and the CV Service Line: Setting up Telehealth in Your Office - Part

Webinar Recording: COVID-19 and the CV Service Line: Setting up Telehealth in Your Office - Part

Webinar Recording: COVID-19 and the CV Service Line: Setting up Telehealth in Your Office - Part 2 Q&A Question Asked Qu An Answer Given Is there any difference on billing for a provider based clinic? If the patient is at home - no

226 views • 4 slides
-arrestins in GPCR Desensitization How Lisp Will Save the World 15,596 abstracts 15

-arrestins in GPCR Desensitization How Lisp Will Save the World 15,596 abstracts 15

-arrestins in GPCR Desensitization How Lisp Will Save the World 15,596 abstracts 15 abstracts 15,596,125 abstracts Cyanobacteria http://www.anselm.edu/homepage/jpitocch/genbios/27-11x1-Cyanobacteria.jpg Prochlorococcus

990 views • 61 slides
What can a 1980s BASIC programming textbook teach us today? Martin Lester Department of

What can a 1980s BASIC programming textbook teach us today? Martin Lester Department of

What can a 1980s BASIC programming textbook teach us today? Martin Lester Department of Computer Science, University of Oxford History and Philosophy of Programming, 20180323 Motivation Elementary Basic: Learning to Program your

306 views • 28 slides
C lt Cultural Heritage in CLEF (CHiC) 2012 l H it i CLEF (CHiC) 2012 Pilot Lab Overview

C lt Cultural Heritage in CLEF (CHiC) 2012 l H it i CLEF (CHiC) 2012 Pilot Lab Overview

C lt Cultural Heritage in CLEF (CHiC) 2012 l H it i CLEF (CHiC) 2012 Pilot Lab Overview Pilot Lab Overview Vivien Petras Humboldt-Universitt zu Berlin Roma, 17. September 2012 Contents x Cultural Heritage Information Systems

334 views • 18 slides
3/23/2014 Stage classification % at 5-year survival diagnosis Localized 8 20% Treatment of

3/23/2014 Stage classification % at 5-year survival diagnosis Localized 8 20% Treatment of

3/23/2014 Stage classification % at 5-year survival diagnosis Localized 8 20% Treatment of locally advanced and borderline resectable pancreatic cancer: Does radiation Locally 31 8% have a role? advanced/ unresectable Metastatic 61

558 views • 7 slides
Pathways analysis in proteomics Angela Bachi Dibit-San Raffaele Scientific Institute, Milano

Pathways analysis in proteomics Angela Bachi Dibit-San Raffaele Scientific Institute, Milano

Pathways analysis in proteomics Angela Bachi Dibit-San Raffaele Scientific Institute, Milano Pathways analysis in proteomics the input is the expression proteomics data and the output is the list of activated or dominant pathways in a given

667 views • 30 slides

More recommend