Overfitting Can Happen Overfitting Can Happen Overfitting Can - PowerPoint PPT Presentation

Nov 25, 2022 •107 likes •138 views

Overfitting Can Happen Overfitting Can Happen Overfitting Can Happen Overfitting Can Happen Overfitting Can Happen 30 25 test 20 error 15 (boosting stumps on heart-disease dataset) train 10 5 0 1 10 100 1000 # rounds

Overfitting Can Happen Overfitting Can Happen Overfitting Can Happen Overfitting Can Happen Overfitting Can Happen 30 25 test 20 error 15 (boosting “stumps” on heart-disease dataset) train 10 5 0 1 10 100 1000 # rounds
Actual Typical Run Actual Typical Run Actual Typical Run Actual Typical Run Actual Typical Run 20 15 error (boosting C4.5 on 10 “letter” dataset) test 5 train 0 10 100 1000 # of rounds ( T ) • test error does not increase, even after 1000 rounds • (total size > 2,000,000 nodes) • test error continues to drop even after training error is zero! # rounds 5 100 1000 train error 0 . 0 0 . 0 0 . 0 test error 8 . 4 3 . 3 3 . 1 • Occam’s razor wrongly predicts “simpler” rule is better
Empirical Evidence: The Margin Distribution Empirical Evidence: The Margin Distribution Empirical Evidence: The Margin Distribution Empirical Evidence: The Margin Distribution Empirical Evidence: The Margin Distribution • margin distribution = cumulative distribution of margins of training examples cumulative distribution 1.0 20 1000 100 15 error 0.5 10 test 5 5 train 0 10 100 1000 -1 -0.5 0.5 1 margin # of rounds ( T ) # rounds 5 100 1000 train error 0 . 0 0 . 0 0 . 0 test error 8 . 4 3 . 3 3 . 1 % margins ≤ 0 . 5 7 . 7 0 . 0 0 . 0 minimum margin 0 . 14 0 . 52 0 . 55

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Learning From Data Lecture 11 Overfitting What is Overfitting When does Overfitting Occur

Learning From Data Lecture 11 Overfitting What is Overfitting When does Overfitting Occur Stochastic and Deterministic Noise M. Magdon-Ismail CSCI 4100/6100 recap: Nonlinear Transforms X -space is R d d Z -space is R 1 1

373 views • 26 slides

The Problem of Overfitting The Problem of Overfitting BR data: neural network with 20%

The Problem of Overfitting The Problem of Overfitting BR data: neural network with 20% classification noise, 307 training examples Overfitting on BR (2) Overfitting on BR (2) Overfitting: h H overfits training set S if there exists H

551 views • 40 slides

Overfitting Validation process. Overfitting Ettore Lanzarone March 18, 2020 LESSON 3 Lesson 3

Lesson 3 MEDICAL SUPPORT SYSTEMS FOR CHRONIC DISEASES Engineering and Management for Health University of Bergamo Overfitting Validation process. Overfitting Ettore Lanzarone March 18, 2020 LESSON 3 Lesson 3 Overfitting Linear

510 views • 23 slides

Overfitting Many hypotheses consistent with/close to the data About this class With enough

Overfitting Many hypotheses consistent with/close to the data About this class With enough features and a rich enough hypothesis space, it becomes easy to find mean- ingless regularity in the data The problem of overfitting and how to deal

197 views • 6 slides

4 2 5 6 6 7 1 Overfitting Overfitting 2 Examples Attributes Sometimes, model

Last Time ML II Decision trees and how to build them What might Information Gain we learn from these Entropy examples? Next up: Elements of a Learning System What can go wrong? How do we know how it went? 1 2

480 views • 6 slides

CSE 446: Week 3: Decision Trees (Apr 4) Instructor: Sergey Levine I. Overfitting idea 1: holdout

CSE 446: Week 3: Decision Trees (Apr 4) Instructor: Sergey Levine I. Overfitting idea 1: holdout crossvalidation What if we could test for overfitting directly while building our tree? Recall what I mentioned about a holdout set in lecture.

68 views • 4 slides

The Paradox of Overfitting Volker Nannen February 1, 2003 Artificial Intelligence

The Paradox of Overfitting Volker Nannen February 1, 2003 Artificial Intelligence Rijksuniversiteit Groningen Contents 1. MDL theory 2. Experimental Verification 3. Results MDL theory 1.1 the problem The paradox of overfitting:

394 views • 38 slides

Regularization The problem of overfitting Machine Learning Example: Linear regression (housing

Regularization The problem of overfitting Machine Learning Example: Linear regression (housing prices) Price Price Price Size Size Size Overfitting: If we have too many features, the learned hypothesis may fit the training set very well (

434 views • 24 slides

Advanced Classification; Overfitting and regularization; From .R to Notebooks Structure of the

Prof. Anton Ovchinnikov Prof. Spyros Zoumpoulis DSB Sessions 7-8, February 7, 2020 Advanced Classification; Overfitting and regularization; From .R to Notebooks Structure of the course SESSIONS 1-2 (AO): Data analytics process; from Excel

639 views • 29 slides

GMM & EM Last time summary Normalization Bias-Variance trade-off Overfitting and

GMM & EM Last time summary Normalization Bias-Variance trade-off Overfitting and underfitting MLE vs MAP estimate How to use the prior LRT (Bayes Classifier) Nave Bayes A simple decision rule If we can know

471 views • 35 slides

Class 2 & 3 Overfitting & Regularization Carlo Ciliberto Department of Computer Science,

Class 2 & 3 Overfitting & Regularization Carlo Ciliberto Department of Computer Science, UCL October 18, 2017 Last Class The goal of Statistical Learning Theory is to find a good estimator f n : X Y , approximating the lowest

567 views • 42 slides

Data Mining Lecture 05: Overfitting Evaluation: accuracy, precision, recall, ROC

CISC 4631 Data Mining Lecture 05: Overfitting Evaluation: accuracy, precision, recall, ROC Theses slides are based on the slides by Tan, Steinbach and Kumar (textbook authors) Eamonn Koegh (UC Riverside) Raymond Mooney

748 views • 62 slides

Machine Learning (CSE 446): (continuation of overfitting &) Limits of Learning Sham M Kakade

Machine Learning (CSE 446): (continuation of overfitting &) Limits of Learning Sham M Kakade 2018 c University of Washington cse446-staff@cs.washington.edu 1 / 17 Announcement Qz section tomo: (basic) probability and linear

1.03k views • 20 slides

Summary Overfitting arises when we evaluate and train on the same data. We can bound error

Summary Overfitting arises when we evaluate and train on the same data. We can bound error of a fixed function with Hoeffdings inequality. Next lecture well get a version sensitive to function class size. 41 / 61 Part 3. . .

1.08k views • 77 slides

recap: Overfitting Fitting the data more than is warranted Learning From Data Data Lecture 12

recap: Overfitting Fitting the data more than is warranted Learning From Data Data Lecture 12 Target Regularization Fit Constraining the Model Weight Decay Augmented Error y M. Magdon-Ismail CSCI 4100/6100 x A M Regularization : 2

97 views • 8 slides

Outline IAML: Overfitting and Capacity Control Generalization error Estimating

Outline IAML: Overfitting and Capacity Control Generalization error Estimating generalization error Example: polynomial regression Charles Sutton and Victor Lavrenko School of Informatics Under- and over-fitting

305 views • 7 slides

Locality lower bounds through round elimination D 1 Jukka Suomela U v 1 D 3 u Aalto

Locality lower bounds through round elimination D 1 Jukka Suomela U v 1 D 3 u Aalto University, Finland v 3 v 2 D 2 1 Joint work with Alkida Balliu Tuomo Lempiinen Sebastian Brandt Dennis Olivetti Orr Fischer

581 views • 57 slides

CS 204: Scheduling Jiasi Chen Lectures: MWF 12:10-1pm in WCH 139

CS 204: Scheduling Jiasi Chen Lectures: MWF 12:10-1pm in WCH 139 http://www.cs.ucr.edu/~jiasi/teaching/cs204_spring16/ 1 Overview What is scheduling? Round robin Q: How should a Weighted round robin common resource be

558 views • 24 slides

CS 5 4 3 : Com puter Graphics Lecture 9 ( Part I ) : Raster Graphics: Draw ing Lines Emmanuel

CS 5 4 3 : Com puter Graphics Lecture 9 ( Part I ) : Raster Graphics: Draw ing Lines Emmanuel Agu 2 D Graphics Pipeline Clipping window to Object Applying Object viewport subset world window World Coordinates mapping Simple 2D Drawing

536 views • 25 slides

Lecture 7 Transport Protocols: UDP, TCP EECS 122 University of California Berkeley TOC:

Lecture 7 Transport Protocols: UDP, TCP EECS 122 University of California Berkeley TOC: Transport Protocols Why? Overview UDP TCP Summary EECS 122 Walrand 2 Transport: Why? IP provides a weak, but efficient service model

401 views • 37 slides

W ORK R OUNDS S MALL G ROUP T EACHING S MALL G ROUP T EACHING B ringing B ringing E

W ORK R OUNDS W ORK R OUNDS S MALL G ROUP T EACHING S MALL G ROUP T EACHING B ringing B ringing E ducation & E ducation & S ervice S ervice T ogether T ogether Learning Objective Learning Objective After participating

190 views • 14 slides

Improved Differential-Linear Cryptanalysis of 7-round Chaskey with Partitioning Gatan Leurent

Chaskey ARX Cryptanalysis Improved Differential-Linear Conclusion Improved Differential-Linear Cryptanalysis of 7-round Chaskey with Partitioning Gatan Leurent Inria, Paris Eurocrypt 2016 m 0 m 1 m 2 K K K Gatan

540 views • 36 slides

Optimal Non-parametric Learning in Repeated Contextual Auctions with Strategic Buyer Alexey

Optimal Non-parametric Learning in Repeated Contextual Auctions with Strategic Buyer Alexey Drutsa Setup Repeated Contextual Posted-Price Auctions Different goods (e.g., ad spaces) described by -dimensional feature vectors (contexts)

377 views • 19 slides

Reserve Pricing in Repeated Second-Price Auctions with Strategic Bidders Alexey Drutsa Setup

Reserve Pricing in Repeated Second-Price Auctions with Strategic Bidders Alexey Drutsa Setup Second-Price (SP) Auction with Reserve Prices Setting A good (e.g., an ad space) is offered for sale by a seller to buyers Each buyer

602 views • 31 slides