introduction to machine learning hyperparameter tuning
play

Introduction to Machine Learning Hyperparameter Tuning - Basic - PowerPoint PPT Presentation

Feb 03, 2023 •970 likes •1.06k views

Introduction to Machine Learning Hyperparameter Tuning - Basic Techniques compstat-lmu.github.io/lecture_i2ml GRID SEARCH Simple technique which is still quite popular, tries all HP combinations on a multi-dimensional discretized grid For

  1. Introduction to Machine Learning Hyperparameter Tuning - Basic Techniques compstat-lmu.github.io/lecture_i2ml

  2. GRID SEARCH Simple technique which is still quite popular, tries all HP combinations on a multi-dimensional discretized grid For each hyperparameter a finite set of candidates is predefined Then, we simply search all possible combinations in arbitrary order Grid search over 10x10 points ● ● ● ● ● ● 10 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 5 TestAccuracy Hyperparameter 2 ● ● ● ● ● ● ● ● ● ● 0.8 ● ● ● ● ● ● ● ● ● ● 0.7 0 ● ● ● ● ● ● ● ● ● ● 0.6 0.5 ● ● ● ● ● ● ● ● ● ● −5 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● −10 ● ● ● ● ● ● ● ● ● ● −10 −5 0 5 10 Hyperparameter 1 � c Introduction to Machine Learning – 1 / 6

  3. GRID SEARCH Advantages Very easy to implement All parameter types possible Parallelizing computation is trivial Disadvantages Scales badly: Combinatorial explosion Inefficient: Searches large irrelevant areas Arbitrary: Which values / discretization? � c Introduction to Machine Learning – 2 / 6

  4. RANDOM SEARCH Small variation of grid search Uniformly sample from the region-of-interest Random search over 100 points ● ● ● ● ● ● ● ● 5 ● ● ● TestAccuracy Hyperparameter 2 ● ● ● 0.8 0 ● ● 0.7 ● ● ● ● ● ● ● 0.6 ● ● ● 0.5 ● ● −5 ● ● ● ● ● ● ● ● ● ● ● −10 ● −10 −5 0 5 10 Hyperparameter 1 � c Introduction to Machine Learning – 3 / 6

  5. RANDOM SEARCH Advantages Like grid search: Very easy to implement, all parameter types possible, trivial parallelization Anytime algorithm: Can stop the search whenever our budget for computation is exhausted, or continue until we reach our performance goal. No discretization: each individual parameter is tried with a different value every time Disadvantages � c Introduction to Machine Learning – 4 / 6

  6. RANDOM SEARCH Inefficient: many evaluations in areas with low likelihood for improvement Scales badly: high dimensional hyperparameter spaces need lots of samples to cover. � c Introduction to Machine Learning – 5 / 6

  7. TUNING EXAMPLE Tuning random forest with random search and 5CV on the sonar data set for AUC: Parameter Type Min Max num.trees integer 3 500 mtry integer 5 50 integer 10 100 min.node.size Maximal AUC 0.93 0.92 0.91 0 50 100 150 Iterations � c Introduction to Machine Learning – 6 / 6

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Introduction to Machine Learning Hyperparameter Tuning - Introduction

Introduction to Machine Learning Hyperparameter Tuning - Introduction

Introduction to Machine Learning Hyperparameter Tuning - Introduction compstat-lmu.github.io/lecture_i2ml MOTIVATING EXAMPLE Given a dataset, we want to train a classification tree. We feel that a maximum tree depth of 4 has worked out well for

499 views • 7 slides
Introduction to Machine Learning Hyperparameter Tuning - Problem Definition

Introduction to Machine Learning Hyperparameter Tuning - Problem Definition

Introduction to Machine Learning Hyperparameter Tuning - Problem Definition compstat-lmu.github.io/lecture_i2ml TUNING Recall: Hyperparameters are parameters that are inputs to the training problem, in which a learner I minimizes the

504 views • 7 slides
Machine learning with H2O Dr. Shirin Glander Data Scientist DataCamp Hyperparameter Tuning in R

Machine learning with H2O Dr. Shirin Glander Data Scientist DataCamp Hyperparameter Tuning in R

DataCamp Hyperparameter Tuning in R HYPERPARAMETER TUNING IN R Machine learning with H2O Dr. Shirin Glander Data Scientist DataCamp Hyperparameter Tuning in R What is H2O? library(h2o) h2o.init() H2O is not running yet, starting it now...

773 views • 30 slides
Machine learning with mlr Dr. Shirin Elsinghorst Data Scientist DataCamp Hyperparameter Tuning

Machine learning with mlr Dr. Shirin Elsinghorst Data Scientist DataCamp Hyperparameter Tuning

DataCamp Hyperparameter Tuning in R HYPERPARAMETER TUNING IN R Machine learning with mlr Dr. Shirin Elsinghorst Data Scientist DataCamp Hyperparameter Tuning in R The mlr package mlr is another framework for machine learning in R. Model

454 views • 28 slides
Performance evaluation and hyperparameter tuning of statistical and machine-learning models using

Performance evaluation and hyperparameter tuning of statistical and machine-learning models using

Performance evaluation and hyperparameter tuning of statistical and machine-learning models using spatial data Patrick Schratz 1 , Jannes Muenchow 1 , Eugenia Iturritxa 2 , Jakob Richter 3 , Alexander Brenning 1 September 27 2018, 10th

632 views • 37 slides
Hyperparameter tuning in caret Dr. Shirin Glander Data Scientist DataCamp Hyperparameter

Hyperparameter tuning in caret Dr. Shirin Glander Data Scientist DataCamp Hyperparameter

DataCamp Hyperparameter Tuning in R HYPERPARAMETER TUNING IN R Hyperparameter tuning in caret Dr. Shirin Glander Data Scientist DataCamp Hyperparameter Tuning in R Voter dataset from US 2016 election Split intro training and test set

779 views • 22 slides
Introduction to Machine Learning Tuning: Nested Resampling Motivation

Introduction to Machine Learning Tuning: Nested Resampling Motivation

Introduction to Machine Learning Tuning: Nested Resampling Motivation compstat-lmu.github.io/lecture_i2ml MOTIVATION Selecting the best model from a set of potential candidates (e.g., different classes of learners, different hyperparameter

662 views • 9 slides
Performance evaluation and hyperparameter tuning of statistical and machine-learning models using

Performance evaluation and hyperparameter tuning of statistical and machine-learning models using

Performance evaluation and hyperparameter tuning of statistical and machine-learning models using spatial data P a tr ic k S ch r a tz 1 , J a nn e s M u e n ch ow 1 , J a ko b R ich t e r 2 , A l e x a n de r B r e nn i n g 1 GIS cie n ce S e m i

688 views • 32 slides
Hyperparameter Search in Machine Learning Marc Claesen and Bart De Moor

Hyperparameter Search in Machine Learning Marc Claesen and Bart De Moor

Introduction Example: optimizing hyperparameters for an SVM classifier Challenges in hyperparameter search State-of-the-art References Hyperparameter Search in Machine Learning Marc Claesen and Bart De Moor marc.claesen@esat.kuleuven.be

770 views • 57 slides
Deep Learning Hyperparameter Optimization with Competing Objectives GTC 2018 - S8136 Scott Clark

Deep Learning Hyperparameter Optimization with Competing Objectives GTC 2018 - S8136 Scott Clark

Deep Learning Hyperparameter Optimization with Competing Objectives GTC 2018 - S8136 Scott Clark scott@sigopt.com OUTLINE 1. Why is Tuning Models Hard? 2. Common Tuning Methods 3. Deep Learning Example 4. Tuning Multiple Metrics 5.

614 views • 43 slides
Hyperparameter Tuning in Python Using Optunity http://www.optunity.net Marc Claesen Jaak Simm

Hyperparameter Tuning in Python Using Optunity http://www.optunity.net Marc Claesen Jaak Simm

Introduction Optunity overview Example: tuning an SVM References Hyperparameter Tuning in Python Using Optunity http://www.optunity.net Marc Claesen Jaak Simm Dusan Popovic Bart De Moor ESAT-STADIUS, KU Leuven iMinds Medical IT Department

306 views • 29 slides
Asynchronous Hyperparameter Tuning and Ablation Studies with Apache Spark Sina Sheikholeslami

Asynchronous Hyperparameter Tuning and Ablation Studies with Apache Spark Sina Sheikholeslami

Asynchronous Hyperparameter Tuning and Ablation Studies with Apache Spark Sina Sheikholeslami Distributed Computing Group, KTH Royal Institute of Technology @cutlash CASTOR Software Days 2019 October 16 2019 sinash@kth.se The Machine

891 views • 43 slides
Parameters vs hyperparameters Dr. Shirin Glander Data Scientist DataCamp Hyperparameter Tuning

Parameters vs hyperparameters Dr. Shirin Glander Data Scientist DataCamp Hyperparameter Tuning

DataCamp Hyperparameter Tuning in R HYPERPARAMETER TUNING IN R Parameters vs hyperparameters Dr. Shirin Glander Data Scientist DataCamp Hyperparameter Tuning in R About me www.shirin-glander.de DataCamp Hyperparameter Tuning in R

610 views • 24 slides
Introduction to hyperparameter tuning MODEL VALIDATION IN P YTH ON Kasey Jones Data Scientist

Introduction to hyperparameter tuning MODEL VALIDATION IN P YTH ON Kasey Jones Data Scientist

Introduction to hyperparameter tuning MODEL VALIDATION IN P YTH ON Kasey Jones Data Scientist Model parameters Parameters are: Learned or estimated from the data The result of tting a model Used when making future predictions Not

1.82k views • 35 slides
Optimizer Benchmarking Needs to Account for Hyperparameter Tuning Prabhu Teja S * 1, 2 Florian Mai

Optimizer Benchmarking Needs to Account for Hyperparameter Tuning Prabhu Teja S * 1, 2 Florian Mai

Optimizer Benchmarking Needs to Account for Hyperparameter Tuning Prabhu Teja S * 1, 2 Florian Mai * 1, 2 Thijs Vogels 2 Martin Jaggi 2 Franois Fleuret 1, 2 1 Idiap Research Institute, 2 EPFL, Switzerland * Equal Contribution prabhu.teja,

282 views • 15 slides
Introduction to Machine Learning Tuning: Nested Resampling compstat-lmu.github.io/lecture_i2ml

Introduction to Machine Learning Tuning: Nested Resampling compstat-lmu.github.io/lecture_i2ml

Introduction to Machine Learning Tuning: Nested Resampling compstat-lmu.github.io/lecture_i2ml NESTED RESAMPLING Just like we can generalize holdout splitting to resampling to get more reliable estimates of the predictive performance, we can

545 views • 7 slides
Advanced Grid Modeling, Simulation, and Computation Building Research Collaborations: Electricity

Advanced Grid Modeling, Simulation, and Computation Building Research Collaborations: Electricity

Advanced Grid Modeling, Simulation, and Computation Building Research Collaborations: Electricity Systems Sven Leyffer , Cosmin Petra, and Steve Wright Argonne National Laboratory August 28-29, 2013 Overview of Challenges in Grid Modeling

396 views • 15 slides
Switching networks Switches and bulbs are 2-state components of electrical circuits.

Switching networks Switches and bulbs are 2-state components of electrical circuits.

519 views • 7 slides
Logic Circuits and Signals Hardware/Software Connection Corrado Santoro ARSLAB - Autonomous and

Logic Circuits and Signals Hardware/Software Connection Corrado Santoro ARSLAB - Autonomous and

Logic Circuits and Signals Hardware/Software Connection Corrado Santoro ARSLAB - Autonomous and Robotic Systems Laboratory Dipartimento di Matematica e Informatica - Universit` a di Catania, Italy santoro@dmi.unict.it L.S.M. Course Corrado

451 views • 19 slides
Depth Lower Bound against Circuits with Sparse Orientation Sajin Koroth 1 Jayalal Sarma 1 1

Depth Lower Bound against Circuits with Sparse Orientation Sajin Koroth 1 Jayalal Sarma 1 1

Depth Lower Bound against Circuits with Sparse Orientation Sajin Koroth 1 Jayalal Sarma 1 1 Algorithms and Complexity Theory Lab Department of Computer Science IIT Madras Chennai Theory Day, 2013 1 / 17 Circuit Model A circuit family C

880 views • 75 slides
Mesoscopic simulations of receptive lattices Limitation of deterministic approaches Limitation

Mesoscopic simulations of receptive lattices Limitation of deterministic approaches Limitation

Mesoscopic simulations of receptive lattices Limitation of deterministic approaches Limitation of deterministic approaches Continuous, deterministic models cant cope with: 1. Sensitivity to a very small number of molecules 2. Protein

1.14k views • 66 slides
Combinatorial Abstraction Refinement for Feasibility Analysis Martin Stigge Uppsala University,

Combinatorial Abstraction Refinement for Feasibility Analysis Martin Stigge Uppsala University,

Combinatorial Abstraction Refinement for Feasibility Analysis Martin Stigge Uppsala University, Sweden Joint work with Wang Yi Problem Overview Workload Model Scheduler Model Task A Task B EDF/Static Prio/... Task C Task A t Task B t

759 views • 51 slides
o Some remarks on the combinatorial Kalman filter R. Frhwirth Tracking meeting July 10,2015

o Some remarks on the combinatorial Kalman filter R. Frhwirth Tracking meeting July 10,2015

o Some remarks on the combinatorial Kalman filter R. Frhwirth Tracking meeting July 10,2015 R. Frhwirth 1 HEPHY Combinatorial Kalman filter Basics Combined track finding and fitting proposed for ZEUS by P . Billoir and S. Qian in

433 views • 4 slides
Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet? Gilles Perrouin

Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet? Gilles Perrouin

Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet? Gilles Perrouin gilles.perrouin@unamur.be @GPerrouin Journe GLE / LOUISE / RIMEL - GDR GPL CNRS - Talence 12 Avril 2019 gilles.perrouin@unamur.be 1 Uniform

361 views • 24 slides

More recommend