neural networks for predicting algorithm runtime
play

Neural Networks for Predicting Algorithm Runtime Distributions - PowerPoint PPT Presentation

Jan 18, 2023 •301 likes •588 views

Neural Networks for Predicting Algorithm Runtime Distributions Katharina Eggensperger, Marius Lindauer & Frank Hutter Paper ID #2772 Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 IJCAI2018 Motivation

  1. Neural Networks for Predicting Algorithm Runtime Distributions Katharina Eggensperger, Marius Lindauer & Frank Hutter Paper ID #2772 Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 IJCAI’2018

  2. Motivation Algorithm portfolios yield state-of-the-art performance for SAT, ASP, Planning, … → to build these we can make use of runtime predictions Other applications: ● Optimal restarts ● Algorithm selection ● Algorithm configurations Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 2 IJCAI’2018

  3. Describing the Runtime of an Algorithm? solve(instance, seed): # do something return solution, runtime runtime Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 3 IJCAI’2018

  4. Describing the Runtime of an Algorithm? solve(instance, seed): # do something return solution, runtime runtime Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 3 IJCAI’2018

  5. Contributions 1 Study how to predict parametric RTDs 2 Propose DistNet , a practical neural network for predicting RTDs 3 Evaluate DistNet and show that it can learn from only a few samples per instance Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 4 IJCAI’2018

  6. Typical Pipeline for Runtime prediction Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 5 IJCAI’2018

  7. Typical Pipeline for Runtime prediction Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 5 IJCAI’2018

  8. Typical Pipeline for Runtime prediction Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 5 IJCAI’2018

  9. Typical Pipeline for Runtime prediction Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 5 IJCAI’2018

  10. Typical Pipeline for Runtime prediction Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 6 IJCAI’2018

  11. Empirical RTDs Clasp-factoring LPG-Zenotravel SAPS-CV-VAR p(solved by this time) Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 7 IJCAI’2018

  12. Typical Pipeline for Runtime prediction Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 8 IJCAI’2018

  13. Considered Parametric Distribution Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 9 IJCAI’2018

  14. Quantifying the Quality of Runtime Distributions (1) observed runtimes distribution parameter Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 10 IJCAI’2018

  15. Quantifying the Quality of Runtime Distributions (1) observed runtimes distribution parameter (2) Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 10 IJCAI’2018

  16. Typical Pipeline for Runtime prediction Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 11 IJCAI’2018

  17. Predicting multiple Runtime Distributions Option 1 For each training instance → fit the parametric distribution’s parameter on observed runtimes. Then for all training instances, for each distribution parameter: fit a model Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 12 IJCAI’2018

  18. Predicting multiple Runtime Distributions Option 1 For each training instance → fit the parametric distribution’s parameter on observed runtimes. Then for all training instances, for each distribution parameter: fit a model Problematic, because models ● can only be as good as each fitted distribution ● do not know about interaction between their outputs ● typically minimize loss in the parameter space Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 12 IJCAI’2018

  19. Predicting multiple Runtime Distributions Option 2 For each training instance → fit the parametric distribution’s parameter on observed runtimes. Then for all training instances, for each distribution parameter: fit a model with multiple outputs Problematic, because model ● can only be as good as each fitted distribution ● does not know about interaction between their outputs ● typically minimizes loss in the parameter space Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 12 IJCAI’2018

  20. Predicting multiple Runtime Distributions DistNet For each training instance → fit the parametric distribution’s parameter on observed runtimes. Then for all training instances, for each distribution parameter: fit a neural network using negative log-likelihood as a loss function Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 12 IJCAI’2018

  21. Results We compared ● DistNet ● independent Random Forests (iRF) ● multi-output Random Forests (mRF) on 7 scenarios from SAT solving and AI planning. [...] Table : Averaged negative log-likelihood achieved for predicting RTDs for Figure : Averaged negative log-likelihood. Smaller values are better. unseen instances Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 16 IJCAI’2018

  22. Results We compared ● DistNet ● independent Random Forests (iRF) ● multi-output Random Forests (mRF) on 7 scenarios from SAT solving and AI planning. [...] Table : Averaged negative log-likelihood achieved for predicting RTDs for Figure : Averaged negative log-likelihood. Smaller values are better. unseen instances Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 16 IJCAI’2018

  23. Results We compared ● DistNet ● independent Random Forests (iRF) ● multi-output Random Forests (mRF) on 7 scenarios from SAT solving and AI planning. [...] → Predicting parameters for RTDs is possible → Joint predictions work better → DistNet provides more robust predictions which are often better than those of Table : Averaged negative log-likelihood achieved for predicting RTDs for Figure : Averaged negative log-likelihood. Smaller values are better. unseen instances competitors Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 16 IJCAI’2018

  24. DistNet on a Low Number of Observations multi-ouput Random Forest averaged NLLH DistNet Distribution fitted on all samples Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 17 IJCAI’2018

  25. Wrap-Up We have proposed DistNet, which + jointly learns distribution parameters + directly optimizes the loss function of interest + performs well even if only few observations per instance are available Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 18 IJCAI’2018

  26. Wrap-Up We have proposed DistNet, which + jointly learns distribution parameters + directly optimizes the loss function of interest + performs well even if only few observations per instance are available Open Questions: - How to automatically determine a well fitting distribution family? - How to handle heterogeneous datasets? Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 18 IJCAI’2018

  27. Wrap-Up We have proposed DistNet, which + jointly learns distribution parameters + directly optimizes the loss function of interest + performs well even if only few observations per instance are available Open Questions: - How to automatically determine a well fitting distribution family? - How to handle heterogeneous datasets? Code and data: https://www.automl.org/distnet/ Eggensperger, Lindauer and Hutter DistNet: Runtime Distribution Prediction #2772 18 IJCAI’2018

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Computational Algorithm Predicting Surface Computational Algorithm Predicting Surface Morphology

Computational Algorithm Predicting Surface Computational Algorithm Predicting Surface Morphology

Computational Algorithm Predicting Surface Computational Algorithm Predicting Surface Morphology Evolution During Electropolishing Joel Thomas 1 , Charles Reece 2 and Stanko R. Brankovic 1,* 1 Cullen College of Engineering University of Houston

399 views • 12 slides
How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . .

How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . .

Need for Machine . . . Neural Networks . . . How to Speed Up . . . Neural Networks: A . . . How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . . . Resulting Algorithm Faster by Taking Into How to

464 views • 17 slides
Radiance-Predicting Neural Networks SIMON KALLWEIT, Disney Research and ETH Zrich et al. ACM

Radiance-Predicting Neural Networks SIMON KALLWEIT, Disney Research and ETH Zrich et al. ACM

Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting Neural Networks SIMON KALLWEIT, Disney Research and ETH Zrich et al. ACM Transactions on Graphics, Publication date: November 2017 Presenter: MinKu Kang In Previous Talk

911 views • 21 slides
Predicting Temporal Sets with Deep Neural Networks Le Yu, Leilei Sun*, Bowen Du, Chuanren Liu, Hui

Predicting Temporal Sets with Deep Neural Networks Le Yu, Leilei Sun*, Bowen Du, Chuanren Liu, Hui

Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 20), August 23 27, 2020, Virtual Event, CA, USA. Predicting Temporal Sets with Deep Neural Networks Le Yu, Leilei Sun*, Bowen Du, Chuanren Liu, Hui

440 views • 18 slides
Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions Jimmy Lei Ba,

Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions Jimmy Lei Ba,

Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions Jimmy Lei Ba, Kevin Swersky, Sanja Fidler, Ruslan Salakhutdinov ICCV 2015 Presenter: Fartash Faghri Zero-shot Learning Classify images of an unseen class

156 views • 12 slides
DeepTMA: Predicting Effective Contention Models for Network Calculus using Graph Neural Networks

DeepTMA: Predicting Effective Contention Models for Network Calculus using Graph Neural Networks

DeepTMA: Predicting Effective Contention Models for Network Calculus using Graph Neural Networks Fabien Geyer 1,2 and Steffen Bondorf 3 INFOCOM 2019 Wednesday 1 st May, 2019 1 Chair of Network Architectures and Services 3 Dept. of Information

711 views • 57 slides
A Self-Organizing Fuzzy Neural Networks H. S. LI N, X. Z. GAO, XI ANLI N HUANG, AND Z. Y. SONG

A Self-Organizing Fuzzy Neural Networks H. S. LI N, X. Z. GAO, XI ANLI N HUANG, AND Z. Y. SONG

A Self-Organizing Fuzzy Neural Networks H. S. LI N, X. Z. GAO, XI ANLI N HUANG, AND Z. Y. SONG Abstract This paper proposes a novel clustering algorithm for the structure learning of fuzzy neural networks. Our clustering algorithm uses

635 views • 37 slides
Outline Evolution of neurocomputing Artificial neural networks Feed forward

Outline Evolution of neurocomputing Artificial neural networks Feed forward

Introduction to Neural Networks and Deep Learning Ling Guan References: S. Haykin, Neural Networks , 2 nd Edition, New Jersey: Prentice Hall, 1. 2004. C. M. Bishop, Neural Networks for Pattern Re cognition, New York: 2. Oxford University

451 views • 17 slides
Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

852 views • 33 slides
Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

1.17k views • 91 slides
CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks Neuron Non-linearity Softmax layer DNN training Loss function and regularization SGD and backprop Learning rate Overfitting

1.8k views • 86 slides
Neural Networks for Machine Learning Lecture 12a The Boltzmann Machine learning algorithm

Neural Networks for Machine Learning Lecture 12a The Boltzmann Machine learning algorithm

Neural Networks for Machine Learning Lecture 12a The Boltzmann Machine learning algorithm Geoffrey Hinton Nitish Srivastava, Kevin Swersky Tijmen Tieleman Abdel-rahman Mohamed The goal of learning We want to maximize the It is also

755 views • 47 slides
Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.57k views • 124 slides
Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.63k views • 119 slides
Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography Hopfield, J. J., "Neural networks and physical systems with emergent collective computational abilities," Proceedings of the National Academy

367 views • 19 slides
Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training

Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training

Neural Networks and their Application to Go A. Bausch Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training neural networks Problems AlphaGo Anne-Marie Bausch The Game of Go Policy Network

280 views • 24 slides
How Better Are Predictive Robust Interval . . . Models: Analysis on the Analysis of the Problem

How Better Are Predictive Robust Interval . . . Models: Analysis on the Analysis of the Problem

Predictions Are Important Traditional Statistics . . . Predictive Approach Measurement . . . How Better Are Predictive Robust Interval . . . Models: Analysis on the Analysis of the Problem How Accurate Are . . . Practically Important

368 views • 22 slides
Modern AI: An Engineering Enterprise Some Success: Space Exploration } Building (partially)

Modern AI: An Engineering Enterprise Some Success: Space Exploration } Building (partially)

What is Artificial Intelligence? } Historical definition (Dartmouth Workshop on AI, 1956): The study of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine

171 views • 3 slides
Week 5: MLR Issues and (Some) Fixes R 2 , multicollinearity, F -test nonconstant variance,

Week 5: MLR Issues and (Some) Fixes R 2 , multicollinearity, F -test nonconstant variance,

BUS41100 Applied Regression Analysis Week 5: MLR Issues and (Some) Fixes R 2 , multicollinearity, F -test nonconstant variance, clustering, panels Max H. Farrell The University of Chicago Booth School of Business A (bad) goodness of fit measure:

573 views • 39 slides
continuous random variables Continuous random variable: takes values in an uncountable set, e.g.

continuous random variables Continuous random variable: takes values in an uncountable set, e.g.

continuous random variables Discrete random variable: takes values in a finite or countable set, e.g. X {1,2, ..., 6} with equal probability X is positive integer i with probability 2 -i continuous random variables Continuous random variable:

763 views • 10 slides
From Binary to Multiclass Classification CS 6355: Structured Prediction 1 We have seen binary

From Binary to Multiclass Classification CS 6355: Structured Prediction 1 We have seen binary

From Binary to Multiclass Classification CS 6355: Structured Prediction 1 We have seen binary classification We have seen linear models Learning algorithms Perceptron SVM Logistic Regression Prediction is simple Given

1.41k views • 78 slides
Gene finding and gene structure prediction Lorenzo Cerutti Swiss Institute of Bioinformatics

Gene finding and gene structure prediction Lorenzo Cerutti Swiss Institute of Bioinformatics

Gene finding and gene structure prediction Lorenzo Cerutti Swiss Institute of Bioinformatics EMBNet course, March 2003 Gene finding EMBNet 2003 Outline Introduction Homology methods Principles Examples of programs using

886 views • 52 slides
Utilizing In-Store Sensors for Revisit Prediction Sundong Kim and Jae-Gil Lee Korea Advanced

Utilizing In-Store Sensors for Revisit Prediction Sundong Kim and Jae-Gil Lee Korea Advanced

Utilizing In-Store Sensors for Revisit Prediction Sundong Kim and Jae-Gil Lee Korea Advanced Institute of Science and Technology https://github.com/kaist-dmlab/revisit While You Are Shopping 2/25 Utilizing In-Store Sensors for Revisit

403 views • 25 slides
Snoqualmie Valley November 8 th 2019 Welcome! Introductions Announcements SVTCs

Snoqualmie Valley November 8 th 2019 Welcome! Introductions Announcements SVTCs

Connecting communities to improve public transportation for Snoqualmie Valley November 8 th 2019 Welcome! Introductions Announcements SVTCs Website Launch www.GetAroundTheValley.org Agenda Review updated project list Draft

515 views • 13 slides

More recommend