SLIDE 1
Bayesian Optimization of Gaussian Processes applied to Performance Tuning
Ramki Ramakrishna @ysr1729 #TwitterVMTeam
QCon Sao Paulo, 2019
A JVM Engineer talks to a Data Scientist
2
Bayesian Optimization of Gaussian Processes applied to Performance - - PowerPoint PPT Presentation
Bayesian Optimization of Gaussian Processes applied to Performance - - PowerPoint PPT Presentation
Bayesian Optimization of Gaussian Processes applied to Performance Tuning Ramki Ramakrishna @ysr1729 #TwitterVMTeam QCon Sao Paulo, 2019 A JVM Engineer talks to a Data Scientist 2 Many Hundreds of Services Several Tens of Thousands
SLIDE 2
SLIDE 3
Many Hundreds
- f
Services
SLIDE 4
Several Tens of Thousands of Physical Servers
SLIDE 5
Several Millions
- f
CPU Cores
SLIDE 6
Several Hundreds of Thousands of Twitter JVMs
SLIDE 7
A Few Hundred Tunable JVM Parameters
SLIDE 8
SLIDE 9
SLIDE 10
SLIDE 11
SLIDE 12
Mining for Gold
- 1930’s South Africa
- Prospecting for gold and other minerals
- Daniel Krige, 1951: “Kriging” in geostatistics
- Jonas Mockus 70’s
- Jones et al. 80’s
- Rasmussen & Williams 90’s : Gaussian Processes
12
SLIDE 13
Applications
- design of expensive experiments
- optimal designs
- optimization of engineered materials
- hyperparameter tuning (architectural
parameters) of neural networks
13
SLIDE 14
Engineering as Optimization
- linear or non-linear objective function
- finite convex or non-convex space, rectangular, linear
(affine) or non-linear constraints
- black box objective function
- black box constraints
- noisy objective function
- noisy constraints
14
SLIDE 15
Black Box Modeling
- Model the unknown objective function
- Model the unknown constraints
- Model is a “surrogate”
- Evaluations are expensive
15
SLIDE 16
Models and Model Parameters
- Parametric models
- Non-parametric models
16
SLIDE 17
Probabilistic Models
- A measure of our uncertainty
- A measure of measurement/observation noise
17
SLIDE 18
Gaussian Process
- : mean function
- : covariance function
18
GP(μ, κ)
μ(x) κ(x, x′)
SLIDE 19
Gaussian Process
- Two different views
- a vector of possibly uncountably many
Gaussian variables with given mean and a joint covariate distribution
- a Gaussian distribution over functions
19
SLIDE 20
Gaussian Process
20
SLIDE 21
Gaussian Process
21
SLIDE 22
Gaussian Process
22
SLIDE 23
Gaussian Process
23
SLIDE 24
Gaussian Process
24
SLIDE 25
Gaussian Process
25
SLIDE 26
Gaussian Process
26
SLIDE 27
Gaussian Process
27
SLIDE 28
28
μn(x) = κT(K + σ2
noiseI)−1Y
κn(x, x′) = κ(x, x′) − κT(K + σ2
noiseI)−1κ′
GPn
SLIDE 29
Covariance Kernel Function
- Squared exponentials (SE)
- “n/2" Matern kernels
29
SLIDE 30
Covariance Kernel Functions
30
SLIDE 31
Covariance Kernel Functions
31
SLIDE 32
Acquisition Function
32
GPprior + Datan →Bayes GPn →? xn+1
SLIDE 33
Acquisition Function
33
GPprior + Datan+1 →Bayes GPn+1 →? xn+2
SLIDE 34
Acquisition Functions
- Thompson Sampling from the posterior GP
(TS)
- Probability of Improvement (PI)
- Upper Confidence Bound (UCB)
- Expected Improvement (EI)
34
SLIDE 35
Thompson Sampling
35
SLIDE 36
Probability of Improvement
36
SLIDE 37
Upper Confidence Bound
37
SLIDE 38
Expected Improvement
38
SLIDE 39
Acquisition Function
- Thompson Sampling from the posterior GP
(TS)
- Probability of Improvement (PI)
- Upper Confidence Bound (UCB)
- Expected Improvement (EI)
39
SLIDE 40
Maximizing the Acquisition Function
- piecewise infinitely smooth
- gradient-based techniques work
- modified Monte-Carlo techniques are typically
used
40
SLIDE 41
Optimizing Performance Parameters
myExpt = Optimizer.declareDevice(Parm1: {Int, Min1, Max1}, Parm2: {Real, Min2, Max2}, Parm3: {Enum, enum1, enum2, enum3} …) myExpt.setSLA(…) // set performance SLA myExpt.setTerminationCriteria(…) // set termination criteria while (!myExpt.shouldTerminate()) { parmSuggestion = myExpt.suggest() // get another test suggestion newRun = myDevice.test(parmSuggestion) // test device at given setting if (myExpt.isValid(newRun)) { // is SLA met? myExpt.update(parmSuggestion, newRun) // update w/new result } } return myExpt.bestConfig()
SLIDE 42
Bayesian Optimization
myExpt = Optimizer.declareDevice(Parm1: {Int, Min1, Max1}, Parm2: {Real, Min2, Max2}, Parm3: {Enum, enum1, enum2, enum3} …) myExpt.setSLA(…) // set performance SLA myExpt.setTerminationCriteria(…) // set termination criteria while (!myExpt.shouldTerminate()) { parmSuggestion = myExpt.suggest() // get another test suggestion newRun = myDevice.test(parmSuggestion) // test device at given setting if (myExpt.isValid(newRun)) { // is SLA met? myExpt.update(parmSuggestion, newRun) // update w/new result } } return myExpt.bestConfig()
SLIDE 43
Constraints
myExpt = Optimizer.declareDevice(Parm1: {Int, Min1, Max1}, Parm2: {Real, Min2, Max2}, Parm3: {Enum, enum1, enum2, enum3} …) myExpt.setSLA(…) // set performance SLA myExpt.setTerminationCriteria(…) // set termination criteria while (!myExpt.shouldTerminate()) { parmSuggestion = myExpt.suggest() // get another test suggestion newRun = myDevice.test(parmSuggestion) // test device at given setting if (myExpt.isValid(newRun)) { // is SLA met? myExpt.update(parmSuggestion, newRun) // update w/new result } } return myExpt.bestConfig()
SLIDE 44
AUTOTUNE AS A SERVICE
SLIDE 45
SLIDE 46
SLIDE 47
SLIDE 48
SLIDE 49
SLIDE 50
SLIDE 51
SLIDE 52
SLIDE 53
SLIDE 54
SLIDE 55
SLIDE 56
SLIDE 57
SLIDE 58
GizmoDuck & Garbage Collection Overhead via Tuning JVM Parameters
SLIDE 59
TweetyPie & CPU Utilization via Tuning Graal JIT Parameters
SLIDE 60