Deconstructing Amazon EC2 Spot Instance Pricing Orna Agmon - - PowerPoint PPT Presentation
Deconstructing Amazon EC2 Spot Instance Pricing Orna Agmon Ben-Yehuda Muli Ben-Yehuda Assaf Schuster Dan Tsafrir Department of Computer Science Technion Israel Institute of Technology CloudCom 2011 (and Epilogue) Agmon Ben-Yehuda,
Deconstructing Amazon EC2 Spot Instance Pricing
Orna Agmon Ben-Yehuda Muli Ben-Yehuda Assaf Schuster Dan Tsafrir
Department of Computer Science Technion — Israel Institute of Technology
CloudCom 2011 (and Epilogue)
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 1/35
Amazon EC2 cloud terminology
Amazon rents virtual machines with prices which vary according to: Instance types Regions Operating systems Commitment level: reserved, on-demand, spot Payment by the hour, except for the last hour fraction of a terminated spot instance.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 2/35
What are spot instances?
Clients bid (attach a maximal price to the instance request). The provider publishes a uniform spot price every so often, which the user pays. As long as the bid exceeds the spot price, the instance can stay. An instance is killed if the price goes above the bid.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 3/35
Why sell spot instances? Idle machines Spot Instances kept for elasticity easily evacuated can be sold cheap must be sold cheap
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 4/35
Amazon EC2 Spot instances declaration
“The Spot Price changes periodically based on supply and
How does Amazon price its spot instances? Are spot prices really based on natural supply and demand? Or Are they artificially set, raised above the market value?
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 5/35
Who cares?
Researchers learn about the market from EC2 price histories; they assume (following Amazon’s statement) that spot prices reflect real bids [Zhang et al. 2011], or represent market clearing prices [Chen et al. 2011]. Clients bid and evaluate bidding strategies using price histories. Other providers seek information about the market and pricing algorithms.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 6/35
Who cares?
Researchers learn about the market from EC2 price histories; they assume (following Amazon’s statement) that spot prices reflect real bids [Zhang et al. 2011], or represent market clearing prices [Chen et al. 2011].
If prices are artificial, their results are questionable.
Clients bid and evaluate bidding strategies using price histories. Other providers seek information about the market and pricing algorithms.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 6/35
Who cares?
Researchers learn about the market from EC2 price histories; they assume (following Amazon’s statement) that spot prices reflect real bids [Zhang et al. 2011], or represent market clearing prices [Chen et al. 2011]. Clients bid and evaluate bidding strategies using price histories.
If prices are artificial, an algorithm change may make the past irrelevant to future predictions.
Other providers seek information about the market and pricing algorithms.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 6/35
Who cares?
Researchers learn about the market from EC2 price histories; they assume (following Amazon’s statement) that spot prices reflect real bids [Zhang et al. 2011], or represent market clearing prices [Chen et al. 2011]. Clients bid and evaluate bidding strategies using price histories. Other providers seek information about the market and pricing algorithms.
If Prices are artificial, they do not supply such information.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 6/35
Examples of spot instance market-driven mechanisms
Clients bid secretly. The provider sorts the bids (descending order). Uniform price for all granted instances. The provider grants only the first N bids. N is limited:
supply revenue maximization minimal price (hidden) reserve price.
Pricing according to minimal price or bid N + 1.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 7/35
Price histories
Amazon encourages clients to look at price histories and bid
date (Dec 2009 − Jul 2010) charged prices as fraction of on demand price
Figure: windows.m1.small.us-east
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 8/35
Alternative view—availability of bid price
The time in which the spot price was below the bid price, divided by the total time.
0.8 1 1.2 1.4 1.6 1.8 2 0.2 0.4 0.6 0.8 1availability declared price [$/hour]
Plateau Knee Straight segmentTypical shape: a straight segment and a high knee.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 9/35
Windows instance availability as a function of price
0.5 1 1.5 2 2.5 0.2 0.4 0.6 0.8 1availability declared price [$/hour]
us−east m1 instances us−east m2.xlarge instance us−east m2 2xlarge and 4xlarge us−east c1 instancesThe typical shape at different prices. Looks similar for Linux.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 10/35
Linux instance availability as a function of normalized price
0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.2 0.4 0.6 0.8 1availability declared price as fraction of on−demand price
us−east m1 instances us−east m2.xlarge instance us−east m2 2xlarge and 4xlarge . us−east c1 instancesTwo groups of regions (one and the rest). The forest disappears.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 11/35
Windows instance availability as a function of normalized price
0.4 0.5 0.6 0.7 0.8 0.9 0.2 0.4 0.6 0.8 1availability declared price as fraction of on−demand price
us−east m1 instances us−east m2.xlarge instance us−east m2 2xlarge and 4xlarge . us−east c1 instancesA repeating pattern within the two region groups. Windows clients differ from Linux clients.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 12/35
Our hypothesis
Natural supply and demand conditions cause all this? Gee, that’s funny. Alternatively... Amazon often changes the auction’s reserve price, independently of client bids. The reserve price’s value and its changing frequency are not market driven. Usually, the spot price is identical to the reserve price. Hence, the spot prices are usually not market-driven.
In contradiction to Amazon’s statement.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 13/35
Why dynamic secret reserve price?
A dynamic reserve price maintains an impression of constant change. Forces clients to
Bid higher or Tolerate sudden unavailability.
A secret dynamic reserve price also masks times of low demand and price inactivity, by giving an illusion of false activity.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 14/35
Planning the dynamic reserve price algorithm
1.05 1.1 1.15 1.2 1.25 1.3 0.2 0.4 0.6 0.8 1availability declared price [$/hour]
Floor Price (F) Ceiling Price (C) Pricing bandAgmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 15/35
Fitting an auto-regressive process AR(1) for ap-southeast.windows types
∆i = −a1∆i−1 + ǫ(σ) ∆i is the difference of two consequent prices. a1 = 0.7. ǫ(σ)—white noise with a standard deviation σ = 0.39(C − F). m1.small matched a1 = 0.5, σ = 0.5(C − F).
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 16/35
Variance of the fitted AR(1) process
0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 −0.02 0.02 0.04 0.06 0.08
band width [$] fitted white noise σ
y = 0.39*x − 0.00026 ap−southeast−1 linear ap−southeast−1.windows.m1.small
The close fit supports our hypothesis.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 17/35
Constructing the reserve price algorithm
Initial price is F. Initial change is −0.1(C − F).
Not all initial conditions are good.
Compute next price change using the fitted AR(1) process. Advance the next price Pi = Pi−1 + ∆i. Truncate the process to the range [F, C] by regenerating the white noise component while Pi is outside the [F, C] range or identical to Pi−1. Round all prices to 0.1 cent.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 18/35
Is the constructed algorithm consistent with reality?
Periodogram (power spectral density): a power-normalized discrete Fourier transform.
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −100 −90 −80 −70 −60 −50 −40Normalized frequency (× π rad/sample) One−sided PSD (dB/rad/sample)
PSD estimate of EC2 ap−southeast trace PSD estimate of AR(1) processThe close fit supports our hypothesis.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 19/35
Is the AR(1) process natural or artificial?
A natural process would have a significant weekly cycle. The normalized weekly averages of ap-southeast.windows types do not show a weekly cycle: The day-of-week impact is smaller than the noise (impact of types).
Sun Mon Tue Wed Thu Fri Sat 0.996 0.997 0.998 0.999 1 1.001 1.002 1.003 1.004 1.005normalized mean daily price
m1.small m1.large m1.xl m2.xl m2.2xl m2.4xl c1.medium c1.xlargeThe AR(1) process is inconsistent with a natural process.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 20/35
Is the AR(1) process partly natural or artificial?
Partly natural: partly real bids within band above the reserve price, partly reserve prices. Expected to have a mean price above mid-range. The mean price is lower than the mid-range (by up to 2%). Many clients already noted that bidding inside the band is not cost effective. The AR(1) process’s average is consistent with an average
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 21/35
Are traces as a whole natural or artificial?
98% of the time, prices are within the band. Traces as a whole are consistent with being artificial 98%
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 22/35
If our hypothesis is correct, then:
98% of the time spot prices carry little information about real client bids! Researchers cannot learn from spot prices about client valuations for products, nor about supply and demand. The spot price is not necessarily a market clearing price.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 23/35
Pricing epochs
Dec Jan Feb Mar Apr May Jun Jul 0.4 0.5 0.6 0.7 0.8 0.9 1date (Dec 2009 − Jul 2010) normalized spot price
2nd epoch 1st epoch 3rd epoch low prices tran− si− tion low and high prices high prices new min. priceAgmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 24/35
Pricing epochs
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 24/35
Pricing epochs
new min. price
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 24/35
Pricing epochs
Feb Mar
date (Dec 2009 − Jul 2010)
low prices
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 24/35
Price changing timing (us-east)
1 2 3 4 5 6 7 8 9 0.2 0.4 0.6 0.8 1
step length: time between price changes [h] probability
Jan 2010 − Jul 2010 Jul 2010 − Feb 2011 Feb 2011 − April 2011 (present day)
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 25/35
Workload modeling
Workload traces of large systems. Truncated to tasks longer than 10 minutes, shorter than 24 hours. Grid: LPC-EGEE, a cluster of a large grid.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 26/35
Customer bid modeling
No data! We test three models, to show that the qualitative results are insensitive to the model. Bids are concentrated between a minimal price (0.4) and the
Pareto distribution (minimal value of 0.4, Pareto index of 2). N(0.7, 0.32), truncated at 0.4. A linear mapping from runtimes to (0.4, 1], which reflects client aversion to having long-running instances terminated.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 27/35
LPC-EGEE
0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1declared price as fraction of on−demand price availability fraction
Figure: Linear segment and knee iff simulating with AR1 dynamic reserve price, insensitive to client bidding. Consistent with traces.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 28/35
Epoch 2
0.41 0.42 0.43 0.44 0.45 0.46 0.47 0.48 0.49 0.5 0.51 0.65 0.7 0.75 0.8 0.85 0.9 0.95declared price as fraction of on−demand price availability fraction
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 29/35
Price trace comparison
500 1000 1500 2000 2500 3000 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 time[h] normalized spot price(a) LPC-EGEE, constant reserve price (b) Second Epoch
The second epoch is consistent with a constant reserve price.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 30/35
Conclusions
It is likely that Amazon sets spot prices using an AR(1) (hidden) reserve price. 98% of the time:
The spot price is probably just the reserve price. EC2 traces do not necessarily represent clearing prices or real bids.
Many features (minimal price, band width, change timing) are artificial, have changed and may suddenly change again.
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 31/35
Post mortem
1 2 3 4 5 6 7 8 9 0.5 0.6 0.7 0.8 0.9 1 1.1 Time (Months of 2011) Spot instance price (normalized) us−east−1.suse.m1.large.csv Paper accepted First paper submitted Paper rejected, Tech report published and re−Twitted End of data used for the paperAgmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 32/35
CloudExchange
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 33/35
Current View on Amazon
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 34/35
Questions?
Contact us at: {ladypine, muli, assaf, dan } at cs.technion.ac.il Thank You!
Agmon Ben-Yehuda, Ben-Yehuda, Schuster, Tsafrir Deconstructing Spot Prices 35/35