SLIDE 1
Operational Experiences with Disk Imaging in a Multi-Tenant Datacenter
Kevin Atkinson, Gary Wong, and Robert Ricci
2
Operational Experiences with Disk Imaging in a Multi-Tenant - - PowerPoint PPT Presentation
Operational Experiences with Disk Imaging in a Multi-Tenant - - PowerPoint PPT Presentation
Operational Experiences with Disk Imaging in a Multi-Tenant Datacenter Kevin Atkinson, Gary Wong, and Robert Ricci 2 2 2 2 2 2 2 Properties of disk images and their usage have consequences for: Storage
SLIDE 2
SLIDE 3
2
SLIDE 4
2
SLIDE 5
2
SLIDE 6
2
SLIDE 7
2
SLIDE 8
2
SLIDE 9
3
Properties of disk images and their usage have consequences for:
- ❖ Storage
❖ Caching ❖ Pre-loading ❖ Distribution
SLIDE 10
4
SLIDE 11
4
What does the working set look like?
SLIDE 12
4
What does the working set look like? What do the images themselves look like?
SLIDE 13
4
What does the working set look like? What do the images themselves look like? What are the key factors in pre-loading?
SLIDE 14
The dataset
❖ Four years (2009-2013): 279,972 requests ❖ Users: 1,301 individuals, 368 organizations ❖ Unique images: 714 ❖ Emulab ❖ ~600 PCs ❖ Facility / user image model
5
SLIDE 15
User Behavior
SLIDE 16
“Emulab is a pretty odd beast and its users are even weirder.”
7
SLIDE 17
–Reviewer D
“Emulab is a pretty odd beast and its users are even weirder.”
7
SLIDE 18
–Reviewer D [Emulab user]
“Emulab is a pretty odd beast and its users are even weirder.”
7
SLIDE 19
Facility vs. user images
8
Facility User 55.6% 44.4%
SLIDE 20
Facility vs. user images
8
Facility User 55.6% 44.4%
SLIDE 21
Facility vs. user images
8
Facility User 55.6% 44.4%
- 1) Most users stick to facility or user images
2) Heaviest users use their own images
SLIDE 22
Image popularity
9
SLIDE 23
Image popularity
9
SLIDE 24
Image popularity
9
SLIDE 25
Image popularity
9
SLIDE 26
Image popularity
9
Exponential
SLIDE 27
Image popularity
9
Exponential Heavy-Tailed
SLIDE 28
Image popularity
9
1) Facility images have a smaller, lighter tail 2) Most popular image < 13% of requests
Exponential Heavy-Tailed
SLIDE 29
Scaling: total images
10
SLIDE 30
Scaling: total images
10
SLIDE 31
Scaling: total images
10
SLIDE 32
Scaling: total images
10
As userbase grows, user images dominate the totals
SLIDE 33
Daily working set
11
SLIDE 34
Daily working set
11
Small image set each day –※ good caching potential
SLIDE 35
Scaling: working set
12
SLIDE 36
Scaling: working set
12
SLIDE 37
Scaling: working set
12
SLIDE 38
Scaling: working set
12
Facility will max out
SLIDE 39
Scaling: working set
12
Facility will max out –※ In the limit, highly popular facility images account for most requests
SLIDE 40
Image Contents
SLIDE 41
Block-level similarity
14
Base
SLIDE 42
Block-level similarity
14
Base Derived
SLIDE 43
Block-level similarity
14
Base Derived
SLIDE 44
Block-level similarity
14
Percentage of blocks that need to be written to transform the base image into derived
Base Derived
SLIDE 45
Block-level similarity
15
Derived: User image Base: Most similar facility image
SLIDE 46
Block-level similarity
15
Derived: User image Base: Most similar facility image
SLIDE 47
Block-level similarity
15
Derived: User image Base: Most similar facility image 1) De-duplicating storage an attractive option 2) Differential loading has potential
SLIDE 48
Pre-Loading
SLIDE 49
Pre-loading: Size
17
SLIDE 50
Pre-loading: Size
17
Spare Capacity
SLIDE 51
Pre-loading: Size
17
Spare Capacity Mostly Full
SLIDE 52
Pre-loading: Size
17
Spare Capacity Mostly Full
WSS for facility images maxes out
- n large facilities
SLIDE 53
Pre-loading: Size
17
Spare Capacity Mostly Full
1) Key: Ratio of WSS to idle capacity 2) Effective when ratio is high WSS for facility images maxes out
- n large facilities
SLIDE 54
Pre-loading: Rate
18
SLIDE 55
Pre-loading: Rate
18
SLIDE 56
Pre-loading: Rate
18
Invest in fast, scalable imaging
SLIDE 57
Conclusions
SLIDE 58
General conclusions
❖ Deduplicating, two-tier storage attractive ❖ Caching can be effective ❖ Image lifespan, idle periods ❖ Treat facility and user images differently ❖ Facility better targets for pre-loading ❖ Differential loading requires new strategies ❖ Potential savings, outline of optimization problem ❖ Images per organization, WSS per week
20
SLIDE 59
21
Explore the data, reproduce our results:
- http://aptlab.net/p/tbres/nsdi14
SLIDE 60
- No dominant images
22
SLIDE 61
- No dominant images
22
No image dominates long-term, popular images change frequently
SLIDE 62
Image lifespan
23
SLIDE 63
Image lifespan
23
A few days
SLIDE 64
Image lifespan
23
A few days Four Years
SLIDE 65
Image lifespan
23
A few days Four Years
Two-tiered storage system attractive
SLIDE 66
Savings from deltas
24
SLIDE 67
Images per organization
25
SLIDE 68
Idle images
26
SLIDE 69
WSS per week
27
SLIDE 70
Top images
28 RHL90-STD [D] 21,993 7.9% FEDORA10-STD 18,042 6.4% UBUNTU10-STD 14,402 5.1% RHL90-STD 13,182 4.7% FC4-UPDATE 12,097 4.3%
u
715/10 11,156 4.0% FBSD410-STD 8,916 3.2% FEDORA8-STD 8,153 2.9%
u
237/69 7,512 2.7%
u
296/35 7,179 2.6%
u
787/24 6,243 2.2% UBUNTU70-STD 6,021 2.2% UBUNTU12-64-STD 5,834 2.1%
SLIDE 71
Size considerations
❖ Small facilities with few idle disks ❖ Pre-loading not valuable ❖ Large facilities - focus on: ❖ Scalable reloading mechanisms ❖ Prediction and optimization for user requests
29