Virtualization in Data Centers ! Data centers use virtualization to - PowerPoint PPT Presentation

A n Empirical Study of Memory Sharing in Virtual Machines Sean Barker , Timothy Wood†, Prashant Shenoy, and Ramesh Sitaraman University of Massachusetts Amherst The George Washington University† Department of Computer Science

Virtualization in Data Centers ! Data centers use virtualization to improve resource utilization • Flexible mapping of resources to users • More servers and applications • Smaller hardware footprint ! Maximizing benefits • Efficient resource sharing • Virtual machine placement VM 2 VM 3 VM 1 VM 4 Server A Server B Sean Barker (sbarker@cs.umass.edu) 2

Content Based Page Sharing ! Eliminate identical pages of memory across multiple VMs Hypervisor Physical RAM A ! Virtual VM pages mapped to D FREE A B physical pages D D VM 2 Page Table FREE B A ! Hypervisor detects duplicates A B B C VM 1 C ! Replaced with copy-on-write Page Table references Sean Barker (sbarker@cs.umass.edu) 3

Page Sharing Systems ! Extensive prior work in exploiting page sharing ! VMware ESX Server [SIGOPS 02] • Periodic memory scanning to detect duplicates • >30% memory savings ! Difference Engine [OSDI 08] • Sub-page sharing and patching • >60% memory savings ! Satori [USENIX 09] • Sharing of short-lived pages • >90% of possible sharing captured Sean Barker (sbarker@cs.umass.edu) 4

Open Questions on Sharing ! What levels of sharing are possible in typical real-world machines ? ! What are the factors that impact sharing potential? • OS family? Versions? Applications? ! How will emerging technologies impact sharing? • New OS technologies? • VDI farms? LAMP clusters? ! Our goal: Provide practical insights into these questions through a careful study of memory data Sean Barker (sbarker@cs.umass.edu) 5

Outline ! Background and motivation ! Data collection and types of sharing ! Study of real-world sharing potential ! Study of the factors impacting sharing ! Conclusions Sean Barker (sbarker@cs.umass.edu) 6

Data Collection ! Real-world memory traces • ~50 real machines (server/desktop mix) • Uncontrolled user workloads • Memory snapshots every 30 minutes ! Supplementary traces from controlled VMs • Mac/Win/Linux, mixed versions, 32/64 bit • 3 application setups per VM: • No workload (freshly booted) • Server apps (LAMP stack) • Desktop apps (office, browser, media player) Sean Barker (sbarker@cs.umass.edu) 7

Types of Sharing ! Self-sharing : sharing within individual VMs • E.g., multiple zero pages Machine 1 Machine 2 Machine 3 A A A B B B C C C Self-Sharing: Self-Sharing: Self-Sharing: 2 pages 2 pages 2 pages Total Self-Sharing: 6 pages ! Inter-VM sharing : sharing across multiple VMs • E.g., shared OS state Shared Machine VM 1 VM 2 VM 3 A B C A B C A B C Total Inter-VM Sharing: 6 pages Sean Barker (sbarker@cs.umass.edu) 8

Outline ! Background and motivation ! Data collection and types of sharing ! Study of real-world sharing potential ! Study of the factors impacting sharing ! Conclusions Sean Barker (sbarker@cs.umass.edu) 9

Self-Sharing in Real-World Traces ! Average sharing of 14% 50 average maximum • Excluding zero pages minimum Self-sharing (% of memory) 40 30 ! Peak sharing up to 50% 20 10 0 A B C D E F G Machine ! Stable ‘baseline’ sharing of 8% ! Significant ( ~15% ) self-sharing potential observed Sean Barker (sbarker@cs.umass.edu) 10

Inter-VM Sharing in Real-World Traces ! ‘High’ average sharing 50 average maximum of just 2% minimum Inter-VM Sharing (% of memory) 40 30 ! <0.1% sharing in 20 15 of 21 pairings 10 0 B/C B/D B/A C/D C/A D/A All Others Machine Pair ! In our traces, inter-VM sharing never above 6% ! Observed minimal ( <2% ) inter-VM sharing potential Sean Barker (sbarker@cs.umass.edu) 11

Real-World Trace Observations ! Typical 15% possible sharing observed • Significant, but less than expected from synthetic workloads ! Most ( 85+% ) sharing derived from self-sharing • What about collocating many VMs? • All 7 machines...still 80+% from self-sharing ! Self-sharing doesn’t require virtualization! • Could capture it within a VM or nonvirtualized host ! Self-sharing is significant, but what causes it? Sean Barker (sbarker@cs.umass.edu) 12

Self-Sharing Case Study ! What causes self-sharing in a Linux desktop? • Looking at nonzero sharing ! Expanded version of Linux memory tracer • Track page contents and processes [libc-2.12.so 000b6000 r-xp]: sshd apache2 ! Group sharing involvement (% of self-sharing) by content and process Sean Barker (sbarker@cs.umass.edu) 13

Self-Sharing by Process ! >30% sharing processes GUI processes CLI processes 50 GUI apps/libraries Sharing Involvement (%) 40 30 ! <20% sharing from 20 other system processes 10 0 xorg firefox oo.org gnome ssh bash Process or Group ! Memory footprint likely dominated by GUI ! Process self-sharing resulting from user workload Sean Barker (sbarker@cs.umass.edu) 14

Self-Sharing by Content ! 94% sharing from individual libraries 50 category totals libraries and heaps Sharing Involvement (%) 40 30 ! Possibly from recreated 20 data structures 10 0 libc libcairo libgtk libxul libraries heaps stacks Page Content ! 2.3 MB sharing from single Xorg heap page (~600 copies) ! Duplicate data allocations evident in processes Sean Barker (sbarker@cs.umass.edu) 15

Outline ! Background and motivation ! Data collection and types of sharing ! Study of real-world sharing potential ! Study of the factors impacting sharing ! Conclusions Sean Barker (sbarker@cs.umass.edu) 16

Factors Impacting Sharing ! How do various properties influence sharing? ! Operating system characteristics • Family (e.g., Linux or Windows) • Version (e.g., Windows XP/7, Ubuntu 10.04/10.10) • Architecture (x86 or x64) ! Application setup (LAMP and VDI setups) ! Sharing granularity (number of pages per chunk) ! New OS technologies (e.g., ASLR) Sean Barker (sbarker@cs.umass.edu) 17

Self-Sharing Across VMs 160 ! ~100 MB differences 140 120 between OS families, Self-sharing (MB) 100 major versions (XP/7) 80 60 40 20 0 mac winxp win7-32 win7-64 ub04-32 ub04-64 ub10-32 ub10-64 co-32 co-64 ! <20 MB differences between minor versions, Virtual Machine architectures ! Large self-sharing variations between ‘base’ OSes Sean Barker (sbarker@cs.umass.edu) 18

Sharing Across VMs 200 no-app server desktop 150 Inter-VM Sharing (MB) 100 50 0 mac/ub10-64 win7-64/ub10-64 mac/win7-64 Virtual Machine Pair Sean Barker (sbarker@cs.umass.edu) 19

Sharing Across VMs 200 no-app server desktop <5 MB across Inter-VM Sharing (MB) 150 OS families 100 50 0 mac/ub10-64 win7-64/ub10-64 mac/win7-64 Virtual Machine Pair Sean Barker (sbarker@cs.umass.edu) 19

Sharing Across VMs 200 no-app server desktop <5 MB across 200 no-app Inter-VM Sharing (MB) 150 OS families server desktop 100 Inter-VM Sharing (MB) 150 50 0 mac/ub10-64 win7-64/ub10-64 mac/win7-64 100 Virtual Machine Pair 50 0 w w i i n n 7 7 - - 3 6 2 4 / / w w i i n n x x p p Virtual Machine Pair Sean Barker (sbarker@cs.umass.edu) 19

Sharing Across VMs 200 200 no-app no-app server server <5 MB base, 50+ MB desktop desktop <5 MB across Inter-VM Sharing (MB) 150 Inter-VM Sharing (MB) 150 app sharing across OS families major versions 100 100 50 50 0 mac/ub10-64 win7-64/ub10-64 mac/win7-64 0 win7-32/winxp win7-64/winxp Virtual Machine Pair Virtual Machine Pair Sean Barker (sbarker@cs.umass.edu) 19

Sharing Across VMs 200 200 no-app no-app server server <5 MB base, 50+ MB desktop desktop <5 MB across 200 Inter-VM Sharing (MB) 150 Inter-VM Sharing (MB) 150 app sharing across no-app OS families server major versions desktop 100 100 150 Inter-VM Sharing (MB) 50 50 0 mac/ub10-64 win7-64/ub10-64 mac/win7-64 0 win7-32/winxp win7-64/winxp 100 Virtual Machine Pair Virtual Machine Pair 50 0 u u b b 0 0 4 4 - - 3 6 2 4 / / u u b b 1 1 0 0 - - 3 6 2 4 Virtual Machine Pair Sean Barker (sbarker@cs.umass.edu) 19

Sharing Across VMs 200 200 no-app no-app server server <5 MB base, 50+ MB desktop desktop <5 MB across Inter-VM Sharing (MB) 150 Inter-VM Sharing (MB) 150 app sharing across OS families major versions 100 100 50 50 0 mac/ub10-64 win7-64/ub10-64 mac/win7-64 0 win7-32/winxp win7-64/winxp Virtual Machine Pair Virtual Machine Pair 200 no-app server desktop 50+ MB across Inter-VM Sharing (MB) 150 minor versions 100 50 0 ub04-32/ub10-32 ub04-64/ub10-64 Virtual Machine Pair Sean Barker (sbarker@cs.umass.edu) 19