SLIDE 1
Agenda Motives History and status Design Problems Benchmark Future Work
Agenda
◮ Motives ◮ History and status ◮ Design ◮ Problems ◮ Preliminary benchmark results ◮ Future work
Chicago – August 18, 2014 vNUMA in Xen 2 / 19
vNUMA in Xen Wei Liu & Elena Ufimtseva Chicago August 18, 2014 - - PowerPoint PPT Presentation
vNUMA in Xen Wei Liu & Elena Ufimtseva Chicago August 18, 2014 - - PowerPoint PPT Presentation
vNUMA in Xen Wei Liu & Elena Ufimtseva Chicago August 18, 2014 Agenda Motives History and status Design Problems Benchmark Future Work Agenda Motives History and status Design Problems Preliminary benchmark
SLIDE 2
SLIDE 3
Agenda Motives History and status Design Problems Benchmark Future Work
Motives
◮ Cross NUMA node memory access is expensive
◮ Need to avoid cross node memory access
◮ Xen is NUMA aware
◮ NUMA aware scheduling ◮ NUMA aware guest memory placement
◮ Operating system like Linux is NUMA aware
◮ NUMA aware scheduling ◮ NUMA aware memory allocation / migration
◮ The missing bits
◮ Memory layout information ◮ CPU topology Chicago – August 18, 2014 vNUMA in Xen 3 / 19
SLIDE 4
Agenda Motives History and status Design Problems Benchmark Future Work
History and status
◮ PV vNUMA presented in Xen Summit 2010 by Dulloor Rao
http://slidesha.re/1AXsFbu
◮ HVM vNUMA patches posted by Andre Przywara circa 2010 ◮ Elena Ufimtseva has been working on upstreamable PV
vNUMA since 2013
Chicago – August 18, 2014 vNUMA in Xen 4 / 19
SLIDE 5
Agenda Motives History and status Design Problems Benchmark Future Work
Design: PV and PVH
◮ Toolstack puts enlightenment information in hypervisor ◮ Guest memory allocation in accordance with enlightenment
information
◮ Guest retrieves enlightenment information via hypercall during
boot up
Chicago – August 18, 2014 vNUMA in Xen 5 / 19
SLIDE 6
Agenda Motives History and status Design Problems Benchmark Future Work
Design: PV and PVH
Enlightenment information structure struct vnuma_info { nr_vnodes; vdistance[nr_vnodes * nr_vnodes]; vcpu_to_vnode[nr_vnodes]; vnode_to_pnode[nr_vnodes]; vmemrange[nr_vnodes]; }
Chicago – August 18, 2014 vNUMA in Xen 6 / 19
SLIDE 7
Agenda Motives History and status Design Problems Benchmark Future Work
Design: HVM
◮ Toolstack puts enlightenment information in hypervisor ◮ Toolstack arranges ACPI tables ◮ Guest memory allocation in accordance with enlightenment
information
◮ Guest retrieves layout information via ACPI tables during boot
up
Chicago – August 18, 2014 vNUMA in Xen 7 / 19
SLIDE 8
Agenda Motives History and status Design Problems Benchmark Future Work
Problems: vNUMA and other features
PV PVH HVM Ballooning Y Y* N PoD N/A ? N
Chicago – August 18, 2014 vNUMA in Xen 8 / 19
SLIDE 9
Agenda Motives History and status Design Problems Benchmark Future Work
Problems: CPU topology
Chicago – August 18, 2014 vNUMA in Xen 9 / 19
SLIDE 10
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark
◮ Host
◮ 2 sockets, 12 PCPUs, HT disabled ◮ 36GB RAM, 2 NUMA nodes ◮ NUMA balancing enabled
◮ Guest
◮ 12 VCPUs ◮ 16GB RAM, 2 virtual NUMA nodes ◮ vnodes mapped to different pnodes, vcpu pinned to pnode ◮ NUMA balancing enabled
◮ Benchmarks to run
◮ Autonuma ◮ SPECJBB ◮ STREAM Chicago – August 18, 2014 vNUMA in Xen 10 / 19
SLIDE 11
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark: Autonuma
Chicago – August 18, 2014 vNUMA in Xen 11 / 19
SLIDE 12
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark: Autonuma
Chicago – August 18, 2014 vNUMA in Xen 12 / 19
SLIDE 13
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark: Autonuma
Chicago – August 18, 2014 vNUMA in Xen 13 / 19
SLIDE 14
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark: SPECJBB
Chicago – August 18, 2014 vNUMA in Xen 14 / 19
SLIDE 15
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark: SPECJBB
Chicago – August 18, 2014 vNUMA in Xen 15 / 19
SLIDE 16
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark: STREAM
Chicago – August 18, 2014 vNUMA in Xen 16 / 19
SLIDE 17
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark: Conclusion
◮ vNUMA improves performance for PV guest ◮ vNUMA has weird result in SPECJBB for PVH guest, but
- ther two benchmarks have good results
Chicago – August 18, 2014 vNUMA in Xen 17 / 19
SLIDE 18
Agenda Motives History and status Design Problems Benchmark Future Work
Future Work
◮ basic vNUMA support for all guest types ◮ Dom0 vNUMA ◮ address vNUMA compatibility issues with PoD and ballooning
for HVM guest
◮ address performance issue for PVH
Chicago – August 18, 2014 vNUMA in Xen 18 / 19
SLIDE 19
Thank you!
Chicago – August 18, 2014 vNUMA in Xen 19 / 19