Real-Time Cloud Computing Chenyang Lu Cyber-Physical Systems - PowerPoint PPT Presentation

Real-Time Cloud Computing Chenyang Lu Cyber-Physical Systems Laboratory Department of Computer Science and Engineering h<p://www.cse.wustl.edu/~lu/

Internet of Things Ø Convergence of q Miniaturized hardware: integrate processor, sensors and radios. q Low-power wireless: connect millions of devices to the Internet. q Data analytics: make sense of sensor data. q Rea-time cloud: scalable real-time computing. Ø Enable real-time control of physical environments R. Dor, G. Hackmann, Z. Yang, C. Lu, Y. Chen, M. Kollef and T.C. Bailey, Experiences with an End-To- End Wireless Clinical Monitoring System, Conference on Wireless Health (WH'12), October 2012.

Real-Time Cloud Ø Internet of Things à large-scale sensing and control q Real-time data analytics (aka Big Data) q Smart manufacturing, smart transportation, smart grid… Ø Example: Intelligent Transportation q Data center collects data from cameras and roadside detectors. q Control traffic signals and message signs in real-time. q Transportation information feed to drivers. q SCATS @ Sydney: controlling 3,400 signals at 1s round-trip latency. Ø Latency-sensitive applications, e.g., cloud gaming q Xbox One: cloud offloading computation of environmental elements q Sony acquired Gaikai, an open cloud gaming platform. 3

Embedded System Virtualiza7on Ø Consolidate 100 ECUs à ~10 multicore processors. Ø Integrate multiple systems on a common platform. q Infotainment on Linux or Android q Safety-critical control on AUTOSAR q Virtualization: COQOS, Integrity Multivisor, Xen automotive Ø Must preserve real-time performance on a virtualized platform! Source: h<p://www.edn.com/design/automoYve/4399434/MulYcore-and-virtualizaYon-in-automoYve-environments 10/14/16 4

Cloud is real-7me today Ø Existing hypervisors provide no guarantee on latency q Xen: credit scheduler, [credit, cap] q VMware ESXi: [reservation, share, limitation] q Microsoft Hyper-V: [reserve, weight, limit] Ø Clouds lack service level agreement on latency q EC2, Compute Engine, Azure: #VCPUs Current clouds provision resources, not latency! 5

Towards Real-Time Clouds Ø Support real-time applications in the cloud. q Latency guarantees for tasks running in virtual machines (VMs). q Real-time performance isolation between VMs. q Resource sharing between real-time and non-real-time VMs. Ø Multi-level real-time performance provisioning. q RT -Xen à real-time VM scheduling in a virtualized host. q VATC à real-time network I/O in a virtualized host. q RT -OpenStack à real-time cloud resource management. VATC: Real-Time -OpenStack Communication RT 6

Xen Virtualiza7on Architecture Ø Xen: type-1, baremetal hypervisor q Domain-0: drivers, tool stack to control VMs. q Guest Domain: para-virtualized or fully virtualized OS. Ø Scheduling hierarchy q Xen schedules VCPUs on PCPUs. q Guest OS schedules threads on VCPUs. q Xen credit scheduler: round-robin with proportional share. VCPU Real-Time Task OS Sched OS Sched OS Sched Xen Scheduler PCPUs 7

RT-Xen Ø Real-time schedulers in the Xen hypervisor. Ø Provide real-time guarantees to tasks in VMs. Ø Incorporated in Xen 4.5 as the real-time scheduler. RT-Xen h<ps://sites.google.com/site/realYmexen/ S. Xi, M. Xu, C. Lu, L. Phan, C. Gill, O. Sokolsky and I. Lee, Real-Time Multi-Core Virtual Machine Scheduling in Xen, ACM International Conference on Embedded Software (EMSOFT'14), October 2014. 8

Composi7onal Scheduling Ø Analytical real-time guarantees to tasks running in VMs. Ø VM resource interfaces q A set of VCPUs each with resource demand <period, budget > q Hides task-specific information q Computed based on compositional scheduling analysis Hypervisor Resource Interface Scheduler Resource Interface Resource Interface Scheduler Scheduler Workload Workload Virtual Machines 9

Real-Time Scheduler Design Ø Global scheduling q Allow VCPU migration across cores q Work conserving – utilize any available cores q Migration overhead and cache penalty Ø Partitioned scheduling q Assign and bind VCPUs to cores q Cores may idle when others have work pending q No migration overhead or cache penalty Ø Enforce resource interface through budget management q Periodic server vs. deferrable server Ø Priority: Earliest Deadline First vs. Deadline Monotonic 10

RT-Xen vs. Xen • Xen misses deadlines at 22% of CPU capacity. • RT -Xen delivers real-time performance at 78% of CPU capacity. 11

Virtualized Network I/O Ø Xen handles all network traffic through Dom0 Ø Real-time and non-real-time traffic share Dom0 q CPU and network contention Ø Long delays for real-time traffic in virtualized hosts Dom0 Dom1 Dom2 Network Non Real-Time Real-Time Components App App Xen Hypervisor CPU Memory Storage NIC 12

Network I/O in Virtualized Hosts Dom1 Dom2 Ø Linux Queueing Discipline Non- Real- q Rate-limit and shape flows Real- Time Time App App q Prioritization or fair packet scheduling Ø Priority inversion in virtualization Virtualiza7on Components components Dom0 q between transmissions q between transmission and reception Queueing Discipline Ø VATC: Virtualization-Aware Traffic Control q Process packets in prioritized kernel threads q Dedicated packet queues per priority NIC C. Li, S. Xi, C. Lu, C. Gill and R. Guerin, Prioritizing Soft Real-Time Network Traffic in Virtualized Hosts Based on Xen, IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS'15), April 2015. 13

Real-Time Traffic Latency VATC reduces priority inversion à lower latency for real-time traffic. 4 Prio, Dom0 − 3.18 3.5 FQ_CoDel, Dom0 − 3.18 VATC Round − trip Latency ( ms) 3 2.5 2 1.5 1 0.5 0 10 16 32 64 128 256 512 1024 Dyn Cons Dyn Interrupt Interval ( µ s) • Median round-trip latency of real-Yme traffic. • CPU contenYon from two small-packet interfering streams. 14

Virtualized Host à Cloud Ø Provide real-time performance to real-time VMs Ø Achieve high resource utilization 15

OpenStack Limita7ons Ø Popular open-source cloud management system Manager Ø VM resource interface q Number of VCPUs Host Host Host q Not real-time VM VM VM VM VM Ø VM-to-host mapping q Filtering (admission control) • VCPU-to-PCPU ratio (16:1), max VMs per host (50) • Coarse-grained admission control for CPU resources q Ranking (VM allocation) • Balance memory usage • No consideration of CPU resources 16

RT-OpenStack Ø Co-hosting real-time VMs with non-real-time VMs Ø Deliver real-time performance q Support RT -Xen resource interface q Real-time-aware VM-to-host mapping Ø Achieve high resource utilization q Co-locate non-real-time VMs with real-time VMs q Non-real-time VMs consume remaining resources without affecting the real-time performance of real-time VMs S. Xi, C. Li, C. Lu, C. Gill, M. Xu, L. Phan, I. Lee, O. Sokolsky, RT-OpenStack: CPU Resource Management for Real- Time Cloud Computing, IEEE International Conference on Cloud Computing (CLOUD'15), June 2015. 17

RT-OpenStack: VM-to-Host Mapping Ø Admission control: RT -Filter q Accept real-time VMs based on real-time schedulability and memory q Consider only accepted real-time VMs Ø VM allocation: RT -Weigher q Balance CPU utilization q Consider only accepted real-time VMs Resource Interface Admission Control VM Alloca7on Schedulability + Real-Time VMs {<period, budget>} CPU UYlizaYon Memory Non-Real-Time VMs Best Effort Memory Memory 18

OpenStack Ø Overload four hosts with real-time VMs à deadline misses. Ø Two hosts running non-real-time VMs only. Ø Unbalanced distribution of real-time domains. Hadoop finish Yme: 314 seconds 73% 47% 32% 75% 29% 30% 36% 31% 37% 61% 13% 19

RT-OpenStack Ø Schedulability guarantees for real-time VMs à no deadline miss. Ø Distribute real-time VMs across hosts. Ø Hadoop makes progress using remaining CPU resources. Hadoop finish Yme: 435 seconds 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 20

Conclusions Ø New applications demand real-time cloud services. q Internet-scale monitoring and control. q Latency-sensitive cloud applications. Ø Towards real-time cloud Ø RT -Xen: real-time VM scheduling in virtualized hosts. Ø VATC: real-time network I/O in virtualized hosts. Ø RT -OpenStack: real-time guarantees at high CPU utilization. Ø RTM: real-time messaging. VATC: Real-Time -OpenStack Communication RT 21