Real-Time Cloud Computing Chenyang Lu Cyber-Physical Systems - - PowerPoint PPT Presentation

▶
real time cloud computing
SMART_READER_LITE
LIVE PREVIEW

Real-Time Cloud Computing Chenyang Lu Cyber-Physical Systems - - PowerPoint PPT Presentation

Sep 11, 2022 260 likes •492 views

Real-Time Cloud Computing Chenyang Lu Cyber-Physical Systems Laboratory Department of Computer Science and Engineering h<p://www.cse.wustl.edu/~lu/ Internet of Things Convergence of q Miniaturized hardware: integrate processor, sensors

slide-1
SLIDE 1

Real-Time Cloud Computing

Chenyang Lu

Cyber-Physical Systems Laboratory Department of Computer Science and Engineering h<p://www.cse.wustl.edu/~lu/

slide-2
SLIDE 2

Internet of Things

Ø Convergence of

q Miniaturized hardware: integrate processor, sensors and radios. q Low-power wireless: connect millions of devices to the Internet. q Data analytics: make sense of sensor data. q Rea-time cloud: scalable real-time computing.

Ø Enable real-time control of physical environments

  • R. Dor, G. Hackmann, Z. Yang, C. Lu, Y. Chen, M.

Kollef and T.C. Bailey, Experiences with an End-To- End Wireless Clinical Monitoring System, Conference

  • n Wireless Health (WH'12), October 2012.
slide-3
SLIDE 3

Real-Time Cloud

Ø Internet of Things à large-scale sensing and control

q Real-time data analytics (aka Big Data) q Smart manufacturing, smart transportation, smart grid…

Ø Example: Intelligent Transportation

q Data center collects data from cameras and roadside detectors. q Control traffic signals and message signs in real-time. q Transportation information feed to drivers. q SCATS @ Sydney: controlling 3,400 signals at 1s round-trip latency.

Ø Latency-sensitive applications, e.g., cloud gaming

q Xbox One: cloud offloading computation of environmental elements q Sony acquired Gaikai, an open cloud gaming platform.

3

slide-4
SLIDE 4

Embedded System Virtualiza7on

Ø Consolidate 100 ECUs à ~10 multicore processors. Ø Integrate multiple systems on a common platform.

q Infotainment on Linux or Android q Safety-critical control on AUTOSAR q Virtualization: COQOS, Integrity Multivisor, Xen automotive

Ø Must preserve real-time performance on a virtualized platform!

10/14/16 4 Source: h<p://www.edn.com/design/automoYve/4399434/MulYcore-and-virtualizaYon-in-automoYve-environments

slide-5
SLIDE 5

Ø Existing hypervisors provide no guarantee on latency

q Xen: credit scheduler, [credit, cap] q VMware ESXi: [reservation, share, limitation] q Microsoft Hyper-V: [reserve, weight, limit]

Ø Clouds lack service level agreement on latency

q EC2, Compute Engine, Azure: #VCPUs

5

Cloud is real-7me today

Current clouds provision resources, not latency!

slide-6
SLIDE 6

Towards Real-Time Clouds

Ø Support real-time applications in the cloud.

q Latency guarantees for tasks running in virtual machines (VMs). q Real-time performance isolation between

VMs.

q Resource sharing between real-time and non-real-time

VMs.

Ø Multi-level real-time performance provisioning.

q RT

  • Xen à real-time

VM scheduling in a virtualized host.

q VATC à real-time network I/O in a virtualized host. q RT

  • OpenStack à real-time cloud resource management.

6

VATC: Real-Time Communication RT

  • OpenStack
slide-7
SLIDE 7

Xen Virtualiza7on Architecture

Ø Xen: type-1, baremetal hypervisor

q Domain-0: drivers, tool stack to control

VMs.

q Guest Domain: para-virtualized or fully virtualized OS.

Ø Scheduling hierarchy

q Xen schedules

VCPUs on PCPUs.

q Guest OS schedules threads on

VCPUs.

q Xen credit scheduler: round-robin with proportional share.

7

PCPUs

OS Sched

Xen Scheduler

OS Sched OS Sched

VCPU Real-Time Task

slide-8
SLIDE 8

RT-Xen

Ø Real-time schedulers in the Xen hypervisor. Ø Provide real-time guarantees to tasks in VMs. Ø Incorporated in Xen 4.5 as the real-time scheduler.

8

RT-Xen

h<ps://sites.google.com/site/realYmexen/

  • S. Xi, M. Xu, C. Lu, L. Phan, C. Gill, O. Sokolsky and I. Lee, Real-Time Multi-Core Virtual Machine

Scheduling in Xen, ACM International Conference on Embedded Software (EMSOFT'14), October 2014.

slide-9
SLIDE 9

Composi7onal Scheduling

Ø Analytical real-time guarantees to tasks running in VMs. Ø VM resource interfaces

q A set of

VCPUs each with resource demand <period, budget >

q Hides task-specific information q Computed based on compositional scheduling analysis

9

Resource Interface Resource Interface Resource Interface

Hypervisor Virtual Machines

Workload Workload

Scheduler Scheduler Scheduler

slide-10
SLIDE 10

Real-Time Scheduler Design

Ø Global scheduling

q Allow

VCPU migration across cores

q Work conserving – utilize any available cores q Migration overhead and cache penalty

Ø Partitioned scheduling

q Assign and bind

VCPUs to cores

q Cores may idle when others have work pending q No migration overhead or cache penalty

Ø Enforce resource interface through budget management

q Periodic server vs. deferrable server

Ø Priority: Earliest Deadline First vs. Deadline Monotonic

10

slide-11
SLIDE 11

RT-Xen vs. Xen

11

  • Xen misses deadlines at 22% of CPU capacity.
  • RT
  • Xen delivers real-time performance at 78% of CPU capacity.
slide-12
SLIDE 12

Virtualized Network I/O

Ø Xen handles all network traffic through Dom0 Ø Real-time and non-real-time traffic share Dom0

q CPU and network contention

Ø Long delays for real-time traffic in virtualized hosts

Xen Hypervisor

Network

Components

Non Real-Time App

Dom2

CPU Memory Storage

Real-Time App

Dom1 Dom0

NIC

12

slide-13
SLIDE 13

Network I/O in Virtualized Hosts

Ø Linux Queueing Discipline

q Rate-limit and shape flows q Prioritization or fair packet scheduling

Ø Priority inversion in virtualization components

q between transmissions q between transmission and reception

Ø VATC: Virtualization-Aware Traffic Control

q Process packets in prioritized kernel threads q Dedicated packet queues per priority

NIC

Queueing Discipline

Dom0

Real- Time App

Dom1

Non- Real- Time App

Dom2

Virtualiza7on Components

13

  • C. Li, S. Xi, C. Lu, C. Gill and R. Guerin, Prioritizing Soft Real-Time Network Traffic in

Virtualized Hosts Based on Xen, IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS'15), April 2015.

slide-14
SLIDE 14

Real-Time Traffic Latency

VATC reduces priority inversion à lower latency for real-time traffic.

14 10 16 32 64 128 256 512 1024 Dyn Cons Dyn 0.5 1 1.5 2 2.5 3 3.5 4 Round−trip Latency ( ms) Interrupt Interval (µs) Prio, Dom0−3.18 FQ_CoDel, Dom0−3.18 VATC

  • Median round-trip latency of real-Yme traffic.
  • CPU contenYon from two small-packet interfering streams.
slide-15
SLIDE 15

Virtualized Host à Cloud

Ø Provide real-time performance to real-time VMs Ø Achieve high resource utilization

15

slide-16
SLIDE 16

OpenStack Limita7ons

Ø Popular open-source cloud management system Ø VM resource interface

q Number of

VCPUs

q Not real-time

Ø VM-to-host mapping

q Filtering (admission control)

  • VCPU-to-PCPU ratio (16:1), max

VMs per host (50)

  • Coarse-grained admission control for CPU resources

q Ranking (VM allocation)

  • Balance memory usage
  • No consideration of CPU resources

16

Manager

Host Host Host VM VM VM VM VM

slide-17
SLIDE 17

RT-OpenStack

Ø Co-hosting real-time VMs with non-real-time VMs Ø Deliver real-time performance

q Support RT

  • Xen resource interface

q Real-time-aware

VM-to-host mapping

Ø Achieve high resource utilization

q Co-locate non-real-time

VMs with real-time VMs

q Non-real-time

VMs consume remaining resources without affecting the real-time performance of real-time VMs

17

  • S. Xi, C. Li, C. Lu, C. Gill, M. Xu, L. Phan, I. Lee, O. Sokolsky, RT-OpenStack: CPU Resource Management for Real-

Time Cloud Computing, IEEE International Conference on Cloud Computing (CLOUD'15), June 2015.

slide-18
SLIDE 18

RT-OpenStack: VM-to-Host Mapping

Ø Admission control: RT

  • Filter

q Accept real-time

VMs based on real-time schedulability and memory

q Consider only accepted real-time

VMs

Ø VM allocation: RT

  • Weigher

q Balance CPU utilization q Consider only accepted real-time

VMs

18

Resource Interface Admission Control VM Alloca7on Real-Time VMs {<period, budget>} Schedulability + Memory CPU UYlizaYon Non-Real-Time VMs Best Effort Memory Memory

slide-19
SLIDE 19

OpenStack

19

13% 36% 31% 61% 37% 75% 30% 29% 47% 32% 73%

Hadoop finish Yme: 314 seconds

Ø Overload four hosts with real-time VMs à deadline misses. Ø Two hosts running non-real-time VMs only. Ø Unbalanced distribution of real-time domains.

slide-20
SLIDE 20

RT-OpenStack

20

0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%

Hadoop finish Yme: 435 seconds

Ø Schedulability guarantees for real-time VMs à no deadline miss. Ø Distribute real-time VMs across hosts. Ø Hadoop makes progress using remaining CPU resources.

slide-21
SLIDE 21

Conclusions

Ø New applications demand real-time cloud services.

q Internet-scale monitoring and control. q Latency-sensitive cloud applications.

Ø Towards real-time cloud

Ø RT

  • Xen: real-time

VM scheduling in virtualized hosts.

Ø VATC: real-time network I/O in virtualized hosts. Ø RT

  • OpenStack: real-time guarantees at high CPU utilization.

Ø RTM: real-time messaging.

21

VATC: Real-Time Communication RT

  • OpenStack