Towards Hybrid Isolation for Shared Multicore Systems 23rd Workshop - - PowerPoint PPT Presentation

▶
towards hybrid isolation for shared multicore systems
SMART_READER_LITE
LIVE PREVIEW

Towards Hybrid Isolation for Shared Multicore Systems 23rd Workshop - - PowerPoint PPT Presentation

Apr 03, 2024 429 likes •639 views

1 Towards Hybrid Isolation for Shared Multicore Systems 23rd Workshop on Job Scheduling Strategies for Parallel Processing, (JSSPP), 2020 Yoonsung Nam , Byeonghun Yoo , Yongjun Choi , Yongseok Son , and Hyeonsang Eom Seoul

slide-1
SLIDE 1

Towards Hybrid Isolation for Shared Multicore Systems

23rd Workshop on Job Scheduling Strategies for Parallel Processing, (JSSPP), 2020

Yoonsung Nam†, Byeonghun Yoo†, Yongjun Choi†, Yongseok Son§, and Hyeonsang Eom† Seoul National University† Chung-Ang University§

1

slide-2
SLIDE 2

Co-locating Workloads in a Server Machine

Latency-critical Workloads Best-Effort Workloads

  • Colocating multiple

workloads in a server

  • Latency-critical
  • Best-effort
  • Benefits
  • Higher resource

efficiency

  • Running multiple

workloads in parallel

Multicore Server Machine

  • bject detection

web server speech recognition in-memory DB scientific simulation graph processing data analytics

2

slide-3
SLIDE 3

Shared Resource Contention in Multicore Systems

Core Core Core Shared Last Level Cache Core Memory

Private Cache Private Cache Private Cache Private Cache

…

LC Workload

  • Problem:

Resource Contention

  • Lower performance
  • Lower resource

efficiency

  • Higher service level
  • bjective violations

BE Workload

3

slide-4
SLIDE 4

Isolation Techniques

  • Software isolation techniques
  • Isolation techniques performed by software
  • cgroup is one of the representative isolation knob in Linux
  • e.g., CPU core allocation, CPU cycle limit, and Thread migration
  • Hardware isolation techniques
  • Isolation techniques performed by hardware
  • Hardware vendors provide their own isolation interfaces (e.g., Intel CAT*, Intel

per-core DVFS, and Intel RAPL**)

  • e.g, Cache partitioning and Per-core Dynamic Voltage Frequency Scaling

(DVFS)

*CAT: Cache Allocation Technology **RAPL: Running Average Power Limit

4

slide-5
SLIDE 5

Related Work

  • How these isolation techniques have been used to mitigate

resource contention?

HW Isolation Techniques SW Isolation Techniques

CPI2[EuroSys’13] (CPU cycle limit & Migration) Memguard[RTAS’13] (CPU cycle limit) Heracles[ISCA’15] (CPU allocation, Intel CAT, Network HTB*, CPUFreq.) PARTIES[ASPLOS’19] (CPU allocation, Intel CAT, Network HTB*, Disk qdisc) Dirigent[ASPLOS’16] (Intel CAT & per-core DVFS) Quasar[ASPLOS’14] (CPU allocation & Migration)

Both

*HTB: Hierarchical Token Bucket

5

slide-6
SLIDE 6

Related Work

  • How these isolation techniques have been used to mitigate

resource contention?

HW Isolation Techniques SW Isolation Techniques

CPI2[EuroSys’13] (CPU cycle limit & Migration) Memguard[RTAS’13] (CPU cycle limit) Heracles[ISCA’15] (CPU allocation, Intel CAT, Network HTB*, CPUFreq.) PARTIES[ASPLOS’19] (CPU allocation, Intel CAT, Network HTB*, Disk qdisc) Dirigent[ASPLOS’16] (Intel CAT & per-core DVFS) Quasar[ASPLOS’14] (CPU allocation & Migration)

Both

*HTB: Hierarchical Token Bucket

HIS[JSSPP’20] (CPU allocation, Intel CAT, CPU Freq, Thread Migration)

6

slide-7
SLIDE 7

Trade-offs in Existing Isolation Techniques

SW Isolation HW Isolation

H/W isolation tech. S/W isolation tech.

Cache Partitioning Per-core DVFS CPU Cycle Limit CPU Allocation Thread Migration Technology Intel CAT Intel Processor (since Haswell) cgroup:cpu cgroup:cpuset cgroup:cpuset, memory Type Partitioning Throttling Throttling Scheduling Scheduling Latency (ms)* 3 2 40~50 3 90 Configurations # of ways (20 per LLC) # of freq. (10 per core) quota/period (100) # of cores (16) # of sockets (2) Strictness High High Medium Medium Low Responsiveness High High Medium High Low Flexibility Low Low Medium High High

I’m flexible! I’m fast and strict

*Latency is time taken for each isolation technique to work on a high memory-intensive workload (SP of NPB)

7

slide-8
SLIDE 8

Ineffective Isolation

  • FG: streamcluster(MemBW-int) & canneal(LLC-int), BG: SP (MemBW-int & LLC-int)
  • Both workloads show the highest performance with the combination of different

isolation techniques (indicated by yellow stars)

  • Some isolation technique combinations show worse performance than baseline that just

allocates 8-cores evenly (indicated by red stars) 16-core Xeon Socket LLC: 40MB Mem BW: 68GB/s FG BG Partitioning(P): Intel CAT Throttling(T): Per-core DVFS Scheduling(S): CPU allocation

P T S P

8

slide-9
SLIDE 9

Trade-off: Strictness

Changes in FG’s LLC Changes in FG’s IPC Changes in BG’s LLC Changes in BG’s IPC

FG: streamcluster BG: SP

*LLC sizes are measured by Intel Resctrl

9

slide-10
SLIDE 10

Trade-off: Responsiveness

FG: Apache web server (client: ab) BG: SP 16-core Xeon Socket Apache web server SP ab

Remote 32-core Xeon machine

  • HW isolation technique (Per-core DVFS) shows lower tail latency for

the latency-critical workload than SW isolation technique (CPU cycle limit)

  • Because, the hardware isolation techniques enables quick and faster

isolation by controlling hardware directly

HW(or SW)

10

slide-11
SLIDE 11

Trade-off: Flexibility

16-core Xeon Socket canneal SP 16-core Xeon Socket swaptions nn x1.6 x1.3 x1.2 Thread Migration

11

slide-12
SLIDE 12

HIS: Hybrid Isolation System

  • Profiler: monitors resource contention for the foreground workload in

an online manner

  • Scheduler: chooses an isolation technique based on the most

contentious resource, available isolation techniques, and type of techniques (SW or HW)

FG Workload BG Workload Server Machine

Profiler Scheduler isolation techniques

User-level

Perf, /proc

OS Kernel (Linux) Feedback/Monitor Flow Control Flow

12

slide-13
SLIDE 13

HIS Online Profiler

  • Profiler:
  • Measuring how much workloads suffer

from resource contentions by calculating contention using samples

rescont = (resco-run/ressolo-run) - 1

  • Running ‘profiling mode’ if there is no

any valid profile (e.g., phase change, no profile data)

FG BG

Profiler FG BG Profiler FG BG Profiler SIGSTOP Paused Profiler FG BG SIGCONT Time

collecting ‘solo-run’ samples of FG during 2 seconds (10 samples) collecting ‘co-run’ samples of FG

If there is no any valid profile, then performing `profiling mode`

Contention Type

Metric Meaning LLC

LLC hit ratio Data reuse

Mem BW

memory BW BW consumption

CPU Core

free/allocated CPUs, active threads CPU demand

13

slide-14
SLIDE 14

HIS Scheduler

FG BG

Scheduler

FG BG

Scheduler

  • 1. Which is the most

contentious one?

  • 2. Which isolation

techniques are available?

  • 3. HW or SW Isolation?
  • 5. Which parameter?

(strengthen or weaken)

  • 6. Get feedback from

profiler

  • 4. Isolation technique is

selected

MemBW CPU allocation or Per-core DVFS CPU allocation (SW) Per-core DVFS (HW) Per-core DVFS (HW) Lower CPU freq. (2.1GHz -> 2.0GHz) 2.1GHz -> 2.0GHz If still, Mem BW contention is high then go step 2

  • 7a. Iterate step 1 ~ 6 by

the feedback

  • 7b. If contentions are

well managed within the threshold, then stop

If the highest contentions are within 5% Strengthens Memory BW Isolation!

Contention Type

Isolation techniques LLC

Intel CAT

Mem BW CPU Allocation or

Per-core DVFS

CPU Core CPU Allocation or

Thread migration

14

Sleeps 200ms

slide-15
SLIDE 15

Experimental Setup

  • Machine
  • Intel XeonE5-2683 v4 (16-core per socket, 2 sockets)
  • LLC: 40MB, RAM: 32GB, Memory BW: 68GB/s
  • Linux 4.19.0
  • Workloads
  • Multi-threaded benchmarks (PARSEC, Rodinia, NPB, Apache web server with ab)
  • Co-locates two workloads (FG and BG) in a machine
  • SP is used for BG workload, because SP shows the highest memory bandwidth and LLC usages
  • Comparing with co-running workloads with static CPU isolation
  • Batch workloads
  • Latency-critical workloads

15

slide-16
SLIDE 16

Evaluation (Batch Workload)

  • FG: benchmarks shown in x-axis & BG: SP of NPB
  • Performance is normalized to the baseline

(co-run with static CPU isolation)

  • It achieves 1.05~1.7x performance than baseline and BG shows degraded performance up to

1.3x

16

slide-17
SLIDE 17

Evaluation (Latency-critical Workload)

  • FG: Apache web benchmark (with ab) & BG: SP
  • client (ab) sends requests generated with Pareto distribution
  • Apache web benchmark achieves 2.14x performance than co-run and also shows slightly

lower latency than the solo-run (8-core) case

  • The maximum number of allocated CPU cores for web server reached to 12-cores

responding to CPU demands

17

slide-18
SLIDE 18

Future work and Conclusion

  • Future Work
  • Testing with other isolation policies (e.g., selecting isolation techniques)
  • Testing with more diverse benchmarks (e.g., phase-changing workloads)
  • Comparing with recent works (e.g., Heracles-like system)
  • Conclusion
  • We have analyzed the tradeoffs in HW/SW isolation techniques
  • We have implemented a prototype for hybrid isolation system
  • We have evaluated our prototype using selected benchmarks and

achieved 1.7~2.14x performance improvement than static isolation

18

slide-19
SLIDE 19

Thank you! Questions?

Yoonsung Nam DCSLab., Seoul National University yoonsung.nam@snu.ac.kr

19