CPU Inheritance Scheduling Bry an F o rd Sai Susa rla - PDF document

CPU Inheritance Scheduling Bry an F o rd Sai Susa rla Computer Systems Lab o rato ry Depa rtment of Computer Science Universit y of Utah flux@cs.utah.edu http://www.cs.utah.edu/projects/flu x/ Octob er 30, 1996 1

Key Concepts Threads schedule each other b y donating the CPU using a directed yield p rimiti ve. One ro ot scheduler thread p er p ro cesso r sources all CPU time. Kernel dispatcher manages threads, events, and CPU donation without making any scheduling p olicy decisions. 2

The Dispatcher Implements thread sleep, w ak eup, schedule, etc. Runs in the context of currently running thread. Has no notion of thread p rio rit y , CPU usage, clo cks, o r timers. Dispatcher w ak es a scheduler thread when: � Scheduler's client blo cks. � Event of interest to the scheduler o ccurs. 3

Scheduling Example Scheduler Scheduler Ready thread CPU queues scheduling requests CPU donation Port 4 Running thread Ready threads Waiting thread App 1 App 2

The schedule() op eration schedule(thr ead , port, sensitivity ) Sensitivit y levels: � ON BLOCK : W ak e the scheduler any time its client thread blo cks. � ON SWITCH : W ak e the scheduler only when a di�erent client is requesting the CPU. � ON CONFLICT : W ak e the scheduler only when t w o o r mo re clients a re runnable at the same time. 5

Implicit Donation W o rks lik e schedule() , except done implici tl y; e.g.: � Thread attempting to lo ck a held mutex donates to current o wner � Client thread donates to server thread fo r the duration of an RPC Analogous to p rio rit y inheritance in traditional systems. 6 CPU S0 T0 (high-priority) T1 (low-priority)

Multip ro cesso r Scheduling Scheduler Scheduler CPU 0 threads Ready queues CPU 1 7 App 1 App 2

Bene�ts � Hiera rchical, stack able scheduling p olicies � Application-sp eci�c scheduling p olicies � Mo dula r CPU usage control � Automatic p rio rit y inheritance � Accurate CPU usage accounting � Naturally extends to multip ro cesso rs � Supp o rts p ro cesso r a�nit y p olicies and scheduler activations 8

Protot yp e Implementation Implemented as a fancy threads pack age in a BSD p ro cess. Schedulers implemented: � Fixed p rio rit y round-robin and FIF O � Rate monotonic � Lottery 9

Scheduling Hiera rchy RM1 Real-time Scheduler Root Scheduler Rate-monotonic Fixed-priority Real-time periodic threads RM2 Timesharing Class Lottery scheduling LS1 Web browser JAVA1 Lottery scheduling Java applet threads 10 JAVA2 FIFO Scheduler FIFO1 Background Non-preemptive Round-robin Cooperating threads RR1 FIFO2 RR2

Results Three measures: � Scheduling b ehavio r (co rrectness) � Overhead � Implementation complexit y 11

Multi-policy Scheduling Behavior 2.5 Rate-monotonic thread 1 (50%) Rate monotonic thread 2 (25%) Lottery thread (Interactive - bursty) Accumulated CPU usage (sec) 2 Round-robin thread 1 (Insatiable) Round-robin thread 2 (Insatiable) 1.5 RM1 (50%) 1 RR1 0.5 (compute) RM2 (25%) LS1 (burst) RR2 (compute) 0 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91 94 97 100 Time (clock ticks)

Modular Control of CPU Usage 100 Relative CPU time allocation (percent) 90 Applet thread 1 80 70 60 Applet thread 2 50 FIFO thread 1 40 FIFO thread 2 30 Round-robin thread 1 20 10 Round-robin thread 2 0 200 600 1000 1400 1800 2200 2600 3000 3400 3800 4200 4600 5000 5400 5800 6200 6600 7000 7400 7800 8200 8600 9000 9400 9800 Time (clock ticks)

Real-time Scheduling Behavior 70 CPU donation on mutex contention 60 No CPU donation Number of occurrences 50 40 30 20 10 0 0 0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 11 11 12 12 13 13 14 14 15 15 Mutex lock latency for real-time thread (clock ticks)

P erfo rmance � Dispatcher overhead { Base cost { Sensitivit y to hiera rchy depth � Context switching overhead { Numb er of additional context switches { Cost of context switches 15

Dispatcher Micro-b enchma rks Scheduling Hiera rchy Depth Dispatch Time ( � s) Ro ot scheduler only 8.0 2-level scheduling 11.2 3-level scheduling 14.0 4-level scheduling 16.2 8-level scheduling 24.4 16

Context switch overhead � In p rotot yp e, measure what p rop o rtion of context switches a re to scheduler threads (i.e., extra) � On a real OS, measure rate of context switches in va rious w o rk loads � Project slo wdo wn in t w o OSs, based on ex- p ected rate and sp eed of context switches 17

Context Switches fo r Simple T ests Client/ P a rallel Real- General Server Database time RM1 57 322 101 RM2 19 26 RM3 19 LS1 25 622 17 JA V A1 46 FIF O1 9 RR1 114 238 249 7 RR2 3 242 14 RR3 234 RR4 243 User invo cations 492 957 1193 165 Ro ot scheduler 262 956 1237 142 Rate monotonic 43 1 65 Lottery scheduler 30 57 3 Applet scheduler 2 FIF O scheduler 1 Round-robin sched 8 8 8 Scheduler invo c. 346 956 1303 218 T otal csw 838 1913 2496 383 Scheduler % 41% 50% 52% 56% 18

Statistics fo r Common Application s gzip gcc tar configure Run time (sec) 26.4 35.3 9.6 26.0 Context switches/sec 11 32 81 202 T raps/sec 10 562 22 3470 System calls/sec 23 651 517 1807 Device interrupts/sec 427 509 3337 1055 19

10 Microkernel:configure (13000 csw/s) Microkernel:gcc (3500 csw/s) Microkernel:gzip (930 csw/s) FreeBSD:configure (202 csw/s) FreeBSD:gcc (32 csw/s) FreeBSD:gzip (11 csw/s) 8 Overall slowdown (percent) 6 4 2 20 0 1 10 100 1000 Additional overhead per context switch (microsec)

Co de Complexit y � Dispatcher : 550 ra w, 160 lines of semicolons � Example schedulers : each is 100{200 semicolons 21

Related W o rk Existing multi-p oli cy systems: � Multi-cl ass systems: Mach, NT � Aegis Exok ernel 22

Related W o rk Existing hiera rchical scheduling p olicies: � KeyK OS meters � Lottery/stride scheduling � Sta rt-tim e F air Queuing (SF Q) CPU inheritance scheduling is not a p olicy . 23

Status W o rks, but needs to b e tried in a real OS Fluk e k ernel implementation in p rogress Source fo r p rotot yp e will b e available from the OSDI and Flux p roject w eb pages: http://www.cs.utah.edu/projects/fl ux/ 24

Conclusion CPU inheritance scheduling: � Provides �exible CPU scheduling, and sup- p o rts many existing p olicies and mecha- nisms � Is e�cient enough fo r common uses � Is straightfo rw a rd to implement (in user mo de) � Supp o rts the Fluk e nested p ro cess mo del 25