Mutual Exclusion, Async Completions Why We Wait 7C. Asynchronous - PDF document

4/15/2017 Mutual Exclusion, Async Completions Why We Wait 7C. Asynchronous Event Completions • We await completion of non-trivial operations 7D. Mutual Exclusion – data to be read from disk – a child process to be created 7E. Implementing Mutual Exclusion • We wait for important events 7F. Asynchronous completion operations – a request/notification from another process 7G. Implementing Asynchronous Completion – an out-of-band error that must be handled • We wait to ensure correct ordering – B cannot be performed until A has completed – if A precedes B, B must see the results of A Mutual Exclusion and Asynchronous Completions 1 Mutual Exclusion and Asynchronous Completions 2 Correct Ordering Problem 2: asynchronous completion • most procedure calls are synchronous server client (3 threads) – we call them, they do their job, they return 1. shutdown send shutdown – when the call returns, the result is ready • many operations cannot happen immediately 1. process cmd 2. status log status 2. return status – waiting for a held lock to be released 3. exit process – waiting for an I/O operation to complete 3. SIGCHLD exit process – waiting for a response to a network request – delaying execution for a fixed period of time “Surely the final status message will be received and processed • we call such completions asynchronous before the SIGCHLD causes the client to shut down! “ Mutual Exclusion and Asynchronous Completions 3 Mutual Exclusion and Asynchronous Completions 4 Approaches to Waiting Condition Variables • create a synchronization object • spinning … “busy waiting” – associate that object with a resource or request – works well if event is independent and prompt – requester blocks awaiting event on that object – wasted CPU, memory, bus bandwidth – upon completion, the event is "posted" – may actually delay the desired event – posting event to object unblocks the waiter • yield and spin … “are we there yet?” – allows other processes access to CPU wait – wasted process dispatches – works very poorly for multiple waiters signal • either may still require mutual exclusion Mutual Exclusion and Asynchronous Completions 5 Mutual Exclusion and Asynchronous Completions 6 1

4/15/2017 Awaiting Asynchronous Events The Mutual Exclusion Challenge pthread_mutex_t lock = PTHEAD_MUTEX_INITIALIZER; • We cannot prevent parallelism pthread_cond_t cond = PTHEAD_COND_INITIALIZER; – it is fundamental to our technology … pthread_mutex_lock(&lock); • We cannot eliminate all shared resources while (ready == 0) – increasingly important to ever more applications pthread_cond_wait(&cond, &lock); pthread_mutex_unlock(&lock) • What we can do is ... … – identify the at risk resources, and risk scenarios if (pthread_mutex_lock(&lock)) { ready = 1; – design those classes to enable protection pthread_cond_signal(&cond); – identify all of the critical sections pthread_mutex_unlock(&lock); – ensure each is correctly protected (case by case) } Mutual Exclusion and Asynchronous Completions 7 Mutual Exclusion and Asynchronous Completions 8 Evaluating Mutual Exclusion Approaches • Effectiveness/Correctness • Avoid shared mutable resources – the best choice … if it is an option – ensures before-or-after atomicity • Interrupt Disables • Fairness – a good tool with limited applicability – no starvation (un-bounded waits) • Spin Locks • Progress – very limited applicability – no client should wait for an available resource • Atomic Instructions – susceptibility to convoy formation, deadlock – very powerful, but difficult w/limited applicability • Performance • Mutexes – delay, instructions, CPU load, bus load – higher level, broad applicability – in contended and un-contended scenarios Mutual Exclusion and Asynchronous Completions 9 Mutual Exclusion and Asynchronous Completions 10 What Happens During an Interrupt? Approach: Interrupt Disables • Interrupt controller requests CPU for service • temporarily block some or all interrupts – can be done with a privileged instruction • CPU stops the executing program – side-effect of loading new Processor Status • Interrupt vector table is consulted • abilities – PC/PS of Interrupt Service Routine (ISR) – prevent Time-Slice End (timer interrupts) • ISR handles the interrupt (just like a trap) – save regs, find/call 2 nd level handler, restore regs – prevent re-entry of device driver code • dangers • Upon return, CPU state is restored – may delay important operations – code resumes w/no clue it was interrupted – a bug may leave them permanently disabled Mutual Exclusion and Asynchronous Completions 11 Mutual Exclusion and Asynchronous Completions 12 2

4/15/2017 Preventing Preemption Preventing Driver Reentrancy zz_io_startup ( struct iorq *bp ) { DLL_insert(DLL *head, DLL*element) { … int save = disableInterrupts(); save = intr_enable( ZZ_DISABLE ); DLL *last = head->prev; element->prev = last; /* program the DMA request */ element->next = head; zzSetReg(ZZ_R_ADDR, bp->buffer_start ); zz_intr_handler () { DLL_insert(DLL *head, DLL*element) { zzSetReg(ZZ_R_LEN, bp->buffer_length); last->next = element; zzSetReg(ZZ_R_BLOCK, bp->blocknum); … DLL *last = head->prev; head->prev = element; zzSetReg(ZZ_R_CMD, bp->write? /* update data read count */ element->prev = last; resid = zzGetReg(ZZ_R_LEN); ZZ_C_WRITE : ZZ_C_READ ); } element->next = head; zzSetReg(ZZ_R_CTRL, ZZ_INTR+ZZ_GO); last->next = element; /* turn off device ability to interrupt */ /* reenable interrupts */ zzSetReg(ZZ_R_CTRL, ZZ_NOINTR); head->prev = element; … intr_enable( save ); } Serious consequences could result if the interrupt handler was called while we were half-way through programming the DMA operation. restoreInterrupts(save); Mutual Exclusion and Asynchronous Completions 13 Mutual Exclusion and Asynchronous Completions 14 Preventing Driver Reentrancy Interrupts and Resource Allocation … • interrupts are usually self-disabling lock(event_list); – CPU may not deliver #2 until #1 is acknowledged add_to_queue(event_list, my_proc); – interrupt vector PS usually disables causing intr unlock(event_list); xx_interrupt: yield(); … • they are restored after servicing is complete … lock(event_list); – ISR may explicitly acknowledge the interrupt post(event_list); return; – return from ISR will restore previous (enabled) PS • drivers usually disable during critical sections – updating registers used by interrupt handlers – updating resources used by interrupt handlers Mutual Exclusion and Asynchronous Completions 15 Mutual Exclusion and Asynchronous Completions 16 Interrupts and Resource Allocation Evaluating Interrupt Disables • Effectiveness/Correctness • interrupt handlers are not allowed to block – ineffective against MP/device parallelism – only a scheduled process/thread can block – only usable by kernel mode code – interrupts are disabled until call completes • Progress • ideally they should never need to wait – deadlock risk (if ISR can block for resources) – needed resources are already allocated • Fairness – operations implemented w/lock-free code – pretty good (assuming disables are brief) • brief spins may be acceptable • Performance – wait for hardware to acknowledge a command – one instruction, much cheaper than system call – wait for a co-processor to release a lock – long disables may impact system performance Mutual Exclusion and Asynchronous Completions 17 Mutual Exclusion and Asynchronous Completions 18 3