Summary Summary What you need to know about concurrency What you - PDF document

Herb Sutter Software and the Concurrency Revolution Software and the Software and the Concurrency Revolution Concurrency Revolution Herb Sutter Herb Sutter Software Architect Software Architect Microsoft Developer Division Microsoft Developer Division Summary Summary What you need to know about concurrency What you need to know about concurrency It’ It ’s here s here parallelism has long been the parallelism has long been the “ “next big thing next big thing” ” – – the future is now the future is now everybody’ ’s doing it (because they have to) s doing it (because they have to) everybody It will directly affect the way we write software It will directly affect the way we write software the free lunch is over the free lunch is over – – for sequential CPU for sequential CPU- -bound apps bound apps only apps with lots of latent concurrency regain the perf. free lunch only apps with lots of latent concurrency regain the perf. free lunch (side benefit: responsiveness, the other reason to want async code) de) (side benefit: responsiveness, the other reason to want async co languages won’ ’t be able to ignore it and stay relevant t be able to ignore it and stay relevant languages won The software industry has a lot of work to do The software industry has a lot of work to do a generational advance >OO to move beyond “ a generational advance >OO to move beyond “threads+locks threads+locks” ” key: incrementally adoptable extensions for existing languages key: incrementally adoptable extensions for existing languages 1 1

Herb Sutter Software and the Concurrency Revolution Truths Truths Consequences Consequences Futures Futures 2 faster more more processors Each year we get faster processors Each year we get • • Historically: Historically: Boost single Boost single- - Moore’s stream performance via stream performance via Law Montecito more complex chips, first more complex chips, first Intel CPU Trends via one big feature, then via one big feature, then (sources: Intel, Wikipedia, K. Olukotun) via lots of smaller features. via lots of smaller features. • Now: Now: Deliver more cores Deliver more cores • per chip. per chip. Pentium • The free lunch is over for • The free lunch is over for today’ today ’s sequential apps s sequential apps 386 and and many concurrent apps many concurrent apps (expect some regressions). (expect some regressions). We need killer apps with We need killer apps with lots of latent parallelism. lots of latent parallelism. • A generational advance A generational advance • >OO is necessary to get >OO is necessary to get above the “ “threads+locks threads+locks” ” above the programming model. programming model. 3 2

Herb Sutter Software and the Concurrency Revolution A Baseline Client Growth Projection A Baseline Client Growth Projection 32 32 OoO - cores 16 16 You are 8 8 here 4 2 2006 2007 2008 2009 2010 2011 2012 2013 4 Two Forces and a Potential Vicious Cycle Two Forces and a Potential Vicious Cycle Moore’s Law Moore’s Law < memory < memory > transistors > transistors bandwidth bandwidth per core per core > cores > cores per chip per chip trade off trade off > hardware > hardware < latency for < latency for parallelism parallelism > bandwidth > bandwidth < transistors < transistors per core per core > threads > threads back to back to per core to per core to In-Order cores? In-Order cores? hide latency hide latency one-time 16x one-time 16x (4x cores, (4x cores, 4x threads) 4x threads) 5 3

Herb Sutter Software and the Concurrency Revolution Potential Client Growth Envelope Potential Client Growth Envelope InO - threads 512 InO - cores the truth is somewhere in here OoO - cores 256 256 You are 128 128 here 64 64 32 32 32 16 8 8 16 4 2 2006 2007 2008 2009 2010 2011 2012 2013 6 The Issue Is (Mostly) On the Client The Issue Is (Mostly) On the Client What’ ’s s “ “already solved already solved” ” and what and what’ ’s not s not What “Solved “ Solved” ”: Server apps (e.g., database servers, web services) : Server apps (e.g., database servers, web services) lots of independent requests – lots of independent requests – one thread per request is easy one thread per request is easy typical to execute many copies of the same code typical to execute many copies of the same code shared data usually via structured databases shared data usually via structured databases (automatic implicit concurrency control via transactions) (automatic implicit concurrency control via transactions) ⇒ with some care, with some care, “ “concurrency problem is already solved concurrency problem is already solved” ” here here ⇒ Not solved: Typical client apps (i.e., not Photoshop) Not solved: Typical client apps (i.e., not Photoshop) somehow employ many threads per user “ “request request” ” somehow employ many threads per user highly atypical to execute many copies of the same code highly atypical to execute many copies of the same code shared data in memory, unstructured and promiscuous shared data in memory, unstructured and promiscuous (error prone explicit locking – (error prone explicit locking – where are the transactions?) where are the transactions?) also: legacy requirements to run on a given thread (e.g., GUI) also: legacy requirements to run on a given thread (e.g., GUI) 7 4

Herb Sutter Software and the Concurrency Revolution Dealing With Ambiguity Dealing With Ambiguity Sequential Programs Concurrent Programs Behavior Deterministic Nondeterministic Memory Stable In flux (unless private, read-only, or protected by lock) Locks Unnecessary Essential (in some form) Invariants Must hold only on Must hold anytime the method entry/exit, or protecting lock is not held calls to external code Deadlock Impossible Possible anytime there are multiple unordered locks Testing Code coverage finds Code coverage insufficient, races most bugs, stress cause hard bugs, and stress testing testing proves quality gives only probabilistic comfort Debugging Trace execution leading Postulate a race and inspect code; to failure; finding a fix root causes easily remain is generally assured unidentified (hard to reproduce, hard to go back in time) 8 Problem 1 (of 2): Threads Threads Problem 1 (of 2): Problem: Unstructured free threading. Problem: Unstructured free threading. • Unconstrained. Arbitrary reentrancy, blocking, affinity. • Unconstrained. Arbitrary reentrancy, blocking, affinity. Today: Mitigate by (often) hand-coded patterns. Today: Mitigate by (often) hand-coded patterns. • Use messages (and variants, e.g., pipelines): • Use messages (and variants, e.g., pipelines): Clearer and easier to reason about successfully. Clearer and easier to reason about successfully. • Use work queues: Manual decomposition of work + • Use work queues: Manual decomposition of work + rightsized thread pool, sometimes semiautomated (e.g., rightsized thread pool, sometimes semiautomated (e.g., BackgroundWorker). BackgroundWorker). Tomorrow: Tomorrow: • Enable better abstractions: • Enable better abstractions: – Active objects with implicit messages. – Active objects with implicit messages. – Futures. – Futures. • (“Don’t roll your own vtables.”) • (“Don’t roll your own vtables.”) 9 5