Hows the Parallel Computing Revolution Going? Towards Parallel, - PowerPoint PPT Presentation

How’s the Parallel Computing Revolution Going? Towards Parallel, Scalable VM Services Kathryn S McKinley The University of Texas at Austin Kathryn McKinley Towards Parallel, Scalable VM Services 1

20 th Century Simplistic Hardware View Faster Processors Frequency Scaling Speculation, OO programs do not change they just run faster Kathryn McKinley Towards Parallel, Scalable VM Services 2

Programming Language Evolution Managed Programming Languages Native Programming Languages 3

20 th Century Simplistic Software View Larger, More Capable Software Managed Languages hardware does not change it just runs faster Kathryn McKinley Towards Parallel, Scalable VM Services 4

Processor Technology Evolution i5 Clarkdale Core 2 Duo (32nm) Core 2 Duo Wolfdale 2010 Conroe Power 5 (45nm) 2009 (65nm) 2 cores 2006 (90nm) 2004 i7 Bloomfield (45nm) 2008 Pentium M Dothan (90nm) 2005 Atom Diamondville Pentium 4 NetBurst (45nm) 2008 (130nm) 2003 5

The 20 th Century Virtuous Cycle ✓ Larger, More Faster Single Capable Processor Software Frequency Scaling Managed Languages Kathryn McKinley Towards Parallel, Scalable VM Services 6

The 21 st Century Virtuous Cycle? ? Scalable Software More Cores Scalable Apps + Chip Multiproccesors CMP Scalable Runtime Kathryn McKinley Towards Parallel, Scalable VM Services 7

How is this new virtuous cycle going? Kathryn McKinley Towards Parallel, Scalable VM Services 8

Measured Power vs Performance 2003 50 2008 Power (W) (log) 2006 ? ? 2010 Pentium 4 (130nm) 2008 Core 2 Duo (65nm) i7 (45nm) Core 2 Duo (45nm) i5 (32nm) 10 0.5 1 5 10 Speedup (v Atom 230) (log) SPEC CPU 2006, DaCapo, SPEC jvm98

How is this new virtuous cycle going for single threaded Java Kathryn McKinley Towards Parallel, Scalable VM Services 10

Performance Scaling Single Threaded Java Benchmarks Core i7: 4 cores, 2 way SMT 4.0 3.5 antlr Speedup 3.0 bloat compress 2.5 db fop 2.0 jack javac 1.5 jess mpegaudio 1.0 pmd raytrace 0.5 geomean 1 2 3 4 5 6 7 8 9 Hardware Contexts Kathryn McKinley Towards Parallel, Scalable VM Services 11

How is this new virtuous cycle going for multi-threaded Java Kathryn McKinley Towards Parallel, Scalable VM Services 12

Performance Scaling Multi-Threaded Java Benchmarks Core i7: 4 cores, 2 way SMT 4.0 avrora 3.5 batik Speedup eclipse 3.0 h2 jython 2.5 luindex lusearch 2.0 mtrt pjbb2005 1.5 sunflow tomcat 1.0 tradebeans tradesoap 0.5 xalan 1 2 3 4 5 6 7 8 9 geomean Hardware Contexts Kathryn McKinley Towards Parallel, Scalable VM Services 13

Power, Performance, and Concurrency Native Java Single threaded hollow; multithreaded solid • Microarchitecture changes from Pentium 4 (130) to i5 (32) • favored parallelism-no surprise Multithreaded performance incurs a significant power cost • 14

Is there hope? Kathryn McKinley Towards Parallel, Scalable VM Services 15

Managed Languages Challenges & Opportunities Kathryn McKinley Towards Parallel, Scalable VM Services 16

Must Start with a Scalable Managed Runtime Kathryn McKinley Towards Parallel, Scalable VM Services 17

Sequential Managed Programs time Single Managed Application Core Runtime • Profiling • Dynamic Analysis • Compilation • Garbage Collection • Other Helper Threads • …… Kathryn McKinley Towards Parallel, Scalable VM Services 18

Steps towards scalability Step 1. Parallel application time Core 0 Core 1 Core 2 Application Core 3 Threads Core 4 Core 5 Core 6 Unused cores Core 7 Each thread has different running time Kathryn McKinley Towards Parallel, Scalable VM Services 19

Steps towards scalability Step 2. Parallel runtime time Core 0 Core 1 Core 2 Application Managed Application Core 3 Runtime Threads Threads Core 4 Core 5 Core 6 Core 7 Runtime waits for all application threads to pause Kathryn McKinley Towards Parallel, Scalable VM Services 20

Steps towards scalability Step 3. Parallel & concurrent runtime time Core 0 Core 1 Core 2 Application Managed Application Core 3 Threads Runtime Threads Core 4 Core 5 Core 6 Core 7 Managed runtime on application’s critical path may perturb its performance Kathryn McKinley Towards Parallel, Scalable VM Services 21

Steps towards scalability Ideal model Step 4. Minimize perturbation time Core 0 Core 1 Core 2 Application Application Core 3 Threads Threads Core 4 Core 5 Whole runtime task taken off critical path Core 6 Core 7 Application offloads work to concurrent runtime threads Kathryn McKinley Towards Parallel, Scalable VM Services 22

Steps towards scalability Ideal model Step 4. Minimize perturbation time Core 0 Core 1 Core 2 Application Application Core 3 Threads Threads Core 4 Core 5 Core 6 Core 7 Worst case is parallel & concurrent Kathryn McKinley Towards Parallel, Scalable VM Services 23

Vision • Scalable Runtimes – Runtime & application parallelism & concurrency – CMP aware runtime improves application scalability • Communication – Cache coherency is expensive and performance sensitive – Memory bandwidth scaling is problematic • Heterogeneity – Move non-critical path off power-hungry cores – Smarter, more aggressive analysis • Specialization? – Tuned cores? Special purpose cores? Kathryn McKinley Towards Parallel, Scalable VM Services 24

Approach • Profiling (feedback directed optimization) – Concurrent analysis – More invasive analysis on low-power cores • GC – High performance concurrent GC – High performance non-moving GC – Reduced synchronization overheads – Distributed & scratchpad GC • JIT – Concurrent, parallel JIT – Cost-benefit shift as low-power cores used • Architecture – Tuned and/or specialized cores for runtime services – Coherence tailed for restricted, common case of GC Kathryn McKinley Towards Parallel, Scalable VM Services 25

Today • Profiling (feedback directed optimization) – Concurrent analysis – More invasive analysis on low-power cores • GC – High performance concurrent GC – High performance non-moving GC – Reduced synchronization overheads – Distributed & scratchpad GC • JIT – Concurrent, parallel JIT – Cost-benefit shift as low-power cores used • Architecture – Tuned and/or specialized cores for runtime services – Coherence tailed for restricted, common case of GC Kathryn McKinley Towards Parallel, Scalable VM Services 26

A Concurrent Dynamic Analysis Framework For CMP Hardware Jungwoo Ha Matthew Arnold U. Texas & UCS/ICI-East IBM Research Stephen M. Blackburn Kathryn S. McKinley Australian National University University of Texas at Austin Kathryn McKinley Towards Parallel, Scalable VM Services 27

Generic Sequential Analysis instrumented code (== overhead) � time Application data collection � analysis � • Difficult to optimize instrumented code • Trade accuracy for overhead (sampling) Kathryn McKinley Towards Parallel, Scalable VM Services 28

Generic Concurrent Analysis instrumented code (reduced overhead) � time Application Application (producer) enqueue � data collection � buffering � Analysis (consumer) dequeue � analysis � • Lower overhead & higher accuracy • Must deal with microarchitectural side-effects Kathryn McKinley Towards Parallel, Scalable VM Services 29

Side-effects to Avoid Core A L1 lower level L1 Core B cache(s) false & true sharing Application Analysis (Producer) (Consumer) High latency memory operation Cache line ping-ponging Kathryn McKinley Towards Parallel, Scalable VM Services 30

Cache-friendly Asymmetric Buffering • Lock-free communication channel between application and analysis thread • Cache-friendly asymmetric buffering – Actively avoids microarchitectural side-effects – Enqueue • light-weight instrumentation • produces one record at time – Dequeue • consumes one chunk (fraction of a buffer) at a time Kathryn McKinley Towards Parallel, Scalable VM Services 31

Cache-friendly Asymmetric Buffering Core A L1 lower level L1 Core B cache(s) Application Analysis (Producer) (Consumer) an analyz yzer w r wai aits s application � ap fo for application writes here � here. � application buffer 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 an analyz yzer � analyzer chunk rea eads here � • 16 slots on the buffer • 4 chunks, 4 slot on each chunk • L1 size == chunk size Kathryn McKinley Towards Parallel, Scalable VM Services 32

Cache-friendly Asymmetric Buffering Core A L1 lower level L1 Core B cache(s) 0 4 8 Application Analysis 5 1 (Producer) (Consumer) 2 6 7 3 application buffer 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 analyzer chunk Delay consumer dequeue operation until cache line is flushed • 2 chunks away (smiley location) – Analyzer operates one chunk at a time • chunk_size > L1 size – In practice, chunk_size >= 2 * L1 works well. – Kathryn McKinley Towards Parallel, Scalable VM Services 33