MLP yes! Definitions ILP no ! MLP ILP = Instruction Level = Memory - - PowerPoint PPT Presentation

▶
mlp yes
SMART_READER_LITE
LIVE PREVIEW

MLP yes! Definitions ILP no ! MLP ILP = Instruction Level = Memory - - PowerPoint PPT Presentation

Jun 26, 2023 318 likes •411 views

MLP yes! Definitions ILP no ! MLP ILP = Instruction Level = Memory Level Parallelism Work on memory level parallelism. Parallelism IPC metric misleading Stop worrying about IPC. = Number cache misses (Inst. per Clock) simultaneously

slide-1
SLIDE 1

MLP yes! ILP no!

Work on memory level parallelism. Stop worrying about IPC. Andy “Krazy” Glew

glew@cs.wisc.edu, glew@hf.intel.com

ASPLOS 98 Wild and Crazy Ideas Session

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 2

Definitions

MLP

= Memory Level Parallelism = Number cache misses simultaneously

  • utstanding
  • esp. for linked lists!

ILP = Instruction Level

Parallelism

IPC metric misleading

(Inst. per Clock)

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 3

Thought Experiment

Assume:

  • Memory latency (cache miss latency)

= 1000x ALU compute latency

  • Memory bandwidth easy to obtain

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 4

Why IPC is misleading

MLP ≠ IPC

  • MLP ≈ 4 cache misses outstanding
  • IPC = 4 / 1004 ≈ 0.004
  • Narrow machine ≈ Wide superscalar
  • e.g. AXPY (trivial MLP)

band- width

  • k

1000 cycles per cache miss 1 operation dispatch per cycle 1 operation complete per cycle

... ...

slide-2
SLIDE 2

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 5

Mindset

Microarchitecture Impact

  • CPU idle

waiting for memory

  • Low IPC <1
  • Present OOO CPUs

I-cache miss D-cache miss critical path non-critical

Branch Prediction Data Value Prediction Prefetching Multithreading

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 6

Main Points

  • Need changed

mindset

to seek MLP

  • ptimizations
  • IPC a bad metric

for ILP Processors with low IPC can have high MLP

– deep instruction windows – highly non-blocking caches

  • MLP enablers

– implicit multithreading – hardware skiplists – large microarchitecture data structures

e.g. main memory compression

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 7

Pointer Chasing is critical

Large linked data structures

e.g. 3D graphics “world” ∝ memory in size >> cache

Brute force doesn’t help

– increasing window – increasing frequency

Solution HW Skip Lists

# skips ∝ memory

⇒ store in main memory

  • w. compression

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 8

MLP processor sketch

Main Memory Compression Compressed Data Hints, e.g. Skip Pts Instruction Stream Retirement Window Forgetful Window Potential Threads Active Threads Same Task Calls Loops Convergences Different Tasks Execution Units

Inner Loop EUs

Register File Cache Data Cache Skip Pointer create Skip Pointer prefetch

slide-3
SLIDE 3

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 9

Conclusion

Inevitable forward march of ILP will continue

– the MT generations

  • explicit
  • implicit

– the MLP generation

No shortage of ideas to make uniprocessors faster.

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 10

Backup Slides

  • Slides not included in the short (8 minute)

presentation,

  • answer likely questions
  • useful if slides photocopied

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 11

Processors with low IPC can have high MLP

Examples

  • Simplifying assumptions

– AXPY:

  • simplest case unoptimized, other cases the usual:

SW pipelining, accumulators, etc.

– Linked Data Structures

  • randomized list / tree layout
  • independent visit functions

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 12

AXPY

a*xi+yi

EUs Misses Uarch TimeN Sketch 1 64 InO N*1000 16 64 OOO N/64*1000 1 64 OOO N/64*1000 + 64*4

slide-4
SLIDE 4

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 13

Tree Traversal

V(p) { p→l && V(p→l); p→r && V(p→r); VV(p); }

EUs Misses Uarch TimeN Sketch 1 64 InO N*(1000+V) 16 64 OOO N*1000 1 64 OOO N*1000 + 64*4 1 64 OOO + skiplists N/m*1000 + 64*4

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 14

Tree Algorithms

  • Combinatoric explosion N-ary trees
  • Traversal order skiplists ≈ threading

– works if similar traversals repeated

  • Searches

– key equal searches → hash tables – proximity and range searches → traversals

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 15

Caveats - Do I believe this s**t?

  • Is there a memory

wall? YES

  • Do I care about branch

prediction? YES, but in the MLP world it is a secondary effect.

  • Do I care about IPC?

In inner loops. Not when latency cache missing.

  • Aren’t circuits getting

slower? NO.Wires are getting slower. Gates are still getting faster. Tricks make ALUs faster still.

  • Isn’t MLP a form of

ILP? YES. IPC is not a metric of ILP.

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 16

Caveats - Do I believe this s**t?

Cache Hierarchies

  • Multilevel cache hierarchies

– latency = sqrt(size) – reportedly of diminishing effectiveness – large working set applications? (not SPEC95)

slide-5
SLIDE 5

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 17

Alternate MLP architecture

  • SW skip lists

– library data structures(STL) – or, compiler…

  • Prefetch instructions

– Eager prefetch of linked data structures ⇒ less speedup than hardware MLP if tree nodes big – Traversal order threading + skip lists ⇒ same speedup as hardware MLP

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 18

Unused Slides

  • The following slides are not used in the

current MLP presentation, and contain new

  • information. E.g. they are skipped just

because of time.

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 19

Data structures + MLP

  • Arrays: trivial; easy MLP
  • Linear linked lists: skip lists work
  • N-ary Trees

– combinatoric explosion – traversals easier to parallelize than searches

  • esp. proximity searches
  • Hash tables

– already minimally cache missing – conflicts: hash probing >MLP chaining

least most sophisticated

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 20

Large Instruction Windows

Brute Force

– large windows spill to RAM – cache frequently used parts

Expandable, Split, Instruction Windows

– Sohi / Multiscalar

Forget / Recompute

2+ windows

  • retirement (=OOO)
  • non-blocking

Oldest instruction blocked ⇒ advance window marking result unknown. Mispredict ⇒ set non-blocking window = retirement window.

slide-6
SLIDE 6

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 21

Datascalar

  • Limited MLP help
  • E.g. 4-way datascalar (planar)

⇒ Msize/4, Mlatency/2 - faster ⇒ Interconnect delay = Mlatency/2 Linked list, randomized: ¼ Mlatency/2 + ¾ Mlatency/2 = Mlatency/2

  • N-way:

          → − +

≥

2 1 1 1 1

latency ct interconne

M L N N N N

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 22

Processors with low IPC can have high MLP

Examples

Here:

  • AXPY
  • Linear

Linked List Backup:

  • Tree Traversal

** 3D graphics large linked structures

≈ memory >> cache randomized

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 23

Processors with low IPC can have high MLP

AXPY

A * Xi + Yi

Wide Superscalar CPU 4 misses deep Narrow CPU 6 misses deep

Many cache misses ⇒ Narrow - Wide = small startup cost Why not spend HW on cache misses, not superscalar EUs?

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 24

EUs Misses Uarch TimeN Sketch 1 * InO N*(1000+V) 16 64 OOO N*1000 1 64 OOO N*1000 + 64*4 1 64 OOO + skiplistsm N/m*1000 + 64*4

Processors with low IPC can have high MLP

Linear Linked List

for(p=hd;p=p→nxt;) visit(p)

slide-7
SLIDE 7

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 25

MLP enablers

Skip Lists

  • Convert list to tree

⇒ eager prefetch now helps ⇒ MLP=m

  • Can be done by

– software (library, compiler?) – hardware

  • store skip pointers in (compressed) main memory

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 26

MLP enablers

Large µarch Data Structures

Problem Solution HW skip lists

≥ 1 pointer per node ∝ memory size >> any “cache”

Store in main memory compress reserve Large Instruction Window Spill to reserved RAM Cache physical registers Forget / recompute

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 27

MLP processor

super non-blocking

  • Narrow EUs

– wide inner loops?

  • Deep instruction

window

– cached / spilled to RAM – forget / recompute

  • Deeply non-blocking

cache

  • HW skiplists

– stored in main memory with compression

  • Smart Sequential

Algorithms

– Belady lookahead in instruction window

  • Dynamic MT

More Backups

Non-obsolete slides added to end because of paper’s updateable qualities

slide-8
SLIDE 8

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 29

Caveats - Do I believe this s**t?

Speed, Voltage, Power

  • Brainiacs vs. Speed

Demons

– déjà vu all over again

  • Power

– Speed ∝ Voltage Power ∝ Voltage2 – Superscalar parallelism to save power may be a good thing ≠? performance

  • Speed is fungible with

superscalarness

– 8GHz 1-way = 1 GHz 8-way – if circuit speeds trade

  • ff

– sequential always easier than parallel

98/10/09 17:01 Glew: MLP yes! IPC no! (ASPLOS98) 30

Caveats - Do I believe this s**t?

Workloads

  • Unabashedly single-user

– servers can use MT

  • Q: next killer app.?

– Is there one? – Does it fit in cache? Glew: no. – Probably something like 3D graphics virtual worlds.