Modeling Cache Sharing for MPI Programs on Multi-core Machines Bin - - PowerPoint PPT Presentation

▶

Jun 02, 2023 141 likes •352 views

Modeling Cache Sharing for MPI Programs on Multi-core Machines Bin Bao, Chen Ding University of Rochester Nov 10, 2011 The 10th Workshop On

SLIDE 1

Nov 10, 2011 The 10th Workshop On Compiler-Driven Performance

Modeling Cache Sharing for MPI Programs on Multi-core Machines

Bin Bao, Chen Ding University of Rochester

Thursday, November 10, 11

SLIDE 2

Multi-core Popularity

✤ More and more cluster machines are using multi-core processors ✤ TOP500.org (June 2011): ✤ “Quad-core processors are used in 46.2 percent of the systems,

while already 42.4 percent of the systems use processors with six or more cores.”

Thursday, November 10, 11

SLIDE 3

Programming on Cluster

✤ MPI (Message-Passing Interface) is still

dominant

✤ Scalability issues, e.g. ✤ Load balance ✤ Communication overhead ✤ Multicore: resource contention

Thursday, November 10, 11

SLIDE 4

Performance Impact of Resource Sharing

✤ Experimental studies: Chai et al. [CCGRID’07], Saini et al. [SC’09],

etc.

cg ep ft is lu mg Speedup 1 2 3 1x2 2x2 4x2 ✤ Intel Nehalem E5520 (4 cores)

(Shared 8MB L3 cache)

✤ GCC 4.4.1, MPICH2 1.4.1

Thursday, November 10, 11

SLIDE 5

Goal: Modeling Cache Contention

✤ Tool: reuse distance (aka LRU stack distance), the number of distinct

data elements accessed between two consecutive references to the same element

a b c c d a

rd=3

✤ Reuse distance can be used to calculate cache miss rate ✤ Program A’s cache miss rate = P(A’s reuse distance ≥ cache size)

Thursday, November 10, 11

SLIDE 6

Locality (Reuse Distance) Scaling

✤ Strong scaling: fixed total problem size ✤ Fixed-distance reuses and scaled-distance reuses

a1,1 a1,2 ... a1,n a2,1 a2,2 ... a2,n ... an,1 an,2 ... an,n X b1 b2 ... bn = c1 c2 ... cn

Thursday, November 10, 11

SLIDE 7

Locality (Reuse Distance) Scaling

✤ Strong scaling: fixed total problem size ✤ Fixed-distance reuses and scaled-distance reuses

a1,1 a1,2 ... a1,n a2,1 a2,2 ... a2,n ... an,1 an,2 ... an,n X b1 b2 ... bn = c1 c2 ... cn

Thursday, November 10, 11

SLIDE 8

Locality (Reuse Distance) Scaling

✤ Strong scaling: fixed total problem size ✤ Fixed-distance reuses and scaled-distance reuses

a1,1 a1,2 ... a1,n a2,1 a2,2 ... a2,n ... an,1 an,2 ... an,n X b1 b2 ... bn = c1 c2 ... cn

Thursday, November 10, 11

SLIDE 9

Reuse Distance Reference Histogram

NAS-LU (B input)

Reference partitions Average reuse distance 200 400 600 800 1000 1KB 32KB 1MB 32MB 1GB 1x2 2x2 4x2

Thursday, November 10, 11

SLIDE 10

Reuse Distance Reference Histogram

NAS-LU (B input)

Reference partitions Average reuse distance 200 400 600 800 1000 1KB 32KB 1MB 32MB 1GB 1x2 2x2 4x2

Thursday, November 10, 11

SLIDE 11

More Examples

Reference partitions Average reuse distance 200 400 600 800 1000 1KB 32KB 1MB 32MB 1GB 1x2 2x2 4x2 Reference partitions Average reuse distance 200 400 600 800 1000 1KB 32KB 1MB 32MB 1GB 1x2 2x2 4x2 Average reuse distance 200 400 600 800 1000 1KB 32KB 1MB 32MB 1GB 1x2 2x2 4x2 Average reuse distance 200 400 600 800 1000 1KB 32KB 1MB 32MB 1GB 1x2 2x2 4x2

CG FT IS MG

Thursday, November 10, 11

SLIDE 12

Linear Regression Based Reuse Distance Prediction

✤ For partition i: ✤ The model captures scaled-distance reuses and fixed-distance reuses ✤ Each partition is independent

rdi = ai × (1/nproc) + bi

Thursday, November 10, 11

SLIDE 13

Cache Sharing

✤ General dilation model

[Xiang et al. PPoPP’11]

✤ Symmetric MPI programs:

uniform interleaving assumption a b c d e f a k l m n o p k

Task A: Task B: rd’ = 5 rd = 5

abcklmdenofpak

Task A&B: rd” = 11

a b c d e f a k m m m n o n

Program A: Program B: ft = 4 rd = 5

akbcmdmemfnona

Program A&B: rd’ = rd+ft = 9

Thursday, November 10, 11

SLIDE 14

Experiments

✤ Pin-based trace cache simulator (16-way LRU, 8MB) ✤ Performance counters (OProfile, LLC Misses)

Pin Tool MPI Task Pin Tool MPI Task Pin Tool MPI Task Pin Tool MPI Task Pin Lock $ Block $ Block ... $ Block ... Pin Lock $ Block $ Block ... $ Block ...

Thursday, November 10, 11

SLIDE 15