Criticality Aware Tiered Cache Hierarchy (CATCH) Anant Nori*, Jayesh - - PowerPoint PPT Presentation

Criticality Aware Tiered Cache Hierarchy (CATCH)

Anant Nori*, Jayesh Gaur*, Siddharth Rai#, Sreenivas Subramoney*, Hong Wang* * Microarchitecture Research Lab, Intel

# Indian Institute of Technology Kanpur, India

Popular Three Level Cache Hierarchy

2

Popular Three Level Cache Hierarchy

2

• Cache capacity ↔ Access latency
• Target low average latency

L1 (Pvt.) 5 cyc L2 (Pvt) 15 cyc LLC (Shared) 40 cyc

Skylake-like Server 1.375MB/core 5.5MB (4 core) Exclusive

Data Code 32 KB

1MB

Popular Three Level Cache Hierarchy

2

• Cache capacity ↔ Access latency
• Target low average latency
• Large distributed LLC, high latency
• Lower L2 latency important

L1 (Pvt.) 5 cyc L2 (Pvt) 15 cyc LLC (Shared) 40 cyc

Skylake-like Server 1.375MB/core 5.5MB (4 core) Exclusive

Data Code 32 KB

1MB

Popular Three Level Cache Hierarchy

2

• Cache capacity ↔ Access latency
• Target low average latency
• Large distributed LLC, high latency
• Lower L2 latency important

L1 (Pvt.) 5 cyc L2 (Pvt) 15 cyc LLC (Shared) 40 cyc

Skylake-like Server 1.375MB/core 5.5MB (4 core) Exclusive

Data Code 32 KB

1MB

• 6.5%
• 7.0%
• 4.2%
• 14.0%
• 6.9%
• 7.8%
• 4.0%
• 6.8%
• 3.3%
• 9.5%
• 2.7%
• 5.1%
• 15%
• 12%
• 9%
• 6%
• 3%

0% client FSPEC HPC ISPEC server GeoMean

• perf. impact

NoL2 + 6.5MB LLC NoL2 + 9.5MB LLC

Popular Three Level Cache Hierarchy

2

• Cache capacity ↔ Access latency
• Target low average latency
• Large distributed LLC, high latency
• Lower L2 latency important
• Trend to larger L2 sizes

L1 (Pvt.) 5 cyc L2 (Pvt) 15 cyc LLC (Shared) 40 cyc

Broadwell-like Server

2MB/core 8MB (4 core) Inclusive 256 KB

Data Code 32 KB

Skylake-like Server 1.375MB/core 5.5MB (4 core) Exclusive

Data Code 32 KB

1MB

• 6.5%
• 7.0%
• 4.2%
• 14.0%
• 6.9%
• 7.8%
• 4.0%
• 6.8%
• 3.3%
• 9.5%
• 2.7%
• 5.1%
• 15%
• 12%
• 9%
• 6%
• 3%

0% client FSPEC HPC ISPEC server GeoMean

• perf. impact

NoL2 + 6.5MB LLC NoL2 + 9.5MB LLC

Popular Three Level Cache Hierarchy

2

• Cache capacity ↔ Access latency
• Target low average latency
• Large distributed LLC, high latency
• Lower L2 latency important
• Trend to larger L2 sizes

Is a large L2 the most efficient design choice?

L1 (Pvt.) 5 cyc L2 (Pvt) 15 cyc LLC (Shared) 40 cyc

Broadwell-like Server

2MB/core 8MB (4 core) Inclusive 256 KB

Data Code 32 KB

Skylake-like Server 1.375MB/core 5.5MB (4 core) Exclusive

Data Code 32 KB

1MB

• 6.5%
• 7.0%
• 4.2%
• 14.0%
• 6.9%
• 7.8%
• 4.0%
• 6.8%
• 3.3%
• 9.5%
• 2.7%
• 5.1%
• 15%
• 12%
• 9%
• 6%
• 3%

0% client FSPEC HPC ISPEC server GeoMean

• perf. impact

NoL2 + 6.5MB LLC NoL2 + 9.5MB LLC

L 2 L 2

L2 L2 LLC LLC Inclusive LLC Exclusive LLC

Large L2 caches

3

• Inclusive LLC → Exclusive LLC

L 2 L 2

L2 L2 LLC LLC Inclusive LLC Exclusive LLC

Large L2 caches

3

• Inclusive LLC → Exclusive LLC
• Lower effective on-die cache per core

L 2 L 2

L2 L2 LLC LLC Inclusive LLC Exclusive LLC

Large L2 caches

3

• Inclusive LLC → Exclusive LLC
• Lower effective on-die cache per core
• Large LLC better for multiple threads

with disparate cache footprints

         

L 2 L 2

L2 L2 LLC LLC Inclusive LLC Exclusive LLC

Large L2 caches

3

• Inclusive LLC → Exclusive LLC
• Lower effective on-die cache per core
• Large LLC better for multiple threads

with disparate cache footprints

• Area for Snoop-filter/Coherence-directory

L 2 L 2

L2 L2 LLC LLC Inclusive LLC Exclusive LLC

Large L2 caches

3

• Inclusive LLC → Exclusive LLC
• Lower effective on-die cache per core
• Large LLC better for multiple threads

with disparate cache footprints

• Area for Snoop-filter/Coherence-directory

Despite area and power overheads, average latency reduction (performance) drives large L2

Program execution expressed in a Data Dependency Graph (Fields et. al.)

4

Loads and Program Criticality

D E C D E C D E C D E C D E C D E C D E C

LLC miss L2 Hit L2 Hit LLC Hit

1 2 3 4 5 6 7

Load Load Load Load

2

200

10

10 2

2 10

30

200 30

2 2 10

Program execution expressed in a Data Dependency Graph (Fields et. al.)

• Execution time governed by “Critical Path”

4

Loads and Program Criticality

E C D C D E C D C D C D E C D

LLC miss L2 Hit L2 Hit LLC Hit

1 2 3 4 5 6 7

Load Load Load Load

2 30 10 2 10

30 D E E E E C 200 10 2 2 2

200

10

Program execution expressed in a Data Dependency Graph (Fields et. al.)

• Execution time governed by “Critical Path”

4

Loads and Program Criticality

E C D C D E C D C D C D E C D

LLC miss L2 Hit L2 Hit LLC Hit

1 2 3 4 5 6 7

Load Load Load Load

2 30 10 2 10

30 D E E E E C 200 10 2 2 2

200

10

Program execution expressed in a Data Dependency Graph (Fields et. al.)

• Execution time governed by “Critical Path”

4

Loads and Program Criticality

E C D C D E C D C D C D E C D

LLC miss L2 Hit L2 Hit LLC Hit

1 2 3 4 5 6 7

Load Load Load Load

2 30 10 2 10

30 D E E E E C 200 10 2 2 2

200

10

CRITICAL L2 HIT LOAD

Program execution expressed in a Data Dependency Graph (Fields et. al.)

• Execution time governed by “Critical Path”

4

Loads and Program Criticality

E C D C D E C D C D C D E C D

LLC miss L2 Hit L2 Hit LLC Hit

1 2 3 4 5 6 7

Load Load Load Load

2 30 10 2 10

30 D E E E E C 200 10 2 2 2

200

10

CRITICAL L2 HIT LOAD ~9% performance loss if L2 HIT → LLC HIT

LLC hit

30 30

Program execution expressed in a Data Dependency Graph (Fields et. al.)

• Execution time governed by “Critical Path”

4

Loads and Program Criticality

E C D C D E C D C D C D E C D

LLC miss L2 Hit L2 Hit LLC Hit

1 2 3 4 5 6 7

Load Load Load Load

2 30 10 2 10

30 D E E E E C 200 10 2 2 2

200

10

CRITICAL L2 HIT LOAD ~9% performance loss if L2 HIT → LLC HIT NON-CRITICAL L2 HIT LOAD

Program execution expressed in a Data Dependency Graph (Fields et. al.)

• Execution time governed by “Critical Path”

4

Loads and Program Criticality

E C D C D E C D C D C D E C D

LLC miss L2 Hit L2 Hit LLC Hit

1 2 3 4 5 6 7

Load Load Load Load

2 30 10 2 10

30 D E E E E C 200 10 2 2 2

200

10

CRITICAL L2 HIT LOAD ~9% performance loss if L2 HIT → LLC HIT NON-CRITICAL L2 HIT LOAD No performance impact if L2 HIT → LLC HIT

LLC hit

30

30

Program execution expressed in a Data Dependency Graph (Fields et. al.)

• Execution time governed by “Critical Path”

4

Loads and Program Criticality

E C D C D E C D C D C D E C D

LLC miss L2 Hit L2 Hit LLC Hit

1 2 3 4 5 6 7

Load Load Load Load

2 30 10 2 10

30 D E E E E C 200 10 2 2 2

200

10

CRITICAL L2 HIT LOAD ~9% performance loss if L2 HIT → LLC HIT NON-CRITICAL L2 HIT LOAD No performance impact if L2 HIT → LLC HIT

Only critical load L2 hits matter to performance.

Cache Hierarchy and Program Criticality

5

Oracle study

• Track critical load PCs
• Increase latencies of targeted load PCs

• 16.1%
• 7.8%
• 4.9%
• 0.8%

49.1% 39.6% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

• 18%
• 16%
• 14%
• 12%
• 10%
• 8%
• 6%
• 4%
• 2%

0% ALL NonCritical ALL NonCritical L1 hits to L2 lat. L2 hits to LLC lat. % loads converted to higer latency

• perf. impact
• Perf. Impact – All loads
• Perf. Impact – NonCritical loads % loads converted

Cache Hierarchy and Program Criticality

5

• 16.1%
• 7.8%
• 4.9%
• 0.8%

49.1% 39.6% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

• 18%
• 16%
• 14%
• 12%
• 10%
• 8%
• 6%
• 4%
• 2%

0% ALL NonCritical ALL NonCritical L1 hits to L2 lat. L2 hits to LLC lat. % loads converted to higer latency

• perf. impact
• Perf. Impact – All loads
• Perf. Impact – NonCritical loads % loads converted

Cache Hierarchy and Program Criticality

5

L2 cache most amenable to criticality optimizations

Criticality Aware Tiered Cache Hierarchy (CATCH)

• A. Track critical load PCs

6

Criticality Aware Tiered Cache Hierarchy (CATCH)

• A. Track critical load PCs
• Served from non-L1 on-die caches

6

Criticality Aware Tiered Cache Hierarchy (CATCH)

• A. Track critical load PCs
• Served from non-L1 on-die caches
• B. Prefetch critical loads into L1
• Accelerate the critical path

6

Criticality Aware Tiered Cache Hierarchy (CATCH)

• A. Track critical load PCs
• Served from non-L1 on-die caches
• B. Prefetch critical loads into L1
• Accelerate the critical path

6

5.5% 5.8% 6.1% 6.6% 6.2% 14.1% 15.5% 17.0% 0% 4% 8% 12% 16% 20% 0% 2% 4% 6% 8% 10% 32 PC 128 PC 2048 PC All PC NoL2 + 2048 PC % L1 misses converted

• perf. impact

Oracle Performance Potential (*prefetchers disabled)

PerfImpact %loads Converted

Criticality Aware Tiered Cache Hierarchy (CATCH)

• A. Track critical load PCs
• Served from non-L1 on-die caches
• B. Prefetch critical loads into L1
• Accelerate the critical path

6

5.5% 5.8% 6.1% 6.6% 6.2% 14.1% 15.5% 17.0% 0% 4% 8% 12% 16% 20% 0% 2% 4% 6% 8% 10% 32 PC 128 PC 2048 PC All PC NoL2 + 2048 PC % L1 misses converted

• perf. impact

Oracle Performance Potential (*prefetchers disabled)

PerfImpact %loads Converted

L2 can become redundant in a criticality aware cache hierarchy

CATCH Configuration Options

7

Level1 (L1) Private Level2 (L2) Private Level3 (L3) Shared

5.5MB Exclusive

1MB

Data Code 32 KB

BASELINE CATCH Hardware

• A. Track critical

load PCs

• B. Prefetch

critical loads in L1

CATCH Configuration Options

7

Level1 (L1) Private Level2 (L2) Private Level3 (L3) Shared

5.5MB Exclusive

1MB

Data Code 32 KB

BASELINE CATCH Hardware

• A. Track critical

load PCs

• B. Prefetch

critical loads in L1

5.5MB Exclusive

1MB

Data Code 32 KB

Three-Level CATCH

Accelerates critical path

CATCH Configuration Options

7

Level1 (L1) Private Level2 (L2) Private Level3 (L3) Shared

5.5MB Exclusive

1MB

Data Code 32 KB

BASELINE CATCH Hardware

• A. Track critical

load PCs

• B. Prefetch

critical loads in L1

5.5MB Exclusive

1MB

Data Code 32 KB

Three-Level CATCH

Accelerates critical path

5.5MB Exclusive

Data Code 32 KB

Two-Level CATCH (NoL2)

Accelerates critical path + Area saving

5.5MB Inclusive

CATCH Configuration Options

7

Level1 (L1) Private Level2 (L2) Private Level3 (L3) Shared

5.5MB Exclusive

1MB

Data Code 32 KB

BASELINE CATCH Hardware

• A. Track critical

load PCs

• B. Prefetch

critical loads in L1

5.5MB Exclusive

1MB

Data Code 32 KB

Three-Level CATCH

Accelerates critical path

5.5MB Exclusive

Data Code 32 KB

Two-Level CATCH (NoL2)

Accelerates critical path + Area saving

5.5MB Exclusive

1MB

Data Code 32 KB

Two-Level CATCH (NoL2, IsoArea)

Accelerates critical path

5.5MB Inclusive

CATCH Configuration Options

7

Level1 (L1) Private Level2 (L2) Private Level3 (L3) Shared

5.5MB Exclusive

1MB

Data Code 32 KB

BASELINE CATCH Hardware

• A. Track critical

load PCs

• B. Prefetch

critical loads in L1

5.5MB Exclusive

1MB

Data Code 32 KB

Three-Level CATCH

Accelerates critical path

5.5MB Exclusive

Data Code 32 KB

Two-Level CATCH (NoL2)

Accelerates critical path + Area saving

Data Code 32 KB

Two-Level CATCH (NoL2, IsoArea)

Accelerates critical path + Power saving

9.5MB Inclusive 5.5MB Inclusive

A) Criticality Detection in Hardware

8

Instructions being executed (ROB) Instructions being allocated

D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C

A) Criticality Detection in Hardware

Buffer execution DDG (Fields et. al.) on instruction retire

8

Instructions being executed (ROB) Instructions being allocated

D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C

History of Retired Instructions (2.5x ROB window)

D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C

A) Criticality Detection in Hardware

Buffer execution DDG (Fields et. al.) on instruction retire Enumerate critical path every 2x ROB instruction retires

8

Instructions being executed (ROB) Instructions being allocated

D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C

History of Retired Instructions (2.5x ROB window)

D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C

A) Criticality Detection in Hardware

Buffer execution DDG (Fields et. al.) on instruction retire Enumerate critical path every 2x ROB instruction retires

8

Instructions being executed (ROB) Instructions being allocated

D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C

History of Retired Instructions (2.5x ROB window)

D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C

32 entry Critical Load PC Table

A) Criticality Detection in Hardware

Buffer execution DDG (Fields et. al.) on instruction retire Enumerate critical path every 2x ROB instruction retires Optimizations: Area of DDG, Fast enumeration of critical path

8

Instructions being executed (ROB) Instructions being allocated

D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C

History of Retired Instructions (2.5x ROB window)

D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C

32 entry Critical Load PC Table

A) Criticality Detection in Hardware

Buffer execution DDG (Fields et. al.) on instruction retire Enumerate critical path every 2x ROB instruction retires Optimizations: Area of DDG, Fast enumeration of critical path

• Uses 3KB of storage. Details in the paper

8

Instructions being executed (ROB) Instructions being allocated

D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C

History of Retired Instructions (2.5x ROB window)

D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C D E C

32 entry Critical Load PC Table

B) Timeliness Aware, Criticality Triggered (TACT) Prefetchers

Critical load PC prefetchers optimized for inter-cache prefetching into Data L1

9

B) Timeliness Aware, Criticality Triggered (TACT) Prefetchers

Critical load PC prefetchers optimized for inter-cache prefetching into Data L1 Data Prefetchers

• Identify “Trigger” load PCs for “Target” critical load PCs

9

B) Timeliness Aware, Criticality Triggered (TACT) Prefetchers

Critical load PC prefetchers optimized for inter-cache prefetching into Data L1 Data Prefetchers

• Identify “Trigger” load PCs for “Target” critical load PCs

Code “Run-Ahead” Prefetcher

• Cover LLC latency instead of L2 (for when the L2 is removed)

9

10

TACT: Data Prefetchers

10

“Cross” Prefetcher:

TACT: Data Prefetchers

LD Trigger T o LD Target C o LD Trigger T o+1 … … LD Target C o+1

10

“Cross” Prefetcher:

• Trigger load PC address @

constant delta from Target/Critical PC

TACT: Data Prefetchers

LD Trigger T o LD Target C o LD Trigger T o+1 … … LD Target C o+1

• Addr. Trigger T to
• Addr. Target C = Δ

10

“Cross” Prefetcher:

• Trigger load PC address @

constant delta from Target/Critical PC

“Self” Deep Prefetcher:

TACT: Data Prefetchers

LD Trigger T o LD Target C o LD Trigger T o+1 … … LD Target C o+1

• Addr. Trigger T to
• Addr. Target C = Δ

a, a+δ, a+2δ, … , a+16δ, a+17δ, a+18δ

10

“Cross” Prefetcher:

• Trigger load PC address @

constant delta from Target/Critical PC

“Self” Deep Prefetcher:

• Upto deep prefetch distance 16

TACT: Data Prefetchers

LD Trigger T o LD Target C o LD Trigger T o+1 … … LD Target C o+1

• Addr. Trigger T to
• Addr. Target C = Δ

a, a+δ, a+2δ, … , a+16δ, a+17δ, a+18δ

10

“Cross” Prefetcher:

• Trigger load PC address @

constant delta from Target/Critical PC

“Self” Deep Prefetcher:

• Upto deep prefetch distance 16

TACT: Data Prefetchers

LD Target D i LD Target D i+1 LD Target D i+2 … … … … LD Feeder F i LD Feeder F i+1 LD Feeder F i+2 LD Feeder F i+3 … … … LD Target D i+3

“Feeder” Prefetcher: Address of Target/Critical load = M * Data of Feeder load + C

LD Trigger T o LD Target C o LD Trigger T o+1 … … LD Target C o+1

• Addr. Trigger T to
• Addr. Target C = Δ

a, a+δ, a+2δ, … , a+16δ, a+17δ, a+18δ

10

“Cross” Prefetcher:

• Trigger load PC address @

constant delta from Target/Critical PC

“Self” Deep Prefetcher:

• Upto deep prefetch distance 16

TACT: Data Prefetchers

LD Target D i LD Target D i+1 LD Target D i+2 … … … … SELF “Deep” Address Prefetch of Feeder F LD Feeder F i LD Feeder F i+1 LD Feeder F i+2 LD Feeder F i+3 … … … LD Target D i+3

“Feeder” Prefetcher: Address of Target/Critical load = M * Data of Feeder load + C

LD Trigger T o LD Target C o LD Trigger T o+1 … … LD Target C o+1

• Addr. Trigger T to
• Addr. Target C = Δ

a, a+δ, a+2δ, … , a+16δ, a+17δ, a+18δ

10

“Cross” Prefetcher:

• Trigger load PC address @

constant delta from Target/Critical PC

“Self” Deep Prefetcher:

• Upto deep prefetch distance 16

TACT: Data Prefetchers

LD Target D i LD Target D i+1 LD Target D i+2 … … … … SELF “Deep” Address Prefetch of Feeder F Feeder Prefetch Data to Prefetch Address of Target D LD Feeder F i LD Feeder F i+1 LD Feeder F i+2 LD Feeder F i+3 … … … LD Target D i+3

“Feeder” Prefetcher: Address of Target/Critical load = M * Data of Feeder load + C

LD Trigger T o LD Target C o LD Trigger T o+1 … … LD Target C o+1

• Addr. Trigger T to
• Addr. Target C = Δ

a, a+δ, a+2δ, … , a+16δ, a+17δ, a+18δ

10

“Cross” Prefetcher:

• Trigger load PC address @

constant delta from Target/Critical PC

“Self” Deep Prefetcher:

• Upto deep prefetch distance 16

TACT: Data Prefetchers

LD Target D i LD Target D i+1 LD Target D i+2 … … … … SELF “Deep” Address Prefetch of Feeder F Feeder Prefetch Data to Prefetch Address of Target D LD Feeder F i LD Feeder F i+1 LD Feeder F i+2 LD Feeder F i+3 … … … LD Target D i+3

“Feeder” Prefetcher: Address of Target/Critical load = M * Data of Feeder load + C

LD Trigger T o LD Target C o LD Trigger T o+1 … … LD Target C o+1

• Addr. Trigger T to
• Addr. Target C = Δ

a, a+δ, a+2δ, … , a+16δ, a+17δ, a+18δ

Implementation details in the paper

TACT: Code “Run-Ahead” Prefetcher

11

Next Instruction Pointer (NIP) Logic … (Branch Prediction etc) Branch Mispredict NextInstructionPointer Code Load Next Line Code Prefetch +

CodeNextPrefetch InstructionPointer Code Prefetch Front End Stall

TACT: Code “Run-Ahead” Prefetcher

On front-end stall:

• Use NIP logic (Branch Prediction, BTB)
• Speculatively prefetch code lines

11

Next Instruction Pointer (NIP) Logic … (Branch Prediction etc) Branch Mispredict NextInstructionPointer Code Load Next Line Code Prefetch +

Evaluation : Configuration

• 4 x86 cores @ 3.2GHz, 4-wide, 224 ROB entries
• 32 KB, 8-way Data and Code L1
• PC-based stride prefetcher, multi-stream prefetchers
• Dual channel DDR4-2400 main memory

12

Evaluation : Configuration

• 4 x86 cores @ 3.2GHz, 4-wide, 224 ROB entries
• 32 KB, 8-way Data and Code L1
• PC-based stride prefetcher, multi-stream prefetchers
• Dual channel DDR4-2400 main memory
• Two baseline L2/LLC configurations
• Large L2 (1MB), Exclusive LLC (5.5MB, 1.375 MB/core)
• Small L2 (256KB), Inclusive LLC (8MB)

12

Evaluation : Configuration

• 4 x86 cores @ 3.2GHz, 4-wide, 224 ROB entries
• 32 KB, 8-way Data and Code L1
• PC-based stride prefetcher, multi-stream prefetchers
• Dual channel DDR4-2400 main memory
• Two baseline L2/LLC configurations
• Large L2 (1MB), Exclusive LLC (5.5MB, 1.375 MB/core)
• Small L2 (256KB), Inclusive LLC (8MB)
• 70 diverse ST workloads – SPEC-06, HPC, Server, Client
• 60 four-way multi-programmed workloads

12

13

Large 1MB L2, Exclusive 1.375 MB LLC per Core ST GeoMean Performance Impact

• 7.8%

4.5% 7.2% 8.4%

• 15.0%
• 10.0%
• 5.0%

0.0% 5.0% 10.0% 15.0% client FSPEC HPC ISPEC server GeoMean

• perf. impact w.r.t.

1MB L2, 5.5 MB LLC NoL2 NoL2 + CATCH NoL2 + 9.5MB LLC + CATCH CATCH

13

Large 1MB L2, Exclusive 1.375 MB LLC per Core ST GeoMean Performance Impact

• 7.8%

4.5% 7.2% 8.4%

• 15.0%
• 10.0%
• 5.0%

0.0% 5.0% 10.0% 15.0% client FSPEC HPC ISPEC server GeoMean

• perf. impact w.r.t.

1MB L2, 5.5 MB LLC NoL2 NoL2 + CATCH NoL2 + 9.5MB LLC + CATCH CATCH

CATCH accelerates the critical path and enables designs with better performance and area/power tradeoffs.

14

Large 1MB L2, Exclusive 1.375 MB LLC per Core ST Per-Workload Performance Impact

0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

• perf. ratio w.r.t.

1MB L2, 5.5MB LLC NoL2 NoL2 + 9.5MB LLC + CATCH CATCH

povray namd

TACT prefetchers recover loss from no L2 in majority of workloads

14

Large 1MB L2, Exclusive 1.375 MB LLC per Core ST Per-Workload Performance Impact

0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

• perf. ratio w.r.t.

1MB L2, 5.5MB LLC NoL2 NoL2 + 9.5MB LLC + CATCH CATCH

povray namd

TACT prefetchers recover loss from no L2 in majority of workloads Research on optimizations to improve remaining outliers

Power Analysis:

Iso-Area NoL2+9.5MB LLC + CATCH

15

Power Analysis:

Iso-Area NoL2+9.5MB LLC + CATCH

15

• Removal of L2
• Reduced cache activity
• Increased interconnect traffic

Power Analysis:

Iso-Area NoL2+9.5MB LLC + CATCH

15

• Removal of L2
• Reduced cache activity
• Increased interconnect traffic
• Increased LLC (+L2 from all cores)
• Reduced DRAM traffic

Power Analysis:

Iso-Area NoL2+9.5MB LLC + CATCH

15

• Removal of L2
• Reduced cache activity
• Increased interconnect traffic
• Increased LLC (+L2 from all cores)
• Reduced DRAM traffic

19.01% 14.36% 5.88% 10.15% 10.62% 10.87% 0% 2% 4% 6% 8% 10% 12% 14% 16% 18% 20% energy savings w.r.t. 1MB L2 , 5.5 MB LLC

Overall impact

• ~11% energy savings
• With 7.2% performance gains

Power Analysis:

Iso-Area NoL2+9.5MB LLC + CATCH

15

• Removal of L2
• Reduced cache activity
• Increased interconnect traffic
• Increased LLC (+L2 from all cores)
• Reduced DRAM traffic

19.01% 14.36% 5.88% 10.15% 10.62% 10.87% 0% 2% 4% 6% 8% 10% 12% 14% 16% 18% 20% energy savings w.r.t. 1MB L2 , 5.5 MB LLC

Overall impact

• ~11% energy savings
• With 7.2% performance gains

For large, power hungry (mesh) interconnects, power impact too high

Increased interconnect traffic

Power Analysis:

Iso-Area NoL2+9.5MB LLC + CATCH

15

• Removal of L2
• Reduced cache activity
• Increased interconnect traffic
• Increased LLC (+L2 from all cores)
• Reduced DRAM traffic

19.01% 14.36% 5.88% 10.15% 10.62% 10.87% 0% 2% 4% 6% 8% 10% 12% 14% 16% 18% 20% energy savings w.r.t. 1MB L2 , 5.5 MB LLC

Overall impact

• ~11% energy savings
• With 7.2% performance gains

For large, power hungry (mesh) interconnects, power impact too high ⇒ Small L2 to absorb L1 evictions preferable

Increased interconnect traffic

16

Summary

16

Summary

• Fundamental re-look at each level of a three level cache hierarchy

16

Summary

• Fundamental re-look at each level of a three level cache hierarchy
• L2 highly amenable to criticality optimizations
• Trend towards large L2 not the most efficient design choice

16

Summary

• Fundamental re-look at each level of a three level cache hierarchy
• L2 highly amenable to criticality optimizations
• Trend towards large L2 not the most efficient design choice
• CATCH introduces:
• Dynamic tracking of critical loads.
• Optimized inter-cache prefetch into L1

16

Summary

• Fundamental re-look at each level of a three level cache hierarchy
• L2 highly amenable to criticality optimizations
• Trend towards large L2 not the most efficient design choice
• CATCH introduces:
• Dynamic tracking of critical loads.
• Optimized inter-cache prefetch into L1
• CATCH enables radical new processor designs
• Efficient area/power/performance tradeoffs

Three-level CATCH 8.4% perf. gain Two-level CATCH 4.2% perf. gain + 30% area saving Two-level CATCH Iso-area 7.2% perf. gain + 11% energy saving

17