MASK: Redesigning the GPU Memory Hierarchy to Support - - PowerPoint PPT Presentation

MASK: Redesigning the GPU Memory Hierarchy to Support Multi-Application Concurrency

Rachata Ausavarungnirun

Vance Miller Joshua Landgraf Saugata Ghose Jayneel Gandhi Adwait Jog Christopher J. Rossbach Onur Mutlu

GPU 2 (Virginia EF) Tuesday 2PM-3PM

GPU Core

En Enabling GPU PU Sharing with Address Translation

2

GPU Core GPU Core GPU Core Page Table Walkers Page Table (in main memory)

App 1 App 2 Virtual Address

GPU Core

En Enabling GPU PU Sharing with Address Translation

3

GPU Core GPU Core GPU Core Page Table Walkers Page Table (in main memory)

High latency page walks

App 1 App 2 Virtual Address

GPU Core

Private TLB

St State-of

• f-the

the-Ar Art T Translation

• n S

Suppor

• rt i

in G GPUs Us

4

GPU Core GPU Core GPU Core Shared TLB

Private TLB

Page Table Walkers Page Table (in main memory)

Private TLB Private TLB

High latency page walks

Private Shared App 1 App 2 Virtual Address

Three Source ces of Ineffici ciency cy in Translation

5

High TLB contention n Inefficient caching Bypass Address-translation is latency sensitive

MASK: A Translation-aware Memory Hierarchy

Three Source ces of Ineffici ciency cy in Translation

6

High TLB contention n

Three Source ces of Ineffici ciency cy in Translation

7

High TLB contention n Inefficient caching Bypass

Three Source ces of Ineffici ciency cy in Translation

8

High TLB contention n Inefficient caching Bypass Address translation is latency-sensitive

Ou Our S Solution

9

MASK: A Translation-aware Memory Hierarchy

Th Three Components of MASK

10

Th Three Components of MASK

11

TLB-fill Tokens Reduces TLB contention

Shared TLB

Th Three Components of MASK

12

TLB-fill Tokens Reduces TLB contention Translation-aware L2 Bypass Improves L2 cache utilization

Shared TLB L2 Data Cache

Translation Data

Th Three Components of MASK

13

TLB-fill Tokens Reduces TLB contention Translation-aware L2 Bypass Improves L2 cache utilization Address-space-aware Memory Scheduler Lowers address translation latency

Shared TLB L2 Data Cache Main Memory

Translation Data Translation Data

Th Three Components of MASK

14

TLB-fill Tokens Reduces TLB contention Translation-aware L2 Bypass Improves L2 cache utilization Address-space-aware Memory Scheduler Lowers address translation latency

MASK improves performance by 57.8%

Shared TLB L2 Data Cache Main Memory

Translation Data Translation Data

MASK: Redesigning the GPU Memory Hierarchy to Support Multi-Application Concurrency

Rachata Ausavarungnirun

Vance Miller Joshua Landgraf Saugata Ghose Jayneel Gandhi Adwait Jog Christopher J. Rossbach Onur Mutlu

GPU 2 (Virginia EF) Tuesday 2PM-3PM