Rethinking the Memory Hierarchy for Modern Languages Po-An Tsai , - PowerPoint PPT Presentation

Rethinking the Memory Hierarchy for Modern Languages Po-An Tsai , Yee Ling Gan, and Daniel Sanchez

Memory systems expose an inexpressive interface 2

Memory systems expose an inexpressive interface Program Flat address Arbitrary loads space and stores Memory hierarchy 2

Memory systems expose an inexpressive interface 0x0000 Obj. A 0xFFFF Program Obj. B Flat address Arbitrary loads space and stores Memory hierarchy Programmers think of objects and pointers among objects 2

Modern languages expose an object-based memory model Program 0x0000 Obj. A Object-based Object accesses 0xFFFF model Obj. B Runtime/Compiler Flat address Loads and stores space to objects Memory hierarchy  Strictly hiding the flat address space provides many benefits:  Memory safety prevents memory corruption bugs  Automatic memory management (garbage collection) simplifies programming 3

The inexpressive flat address space is inefficient Program Object-based model Runtime/Compiler Flat address space Memory hierarchy 4

The inexpressive flat address space is inefficient Program Object-based model Runtime/Compiler Flat address space Memory hierarchy Semantic gap between programs and the memory hierarchy 4

The inexpressive flat address space is inefficient Program Object-based Main Mem. model 0x0000 Obj. A Obj Runtime/Compiler L1 L2 B Obj. C Flat address Core $ $ space Memory hierarchy 0xFFFF Semantic gap between programs and the memory hierarchy 4

The inexpressive flat address space is inefficient Program Object-based Main Mem. model 0x0000 Obj. A Obj Runtime/Compiler L1 L2 B Obj. C Flat address Core $ $ space Memory hierarchy 0xFFFF Semantic gap between programs and the memory hierarchy Mismatch between objects and cache lines 4

The inexpressive flat address space is inefficient Program Object-based Main Mem. model 0x0000 Obj. A Obj Runtime/Compiler L1 L2 B Obj. C Flat address Core $ $ space Memory hierarchy 0xFFFF Semantic gap between programs and the memory hierarchy Mismatch between Costly associative objects and cache lines lookups 4

Hotpads: An object-based memory hierarchy 5

Hotpads: An object-based memory hierarchy  A memory hierarchy designed from the ground up for object-based programs  Provides first-class support for objects and pointers in the ISA  Hides the memory layout from software and takes control over it 5

Hotpads: An object-based memory hierarchy  A memory hierarchy designed from the ground up for object-based programs  Provides first-class support for objects and pointers in the ISA  Hides the memory layout from software and takes control over it Program Object-based ISA Object operations Hotpads Manages objects 5

Hotpads: An object-based memory hierarchy  A memory hierarchy designed from the ground up for object-based programs  Provides first-class support for objects and pointers in the ISA  Hides the memory layout from software and takes control over it Program Object-based ISA Object operations Hotpads Manages objects L2 L3 L1 Core pad pad pad 5

Hotpads: An object-based memory hierarchy  A memory hierarchy designed from the ground up for object-based programs  Provides first-class support for objects and pointers in the ISA  Hides the memory layout from software and takes control over it Program Hotpads manages objects instead of cache lines Object-based ISA Object operations Hotpads Manages objects L2 L3 L1 Core pad pad pad 5

Hotpads: An object-based memory hierarchy  A memory hierarchy designed from the ground up for object-based programs  Provides first-class support for objects and pointers in the ISA  Hides the memory layout from software and takes control over it Program Hotpads manages objects instead of cache lines Object-based ISA Object operations Hotpads Hotpads rewrites pointers to Manages objects reduce associative lookups L2 L3 L1 Core pad pad pad 5

Hotpads: An object-based memory hierarchy  A memory hierarchy designed from the ground up for object-based programs  Provides first-class support for objects and pointers in the ISA  Hides the memory layout from software and takes control over it Program Hotpads manages objects instead of cache lines Object-based ISA Object operations Hotpads Hotpads rewrites pointers to Manages objects reduce associative lookups Hotpads provides architectural L2 L3 L1 Core support for in-hierarchy object pad pad pad allocation and recycling 5

Prior architectural support for object-based programs 6

Prior architectural support for object-based programs  Object-oriented/typed systems [iAPX432, Dally ISCA’85, CHERI ISCA’14, Kim et al. ASPLOS’17] focus on core microarchitecture design Core  Accelerate virtual calls, object references Type check unit L2 and dynamic type checks L1 $ vFunction unit $ 6

Prior architectural support for object-based programs  Object-oriented/typed systems [iAPX432, Dally ISCA’85, CHERI ISCA’14, Kim et al. ASPLOS’17] focus on core microarchitecture design Core  Accelerate virtual calls, object references Type check unit L2 and dynamic type checks L1 $ vFunction unit $  Hardware accelerators for GC [The Lisp Machine, GC unit Joao et al. ISCA’09, Maas et al. ISCA’18] Core L2 L1 $ $ 6

Prior architectural support for object-based programs  Object-oriented/typed systems [iAPX432, Dally ISCA’85, CHERI ISCA’14, Kim et al. ASPLOS’17] focus on core microarchitecture design Core  Accelerate virtual calls, object references Type check unit L2 and dynamic type checks L1 $ vFunction unit $  Hardware accelerators for GC [The Lisp Machine, GC unit Joao et al. ISCA’09, Maas et al. ISCA’18] Core L2 L1 $ $ Prior work uses standard cache hierarchies 6

Prior architectural support for object-based programs  Object-oriented/typed systems [iAPX432, Dally ISCA’85, CHERI ISCA’14, Kim et al. ASPLOS’17] focus on core microarchitecture design Core  Accelerate virtual calls, object references Type check unit L2 and dynamic type checks L1 $ vFunction unit $  Hardware accelerators for GC [The Lisp Machine, GC unit Joao et al. ISCA’09, Maas et al. ISCA’18] Core L2 L1 $ $ Prior work uses standard cache hierarchies We focus on redesigning the memory hierarchy 6

Hotpads overview 7

Hotpads overview L2 L3 L1 Core pad pad pad 7

Hotpads overview  Data array  Managed as a circular buffer using simple L2 L3 L1 Core pad pad bump pointer allocation pad  Stores variable-sized objects compactly Data Array Objects Free space 7

Hotpads overview  Data array  Managed as a circular buffer using simple L2 L3 L1 Core pad pad bump pointer allocation pad  Stores variable-sized objects compactly Data Array Obj. A Objects Free space 7

Hotpads overview  Data array  Managed as a circular buffer using simple L2 L3 L1 Core pad pad bump pointer allocation pad  Stores variable-sized objects compactly Data Array Obj. A Objects Obj. B Free space 7

Hotpads overview  Data array  Managed as a circular buffer using simple L2 L3 L1 Core pad pad bump pointer allocation pad  Stores variable-sized objects compactly  C-Tags Data Array  Decoupled tag store used only for a fraction Obj. A Objects of accesses Obj. B Free space C-Tags 7

Hotpads overview  Data array  Managed as a circular buffer using simple L2 L3 L1 Core pad pad bump pointer allocation pad  Stores variable-sized objects compactly  C-Tags Data Array  Decoupled tag store used only for a fraction (word/object) Obj. A Metadata Objects of accesses Obj. B Free space  Metadata C-Tags  Pointer? valid? dirty? recently-used? 7

Hotpads example 8

Hotpads example class Node { int value; Node next; } 8

Hotpads example class Node { RegFile L1 Pad L2 Pad Main Mem int value; r0 Node next; Objects r1 A } Free r2 B space r3 Initial state. 8

Hotpads moves object implicitly class Node { RegFile L1 Pad L2 Pad Main Mem int value; r0 Node next; r1 A } A r2 B r3 Core issues access to A. Program code: Hotpads instructions: A is copied into L1 pad. int v = A.value; ld r0, (r1).value 9

Hotpads moves object implicitly class Node { RegFile L1 Pad L2 Pad Main Mem int value; r0 Node next; r1 A } A r2 B r3 Core issues access to A. Program code: Hotpads instructions: A is copied into L1 pad. int v = A.value; ld r0, (r1).value  All loads/stores follow a single addressing mode: Base+offset 9

Hotpads moves object implicitly class Node { RegFile L1 Pad L2 Pad Main Mem int value; r0 Node next; r1 A } A r2 B r3 Core issues access to A. Program code: Hotpads instructions: A is copied into L1 pad. int v = A.value; ld r0, (r1).value  All loads/stores follow a single addressing mode: Base+offset  Bump pointer allocation stores A compactly after other objects 9

Hotpads rewrites pointers to avoid associative lookups class Node { RegFile L1 Pad L2 Pad Main Mem int value; r0 Node next; r1 A } A r2 B r3 Core issues access to A. Program code: Hotpads instructions: A is copied into L1 pad. int v = A.value; ld r0, (r1).value r1 is rewritten to A’s L1 pad address. 10