Efficient Architectural Support for Persistent Memory Vijay - PowerPoint PPT Presentation

Efficient Architectural Support for Persistent Memory Vijay Nagarajan

People Marcelo Cintra (Intel) Stratis Viglas (Google) Arpit Joshi(Edinburgh) 2

Emerging System Core Core Cache Cache DRAM NVM Secondary Storage Secondary Storage 3

Emerging System Core Core Cache Cache DRAM NVM Software Controlled Secondary Storage Secondary Storage 3

Emerging System Core Core Cache Cache Hardware Controlled DRAM NVM Software Controlled Secondary Storage Secondary Storage 3

Emerging System Core Core Cache Cache Hardware Controlled Need Efficient Persistency Primitives! DRAM NVM Software Controlled Secondary Storage Secondary Storage 3

Outline • Emergence of Persistent Memory • Ordering Persist Operations [MICRO ’15] • Atomic Durability [HPCA ’17] • Atomic (durability + visibility): Durable HTM • Conclusions 4

Linked List - Naïve Cache HEAD Pseudo-code Node Node Node 1. Create Node 0 1 2 2. Update Node Pointer 3. Update Head Pointer HEAD Node Node Node 0 1 2 NVM 5

Linked List - Naïve Cache HEAD Pseudo-code Node Node Node 1. Create Node 0 1 2 2. Update Node Pointer 3. Update Head Pointer HEAD Node Node Node 0 1 2 NVM 5

Linked List - Naïve Cache Pseudo-code 1. Create Node System Crash! 2. Update Node Pointer 3. Update Head Pointer HEAD Node Node Node 0 1 2 Reordering of writes to NVM renders data inconsistent. NVM 5

Linked List - Failsafe Cache Pseudo-code HEAD 1. Create Node Node Node Node 0 1 2 2. Update Node Pointer 3. Persist Barrier HEAD 4. Update Head Pointer Node Node Node 0 1 2 NVM 6

Linked List - Failsafe Cache Pseudo-code HEAD 1. Create Node Node Node Node 0 1 2 2. Update Node Pointer 3. Persist Barrier HEAD 4. Update Head Pointer Node Node Node 0 1 2 NVM 6

Strict Barrier* St a St b St c St a Epoch 1 Epoch 2 Epoch 3 Persist Barrier a b c a d e d p q d Visibility St d St e b a c d e Persistence St d Persist Barrier St p St q St d … * Pelley et. al., “Memory Persistency”, in ISCA-2014. 7

Strict Barrier* St a St b St c St a Epoch 1 Epoch 2 Epoch 3 Persist Barrier a b c a d e d p q d Visibility St d St e b a c d e Persistence St d Persist Barrier St p St q St d … * Pelley et. al., “Memory Persistency”, in ISCA-2014. 7

Strict Barrier* St a St b St c St a Epoch 1 Epoch 2 Epoch 3 Persist Barrier a b c a d e d p q d Visibility St d St e b a c d e Persistence St d Persist Barrier St p St q St d … * Pelley et. al., “Memory Persistency”, in ISCA-2014. 7

Strict Barrier* St a St b St c St a Epoch 1 Epoch 2 Epoch 3 Persist Barrier a b c a d e d p q d Visibility St d St e b a c d e Persistence St d Persist Barrier St p St q St d … * Pelley et. al., “Memory Persistency”, in ISCA-2014. 7

Strict Barrier* St a St b St c St a Epoch 1 Epoch 2 Epoch 3 Persist Barrier a b c a d e d p q d Visibility St d St e b a c d e Persistence St d Persist Barrier St p St q St d … * Pelley et. al., “Memory Persistency”, in ISCA-2014. 7

Strict Barrier* St a St b St c St a Epoch 1 Epoch 2 Epoch 3 Persist Barrier a b c a d e d p q d Visibility St d St e b a c d e Persistence St d Persist Barrier St p St q St d … * Pelley et. al., “Memory Persistency”, in ISCA-2014. 7

Strict Barrier* St a St b St c St a Epoch 1 Epoch 2 Epoch 3 Persist Barrier a b c a d e d p q d Visibility St d St e b a c d e Persistence St d Persist Barrier St p St q St d … Persist operations happen in the critical path of execution. * Pelley et. al., “Memory Persistency”, in ISCA-2014. 7

Lazy Barrier (LB)* • Durability lags visibility • Buffered barrier semantics • To allow performing persist operations out of critical path * Pelley et. al., “Memory Persistency”, in ISCA-2014. * Condit et. al., “Better I/O through byte-addressable, persistent memory”, in SOSP-2009. 8

Lazy Barrier (LB)* • Durability lags visibility • Buffered barrier semantics • To allow performing persist operations out of critical path Significant perf. improvement over strict barrier * Pelley et. al., “Memory Persistency”, in ISCA-2014. * Condit et. al., “Better I/O through byte-addressable, persistent memory”, in SOSP-2009. 8

Conflicts: Lazy Barrier (LB) Epoch 1 Epoch 2 Epoch 3 Visibility a b c a d e d p q d − Conflicting request a b c d e Persistence Cache Line Eviction * Pelley et. al., “Memory Persistency”, in ISCA-2014. * Condit et. al., “Better I/O through byte-addressable, persistent memory”, in SOSP-2009. 9

Conflicts: Lazy Barrier (LB) Epoch 1 Epoch 2 Epoch 3 Visibility a b c a d e d p q d − Conflicting request a b c d e Persistence Cache Line Eviction * Pelley et. al., “Memory Persistency”, in ISCA-2014. * Condit et. al., “Better I/O through byte-addressable, persistent memory”, in SOSP-2009. 9

Conflicts: Lazy Barrier (LB) Epoch 1 Epoch 2 Epoch 3 Visibility a b c a d e d p q d − Conflicting request a b c d e Persistence Cache Line Eviction * Pelley et. al., “Memory Persistency”, in ISCA-2014. * Condit et. al., “Better I/O through byte-addressable, persistent memory”, in SOSP-2009. 9

Conflicts: Lazy Barrier (LB) Epoch 1 Epoch 2 Epoch 3 Visibility a b c a d e d p q d − Conflicting request a b c d e Persistence * Pelley et. al., “Memory Persistency”, in ISCA-2014. * Condit et. al., “Better I/O through byte-addressable, persistent memory”, in SOSP-2009. 9

Conflicts: Lazy Barrier (LB) Epoch 1 Epoch 2 Epoch 3 Visibility a b c a d e d p q d − Conflicting request a b c d e Persistence Conflicts bring persist operations back in the critical path. * Pelley et. al., “Memory Persistency”, in ISCA-2014. * Condit et. al., “Better I/O through byte-addressable, persistent memory”, in SOSP-2009. 9

Conflicts: Lazy Barrier (LB) Intra-thread conflict Epoch 1 Epoch 2 Epoch 3 Visibility a b c a d e d p q d − Conflicting request a b c d e Persistence Conflicts bring persist operations back in the critical path. * Pelley et. al., “Memory Persistency”, in ISCA-2014. * Condit et. al., “Better I/O through byte-addressable, persistent memory”, in SOSP-2009. 9

Inter-thread Conflict Epoch E 00 Thread T 0 W W W W Visibility A B E F A B E F Persistence Z E R P R B R Q R X R Y W W Thread T 1 Z E Visibility Epoch E 10 Epoch E 11 10

Inter-thread Conflict Epoch E 00 Thread T 0 W W W W Visibility A B E F A B E F Persistence Z E R P R B R Q R X R Y W W Thread T 1 Z E Visibility Epoch E 10 Epoch E 11 10

Inter-thread Conflict Epoch E 00 Thread T 0 W W W W Visibility A B E F A B E F Persistence Z E R P R B R Q R X R Y W W Thread T 1 Z E Visibility Epoch E 10 Epoch E 11 10

Inter-thread Conflict Epoch E 00 Thread T 0 W W W W Visibility A B E F A B E F Persistence Z E R P R B R Q R X R Y W W Thread T 1 Z E Visibility Epoch E 10 Epoch E 11 10

Two Ideas • Proactive flushing (PF) • Predict when a cache block val is final and flush • Inter-thread dependence tracking (IDT) • Track inter-thread dependencies, enforce lazily 11

Evaluation LB Lazy barrier LB+IDT Lazy barrier with inter-thread dependence tracking (IDT) LB+PF Lazy barrier with proactive flush (PF) LB++ Lazy barrier with both IDT and PF Persist Barrier Designs • System Configuration • We evaluate proposed design using GEM5 full- system simulation mode • 32 Core CMP with 32x1MB LLC cache banks and 4 memory controllers 12

Transaction Throughput Higher is Better 13

Transaction Throughput 15% Higher is Better 13

Transaction Throughput 22% Higher is Better 13

Outline • Emergence of Persistent Memory • Ordering Persist Operations • Atomic Durability • Atomic (durability + visibility): Durable HTM • Conclusions 14

Atomic Durability • All or nothing persists: think transactions ( A CI D ) Final State Initial State A 0 B 0 A 0 B 0 Final State A 1 B 1 Atomic_Begin A = 1 Final State Final State B = 1 Atomic_End A 1 B 0 A 0 B 1 15

Mechanisms Write-Ahead-Logging REDO UNDO ➡ read redirection ➡ fine grained log->data ordering 16

ATOM • Create Undo Log Core • on a store, cache writes old value to log Cache A 0 • Flush Undo Log • enforce log —> data NVM Data Log ordering A 0 17

ATOM • Create Undo Log A = 1 Core • on a store, cache writes old value to log Cache A 0 • Flush Undo Log • enforce log —> data NVM Data Log ordering A 0 17

ATOM • Create Undo Log A = 1 Core • on a store, cache writes old value to log Cache A 0 L(A) = 0 • Flush Undo Log • enforce log —> data NVM Data Log ordering A 0 A 0 17

ATOM • Create Undo Log A = 1 Core • on a store, cache Log writes old value to log Cache Done A 0 1 L(A) = 0 • Flush Undo Log • enforce log —> data NVM Data Log ordering A 0 A 0 17