Architectural Principles for Database Systems on Storage-Class - - PowerPoint PPT Presentation

Architectural Principles for Database Systems on Storage-Class Memory

Dr.-Ing. Ismail Oukid Dissertationspreis Vortrag BTW Rostock, March 8, 2019

2

Storage-Class Memory

21 23 25 27 29 211 213 215 217 219 221 223 DRAM SCM Flash Hard Drive LL Cache EDRAM L1 Cache SRAM Main Memory High Performance Disk

Access Latency in Cycles for a 4 GHz Processor [1]

[1] Qureshi et al. “Phase change memory: From devices to systems”. Synthesis Lectures of Computer Science, 2011.

SCM is a merging point between memory and storage

SCM is byte-addressable non-volatile memory

3

Research Question

What are the implications of SCM for Database Systems?

Goal: Explore opportunities and challenges

Already investigated in the 1980s for FRAM and battery-backed DRAM What is different this time? Supported by Linux and Windows Standardized NVDIMM form factor New SCM-targeted instructions

Slide partially inspired from Joy Arulraj‘s talk, “Write-Behind Logging”, SAP, September 2017

4

Disk

buffer pool load/store operations file API

DRAM

SCM in the Memory Hierarchy

L3 - Cache Core Core L1 L2 L1 L2

Store Buffer Store Buffer MOV

…

CPU

5

Disk

buffer pool persistency boundary

DRAM

SCM in the Memory Hierarchy

L3 - Cache Core Core L1 L2 L1 L2

Store Buffer Store Buffer

…

CPU

Propagation to stable storage under control of the DBMS

UNDO / REDO

6

Disk

buffer pool same load/store operations file API

DRAM

SCM in the Memory Hierarchy

L3 - Cache Core Core L1 L2 L1 L2

Store Buffer Store Buffer MOV

…

CPU

MOV

SCM

7

Disk

buffer pool

DRAM SCM

SCM in the Memory Hierarchy

L3 - Cache Core Core L1 L2 L1 L2

Store Buffer Store Buffer

…

CPU persistency boundary

Hardware controls data propagation to SCM

9

Testing Framework for SCM-Based Databases Persistent Memory Management Concurrency Control and Recovery Primitives Recovery Strategies

SOFORT

Persistent Data Structures

Path-Finding for Database Systems on SCM

10

Testing Framework for SCM-Based Databases Persistent Memory Management Concurrency Control and Recovery Primitives Recovery Strategies

SOFORT

Persistent Data Structures

Path-Finding for Database Systems on SCM

Oukid et al. Memory Management Techniques for Large-Scale Persistent-Main-Memory Systems. In VLDB‘17

11

Persistent vs. Transient Memory Allocator

• 1. Provide a recoverable addressing scheme
• 2. Avoid persistent memory leaks

SCM DRAM Application application address space Virtual memory subsystem Persistent allocator Transient allocator

12

Testing Framework for SCM-Based Databases Persistent Memory Management Concurrency Control and Recovery Primitives Recovery Strategies

SOFORT

Persistent Data Structures

Path-Finding for Database Systems on SCM

Oukid et al. FPTree: A Hybrid SCM-DRAM Persistent and Concurrent B-Tree for Storage Class Memory. In SIGMOD‘16

13

• 1. Selective Persistence

2 5 7 6 7 3 4 5 1 2 L5 L4 L3

Persistent (SCM)

L2 L1 L7 L6

Volatile (DRAM) Inner nodes in DRAM for better performance Leaves in SCM to ensure durability Inner nodes rebuilt from leaves upon recovery in O(#entries)

Recovery is up to 100x faster than a full rebuild

15

Hardware Transactional Memory (HTM) 1 2

XBEGIN XEND

Critical section Time Thread 1 Thread 2

1 2

L1 Cache

XBEGIN XEND

Critical section

HTM and SCM are apparently incompatible

Transactions keep read and write sets in L1 cache

FLUSH → Abort!

Allows optimistic execution of critical sections

16

Persistent – Fine-grained locking Transient – Hardware Transactional Memory

• 4. Selective Concurrency

2 5 7 6 7 3 4 5 1 2 L5

L4

L7 L6 L4 L1 L2 L3

17

Testing Framework for SCM-Based Databases Persistent Memory Management Concurrency Control and Recovery Primitives Recovery Strategies

SOFORT

Persistent Data Structures

Path-Finding for Database Systems on SCM

Oukid et al. SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery. In DaMoN‘14 Oukid et al. Instant Recovery for Main-Memory Databases. In CIDR‘15

18

Adapting MVCC to SCM DRAM SCM

Transaction object

• Transaction ID (TXID)
• Start timestamp
• List of read rows
• List of modified rows
• Commit timestamp (CTS)

Robinson Crusoe Despair Island CTS

Undo Log

Robinson Crusoe London CTS

Redo Log

Employee Office Timestamps Robinson Crusoe Despair Island (1, 4) Oliver Twist London (2, inf) August the Strong Dresden (3, inf) Robinson Crusoe London (4, inf)

Intuition: {list of modified rows + versioned data} ~= log

Transaction object

• List of modified rows
• Commit timestamp

No need for a central logging component

19

Benefits of our MVCC Adaption to SCM

Less system architecture complexity Potential for better scalability ✓ No central logging component Reduced data redundancy Faster recovery ✓ No redo phase during recovery Lower transaction latency ✓ No need for group commit

20

Path-Finding for Database Systems on SCM

Oukid et al. Adaptive Recovery for SCM-Enabled Databases. In ADMS‘17

Testing Framework for SCM-Based Databases Persistent Memory Management Concurrency Control and Recovery Primitives Recovery Strategies

SOFORT

Persistent Data Structures

21

Adaptive Recovery

Intuition 1: Secondary data structures are not equally important Intuition 2: Some indexes are not useful for the current workload Recover indexes in order of importance → How to rank indexes? Release recovery resources after rebuilding important indexes → When to release resources?

22

Testing Framework for SCM-Based Databases Persistent Memory Management Concurrency Control and Recovery Primitives Recovery Strategies

SOFORT

Persistent Data Structures

Path-Finding for Database Systems on SCM

Oukid et al. On Testing Persistent-Memory-Based Software. In DaMoN‘16

23

Testing of SCM-Based Software

24

Thesis Goal: Research SCM opportunities and challenges for database systems Thesis Contributions: → Identified several SCM opportunities and challenges → Devised building blocks to solve the challenges and leverage the opportunities → Built SOFORT to demonstrate the opportunities brought by SCM for database systems

Pave the way for future database systems on SCM Conclusion

25

Acknowledgements

Adrien Lespinasse Johan Lasperas Daniel Bossle Grégoire Gomes

Supervised Students

• Prof. Wolfgang Lehner

Thomas Willhalm Anisoara Nica Daniel Booss Peter Bumbulis

Supervisors

26

Acknowledgements

Arne Schwarz Ingo Müller Marcus Paradies Michael Rudolf Hannes Rauhe Iraklis Psaroudakis Elena Vasilyeva Florian Wolf David Kernert Robert Brunel Georgios Psaropoulos Lucas Lersch Thomas Bach

SAP HANA Campus TUD Database Systems Group

Frank Tetzel Matthias Hauck Robin Rehrmann Stefan Noll Jonathan Dees Philipp Große Martin Kaufmann Francesc Trull Lars Dannecker Ulrike Fischer Katrin Braunschweig Julian Eberius Tim Kiefer Robert Ulbricht Ines Funke Ulrike Schöbel Dirk Habich Maik Thiele Martin Hahmann Hannes Voigt Thomas Kissinger Tobias Jäkel Tomas Karnagel Till Kolditz Claudio Hartmann Kai Herrmann Kasun Perera Rihan Hai Ahmad Ahmadov Johannes Luong Annett Ungethüm Patrick Damme Lars Kegel Juliana Hildebrandt Alexander Krause Mikhail Zarubin Elvis Koci Nusrat Lisa Jahan Muhammad Idris Rana Faisal Munir

SAP and Intel Colleagues

Norman May Alexander Boehm Roman Dementiev Otto Bruggeman Oliver Rebholz Abdelkader Sellami

Thesis Co-examiner

• Prof. Stefan

Manegold

27

Thesis Goal: Research SCM opportunities and challenges for database systems Thesis Contributions: → Identified several SCM opportunities and challenges → Devised building blocks to solve the challenges and leverage the opportunities → Built SOFORT to demonstrate the opportunities brought by SCM for database systems

Pave the way for future database systems on SCM Conclusion Thank you!