Pawe Lebioda, Tomasz Kapela Agenda Introduction: Tutorial: - PowerPoint PPT Presentation

Paweł Lebioda, Tomasz Kapela

Agenda Introduction: Tutorial: Persistent Memory Kernel support • • NVML NVML overview • • PMEMCHECK libpmem • • libpmemblk • libpmemobj C API • libpmemobj C++ API • PMEMCHECK • 2

Just a little bit of theory...

What is it ?

Persistent Memory What is Persistent Memory ?  Non-volatile  Retains data without power  Byte addressable  Load/Store access  Memory-like performance  DMA-able  NVDIMM 5

Persistent Memory use cases  Block storage  Volatile Memory  Persistent Memory aware filesystem  Persistent Memory aware applications 6

Persistent Memory characteristics  Stores are not durable  The data must be flushed  8-byte atomicity  Virtual address may change 7

Persistent Memory kernel support  NVDIMM subsystem  DAX (Direct Access)  ndctl – github.com/pmem/ndctl  DAX support in ext2, ext4, xfs 8

Non-Volatile Memory Libraries https://github.com/pmem/nvml

Persistent Memory Programming Model 10

NVML introduction  Set of user-space libraries  PMEM-aware applications  Source code on github - github.com/pmem/nvml  BSD license  Web page - pmem.io  Distros (Fedora, openSUSE) 11

NVML libraries Library Description libvmem Volatile memory allocator libvmmalloc malloc() replacement using libvmem libpmem Low-level support for persistence libpmemlog Append only log structure libpmemblk Atomic access to memory blocks libpmemobj Transactional object store libpmempool Managing library 12

„In theory, theory and practice are the same. In practice, they are not” Albert Einstein

How to configure my Linux OS for persistent memory ?

Kernel support  Kernel config: NVDIMM, DAX  Persistent memory emulation: – http://pmem.io/2016/02/22/pm-emulation.html – X86_PMEM_LEGACY – Memmap=<size>!<offset>  Create filesystem  Mount with DAX 15

Persistent Memory configuration 16

Quick insight into source tree

NVML overview  Source tree organization  Building  Running tests  Examples  Packages 18

NVML packages 19

Low level persistent memory support

libpmem examples  How to check if memory is Persistent Memory ?  How to access Persistent Memory ?  Debug library and log messages  How to make memory persistent ?  Running examples under Valgrind pmemcheck 21

How to check if memory is Persistent Memory ? 22

How to access Persistent Memory ? 23

Running pmem_is_pmem example 24

libpmem: turning on log messages 25

libpmem: pmem_persist - 1 26

libpmem: pmem_persist - 2 27

Persistent Memory aware tool for Valgrind github.com/pmem/valgrind

PMEMCHECK  Valgrind tool  Need to build nvml with special flags  Checks correctness of persistent memory usage  Checking data persistence  Support for transactions (libpmemobj)  NVML: make EXTRA_CFLAGS=-DUSE_VALGRIND 29

libpmem: valgrind support 30

libpmem: pmem_persist + valgrind pmemcheck 31

libpmem: pmem_persist - 2 32

Persistent Memory pools utils

pmempool: available commands 34

Persistent Memory resident array of blocks

Libpmemblk: examples  Pmempool introduction  Creating pmemblk pool with pmempool  Opening pmemblk pool  Reading pool parameters  Reading and writing blocks 36

pmempool: create command 37

Opening pmemblk pool 38

Reading and writing block 39

libpmempool: running example 40

Persistent Memory Transactional Object Store

libpmemobj  Creating pmemobj pool with pmempool  Opening pmemobj pool  PMEMoid – persistent memory pointer  Root object  Allocations 42

Libpmemobj: simple example  Application which stores list of people  Stores first and last name  Constant size array 43

Data structures 44

Main function 45

List of people 46

Insert function - 1 47

Insert function - 2 48

Object’s constructor 49

Clear function 50

Recovery 51

Running example 52

Missing persist 53

Valgrind pmemcheck 54

libpmemobj: type safety macros  PMEMoid as void *  Type safety macros for convenience  No need to track persistent and volatile pointers 55

Type safety macros: data structures 56

Type safety macros: insert function 57

Type safety macros: clear function 58

libpmemobj: foreach macros  API for traversing through all objects  Foreach object per type  Useful for recovery actions  Avoids persistent memory leaks 59

Foreach type macro 60

libpmemobj: atomic lists API  Circular doubly linked list  Atomic operations  sys/queue – like API  Useful for hybrid persistent-volatile data structures 61

atomic lists API: data structures 62

Atomic lists API: insert function 63

Atomic lists: clear function 64

Atomic lists API: listing people 65

Atomic lists API: recovery 66

libpmemobj: transactions  Most generic API  No need for recovery  Convenient macros  Operations: – Alloc – Free – Modify 67

Transactions: data structures 68

Transactions: insert function 69

Transactions: entry initialization 70

Transactions: inserting elements 71

Transactions: clear function 72

Transactions: listing people 73

Transactions: valgrind pmemcheck 74

Transactions: valgrind pmemcheck 75

C++ API  Using language features  Overloading operators  Hiding required operations  Scope transactions  Lambda 76

C++ API: main function 77

C++ API: list of people class 78

C++ API: listing people 79

C++ API: insert function 80

C++ API: clear function 81

C++ API: list entry class 82

C++ API: persistent string 83

C++ API: valgrind pmemcheck 84

C++ API: valgrind pmemcheck - 1 85

C++ API: valgrind pmemcheck - 2 86

C++ STL  STL allocator concept for libpmemobj  Using STL containers  Need some changes in std libraries 87

C++ STL: list of people class 88

C++ STL: list entry class 89