SSDAlloc: Hybrid SSD/RAM Memory Management Made Easy Anirudh Badam - - PowerPoint PPT Presentation

ssdalloc hybrid ssd ram memory management made easy
SMART_READER_LITE
LIVE PREVIEW

SSDAlloc: Hybrid SSD/RAM Memory Management Made Easy Anirudh Badam - - PowerPoint PPT Presentation

SSDAlloc: Hybrid SSD/RAM Memory Management Made Easy Anirudh Badam Vivek S. Pai Princeton University 03/31/2011 1 Memory in Networked Systems 2 Memory in Networked Systems As a cache to reduce pressure on the disk Memcache like


slide-1
SLIDE 1

SSDAlloc: Hybrid SSD/RAM Memory Management Made Easy

Anirudh Badam Vivek S. Pai

Princeton University 03/31/2011

1

slide-2
SLIDE 2

Memory in Networked Systems

2

slide-3
SLIDE 3

Memory in Networked Systems

  • As a cache to reduce pressure on the disk
  • Memcache like tools
  • Act as front-end caches for Web data back-end

2

slide-4
SLIDE 4

Memory in Networked Systems

  • As a cache to reduce pressure on the disk
  • Memcache like tools
  • Act as front-end caches for Web data back-end
  • As an index to reduce pressure on the disk
  • Indexes for proxy caches, WAN accelerators and

inline data-deduplicators

  • Help avoid false positives and use the disk

effectively

2

slide-5
SLIDE 5

Problem: Memory Density

3

slide-6
SLIDE 6

Problem: Memory Density

8 16 32 64 128 256 512 1024 37.5 75 112.5 150

$ / GB (Total Cost) Total DRAM

3

slide-7
SLIDE 7

Problem: Memory Density

8 16 32 64 128 256 512 1024 37.5 75 112.5 150

$ / GB (Total Cost) Total DRAM

3

slide-8
SLIDE 8

Problem: Memory Density

8 16 32 64 128 256 512 1024 37.5 75 112.5 150

$ / GB (Total Cost)

Breaking Point

3

slide-9
SLIDE 9

Problem: Memory Density

8 16 32 64 128 256 512 1024 37.5 75 112.5 150

$ / GB (Total Cost)

Breaking Point

? ?

3

slide-10
SLIDE 10

Problem: Disk Speed Limits

4

slide-11
SLIDE 11
  • Magnetic disk speed is not scaling well
  • Capacity is increasing but seek latency is not

decreasing

  • About 200 seeks/disk/sec

Problem: Disk Speed Limits

4

slide-12
SLIDE 12
  • Magnetic disk speed is not scaling well
  • Capacity is increasing but seek latency is not

decreasing

  • About 200 seeks/disk/sec
  • High speed disk arrays: many smaller capacity drives
  • Total cost about 10X more compared to similar

capacity 7200 rpm drives

  • Use more rack space per byte

Problem: Disk Speed Limits

4

slide-13
SLIDE 13

Proposal: Use Flash as Memory

5

slide-14
SLIDE 14

Proposal: Use Flash as Memory

  • Address DRAM density limitation
  • Overcome per system DRAM limits via flash memory
  • Provide a choice -- more servers or a single server + flash

memory

5

slide-15
SLIDE 15

Proposal: Use Flash as Memory

  • Address DRAM density limitation
  • Overcome per system DRAM limits via flash memory
  • Provide a choice -- more servers or a single server + flash

memory

  • Reduce total cost of ownership
  • “Long-tailed” workloads are important
  • DRAM too expensive and disk too slow
  • CPU under-utilized due to DRAM limit

5

slide-16
SLIDE 16

Proposal: Use Flash as Memory

  • Address DRAM density limitation
  • Overcome per system DRAM limits via flash memory
  • Provide a choice -- more servers or a single server + flash

memory

  • Reduce total cost of ownership
  • “Long-tailed” workloads are important
  • DRAM too expensive and disk too slow
  • CPU under-utilized due to DRAM limit
  • How to ease application development with flash

memory?

5

slide-17
SLIDE 17

Flash Memory Primer

6

slide-18
SLIDE 18

Flash Memory Primer

  • Fast random reads (upto 1M IOPS per drive)

6

slide-19
SLIDE 19

Flash Memory Primer

  • Fast random reads (upto 1M IOPS per drive)
  • Writes happen after an erase
  • Limited lifetime and endurance

6

slide-20
SLIDE 20

Flash Memory Primer

  • Fast random reads (upto 1M IOPS per drive)
  • Writes happen after an erase
  • Limited lifetime and endurance
  • No seek latency (only read/write latency)

6

slide-21
SLIDE 21

Flash Memory Primer

  • Fast random reads (upto 1M IOPS per drive)
  • Writes happen after an erase
  • Limited lifetime and endurance
  • No seek latency (only read/write latency)
  • Large capacity (single 2.5’’ disk ~ 512GB)
  • PCIe 10.2 TB - Fusion-io io-octal drive

6

slide-22
SLIDE 22

Question of Hierarchy

7

slide-23
SLIDE 23

Disk

Question of Hierarchy

Memory

7

slide-24
SLIDE 24

Disk Block Addressable

Question of Hierarchy

Memory Byte Addressable

7

slide-25
SLIDE 25

Disk Block Addressable Directly Addressed

Question of Hierarchy

Memory Byte Addressable Virtually Addressed

7

slide-26
SLIDE 26

Disk Block Addressable Directly Addressed High Latency

Question of Hierarchy

Memory Byte Addressable Virtually Addressed Low Latency

7

slide-27
SLIDE 27

Disk Block Addressable Directly Addressed High Latency Non-volatile

Question of Hierarchy

Memory Byte Addressable Virtually Addressed Low Latency Volatile

7

slide-28
SLIDE 28

Disk Block Addressable Directly Addressed High Latency Non-volatile

Question of Hierarchy

Memory Byte Addressable Virtually Addressed Low Latency Volatile

7

slide-29
SLIDE 29

SSDs

Disk Block Addressable Directly Addressed High Latency Non-volatile

Question of Hierarchy

Memory Byte Addressable Virtually Addressed Low Latency Volatile

7

slide-30
SLIDE 30

SSDs

Disk Block Addressable Directly Addressed High Latency Non-volatile

Question of Hierarchy

Memory Byte Addressable Virtually Addressed Low Latency Volatile Flash has low latency

7

slide-31
SLIDE 31

SSDs

Disk Block Addressable Directly Addressed High Latency Non-volatile

Question of Hierarchy

Memory Byte Addressable Virtually Addressed Low Latency Volatile Flash has low latency

7

slide-32
SLIDE 32

Transparent Tiering Today

8

slide-33
SLIDE 33
  • Use it as memory via swap or mmap
  • Application need not be modified
  • Pages transparently swapped in and out based
  • n usage in DRAM

Transparent Tiering Today

8

slide-34
SLIDE 34
  • Use it as memory via swap or mmap
  • Application need not be modified
  • Pages transparently swapped in and out based
  • n usage in DRAM
  • Native pager delivers only 10% of the SSD’s

performance

  • Flash aware pager delivers only 30% of the SSD’s

performance

  • OS pager optimized for seek limited disks and

was designed as a “dead page storage”

Transparent Tiering Today

8

slide-35
SLIDE 35

Transparent Tiering Today

1 2 3 4 13 14 15 16 5 6 7 8 9 10 11 12 1 2 3 4 17 18 19 20 33 34 35 36 49 50 51 52 5 6 7 8 21 22 23 24 37 38 39 40 53 54 55 56 9 10 11 12 25 26 27 28 41 42 43 44 57 58 59 60 29 30 31 32 45 46 47 48 61 62 63 64

RAM SSD

13 14 15 16 29 30 31 32

9

slide-36
SLIDE 36

Transparent Tiering Today

1 2 3 4 13 14 15 16 5 6 7 8 9 10 11 12 1 2 3 4 17 18 19 20 33 34 35 36 49 50 51 52 5 6 7 8 21 22 23 24 37 38 39 40 53 54 55 56 9 10 11 12 25 26 27 28 41 42 43 44 57 58 59 60 29 30 31 32 45 46 47 48 61 62 63 64

RAM SSD

13 14 15 16 read 29 30 31 32

9

slide-37
SLIDE 37

Transparent Tiering Today

1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 17 18 19 20 33 34 35 36 49 50 51 52 5 6 7 8 21 22 23 24 37 38 39 40 53 54 55 56 9 10 11 12 25 26 27 28 41 42 43 44 57 58 59 60 29 30 31 32 45 46 47 48 61 62 63 64

RAM SSD

13 14 15 16 read 29 30 31 32

9

slide-38
SLIDE 38

Transparent Tiering Today

1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 17 18 19 20 33 34 35 36 49 50 51 52 5 6 7 8 21 22 23 24 37 38 39 40 53 54 55 56 9 10 11 12 25 26 27 28 41 42 43 44 57 58 59 60 29 30 31 32 45 46 47 48 61 62 63 64

RAM SSD

13 14 15 16 read 29 30 31 32

9

slide-39
SLIDE 39

Transparent Tiering Today

1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 17 18 19 20 33 34 35 36 49 50 51 52 5 6 7 8 21 22 23 24 37 38 39 40 53 54 55 56 9 10 11 12 25 26 27 28 41 42 43 44 57 58 59 60 29 30 31 32 45 46 47 48 61 62 63 64

RAM SSD

13 14 15 16 read 29 30 31 32

9

slide-40
SLIDE 40

Transparent Tiering Today

1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 17 18 19 20 33 34 35 36 49 50 51 52 5 6 7 8 21 22 23 24 37 38 39 40 53 54 55 56 9 10 11 12 25 26 27 28 41 42 43 44 57 58 59 60 29 30 31 32 45 46 47 48 61 62 63 64

RAM SSD

13 14 15 16 read 29 30 31 32 write

9

slide-41
SLIDE 41

Transparent Tiering Today

1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 17 18 19 20 33 34 35 36 49 50 51 52 5 6 7 8 21 22 23 24 37 38 39 40 53 54 55 56 9 10 11 12 25 26 27 28 41 42 43 44 57 58 59 60 29 30 31 32 45 46 47 48 61 62 63 64

RAM SSD

13 14 15 16 read 29 30 31 32 write

9

slide-42
SLIDE 42

Transparent Tiering Today

1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 17 18 19 20 33 34 35 36 49 50 51 52 5 6 7 8 21 22 23 24 37 38 39 40 53 54 55 56 29 30 31 32 45 46 47 48 61 62 63 64

RAM SSD

13 14 15 16 read 29 30 31 32 write

9

slide-43
SLIDE 43

Transparent Tiering Today

1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 17 18 19 20 33 34 35 36 49 50 51 52 5 6 7 8 21 22 23 24 37 38 39 40 53 54 55 56 9 10 11 12 25 26 27 28 41 42 43 44 57 58 59 60 29 30 31 32 45 46 47 48 61 62 63 64

RAM SSD

13 14 15 16 read 29 30 31 32 write

9

slide-44
SLIDE 44

Transparent Tiering Today

1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 17 18 19 20 33 34 35 36 49 50 51 52 5 6 7 8 21 22 23 24 37 38 39 40 53 54 55 56 9 10 11 12 25 26 27 28 41 42 43 44 57 58 59 60 29 30 31 32 45 46 47 48 61 62 63 64

RAM SSD

13 14 15 16 read 29 30 31 32 write free()

9

slide-45
SLIDE 45

Transparent Tiering Today

1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 17 18 19 20 33 34 35 36 49 50 51 52 5 6 7 8 21 22 23 24 37 38 39 40 53 54 55 56 9 10 11 12 25 26 27 28 41 42 43 44 57 58 59 60 29 30 31 32 45 46 47 48 61 62 63 64

RAM SSD

13 14 15 16 read 29 30 31 32 write free()

9

slide-46
SLIDE 46

Transparent Tiering Today

1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 17 18 19 20 33 34 35 36 49 50 51 52 9 10 11 12 25 26 27 28 41 42 43 44 57 58 59 60 29 30 31 32 45 46 47 48 61 62 63 64

RAM SSD

13 14 15 16 read 29 30 31 32 write free()

9

slide-47
SLIDE 47

Transparent Tiering Today

1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 17 18 19 20 33 34 35 36 49 50 51 52 5 6 7 8 21 22 23 24 37 38 39 40 53 54 55 56 9 10 11 12 25 26 27 28 41 42 43 44 57 58 59 60 29 30 31 32 45 46 47 48 61 62 63 64

RAM SSD

13 14 15 16 read 29 30 31 32 write free()

9

slide-48
SLIDE 48

Transparent Tiering Today

1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 17 18 19 20 33 34 35 36 49 50 51 52 5 6 7 8 21 22 23 24 37 38 39 40 53 54 55 56 9 10 11 12 25 26 27 28 41 42 43 44 57 58 59 60 29 30 31 32 45 46 47 48 61 62 63 64

RAM SSD

13 14 15 16 read 29 30 31 32 write free()

Indirection Table

(log structured page store)

In the OS

  • r in the FTL

9

slide-49
SLIDE 49

Non-Transparent Tiering

10

slide-50
SLIDE 50

Non-Transparent Tiering

  • Redesign application to be flash aware
  • Custom object store with custom pointers
  • Reads, writes and garbage collection at an

application object granularity

  • Avoid in-place writes (objects could be small)
  • Obtain the best performance and lifetime from

flash memory device

10

slide-51
SLIDE 51

Non-Transparent Tiering

  • Redesign application to be flash aware
  • Custom object store with custom pointers
  • Reads, writes and garbage collection at an

application object granularity

  • Avoid in-place writes (objects could be small)
  • Obtain the best performance and lifetime from

flash memory device

  • Intrusive modifications needed
  • Expertise with flash memory needed

10

slide-52
SLIDE 52

Non-Transparent Tiering

MyObject* obj = malloc( sizeof( MyObject ) );

  • bj->x = 0;
  • bj->y = 1;
  • bj->z = 2;

free( obj );

malloc + SSD-swap

MyObjectID oid = createObject( sizeof( MyObject ) ); MyObject* obj = malloc( sizeof( MyObject ) ); readObject( oid, obj );

  • bj->x = 0;
  • bj->y = 1;
  • bj->z = 2;

writeObject( oid, obj ); free( obj );

Application Rewrite

11

slide-53
SLIDE 53

Our Goal

12

slide-54
SLIDE 54
  • Run mostly unmodified applications
  • Work via memory allocators in C-style programs

Our Goal

12

slide-55
SLIDE 55
  • Run mostly unmodified applications
  • Work via memory allocators in C-style programs
  • Use the DRAM effectively
  • Use it as an object cache (not as a page cache)

Our Goal

12

slide-56
SLIDE 56
  • Run mostly unmodified applications
  • Work via memory allocators in C-style programs
  • Use the DRAM effectively
  • Use it as an object cache (not as a page cache)
  • Use the SSD wisely
  • As a log-structured object store

Our Goal

12

slide-57
SLIDE 57
  • Run mostly unmodified applications
  • Work via memory allocators in C-style programs
  • Use the DRAM effectively
  • Use it as an object cache (not as a page cache)
  • Use the SSD wisely
  • As a log-structured object store
  • Reorganize virtual memory allocation to discern object

information

Our Goal

12

slide-58
SLIDE 58

SSDAlloc Overview

Application Virtual Memory (Object per page - OPP) Physical Memory SSD

13

slide-59
SLIDE 59

SSDAlloc Overview

Application Virtual Memory (Object per page - OPP) Physical Memory SSD

Memory Manager: Creates 64 objects of 1KB size

13

slide-60
SLIDE 60

SSDAlloc Overview

Application Virtual Memory (Object per page - OPP) Physical Memory SSD

Memory Manager: Creates 64 objects of 1KB size

1 2 3 4

...

61 62 63 64

13

slide-61
SLIDE 61

SSDAlloc Overview

Application Virtual Memory (Object per page - OPP) Physical Memory SSD

Memory Manager: Creates 64 objects of 1KB size

1 2 3 4

...

61 62 63 64 Page Buffer 1 12 33

13

slide-62
SLIDE 62

SSDAlloc Overview

Application Virtual Memory (Object per page - OPP) Physical Memory SSD

Memory Manager: Creates 64 objects of 1KB size

1 2 3 4

...

61 62 63 64 Page Buffer 1 12 33 RAM Object Cache

...

2 3 4 5 6 7 8 9 10 11 13 14 15 16 17 18 19 20 21 22 23 24 25 26

13

slide-63
SLIDE 63

SSDAlloc Overview

Application Virtual Memory (Object per page - OPP) Physical Memory SSD

Memory Manager: Creates 64 objects of 1KB size

1 2 3 4

...

61 62 63 64 Page Buffer 1 12 33

Log structured object store

RAM Object Cache

...

2 3 4 5 6 7 8 9 10 11 13 14 15 16 17 18 19 20 21 22 23 24 25 26

13

slide-64
SLIDE 64

SSDAlloc Options

14

slide-65
SLIDE 65

Object Per Page (OPP) Memory Page (MP) Data Entity Application Defined Objects 4KB objects (like pages)

SSDAlloc Options

14

slide-66
SLIDE 66

Object Per Page (OPP) Memory Page (MP) Data Entity Application Defined Objects 4KB objects (like pages) Memory Manager Pool Allocator Coalescing Allocator

SSDAlloc Options

14

slide-67
SLIDE 67

Object Per Page (OPP) Memory Page (MP) Data Entity Application Defined Objects 4KB objects (like pages) Memory Manager Pool Allocator Coalescing Allocator Virtual Memory

  • No. of objects * page_size
  • No. of pages *

page_size

SSDAlloc Options

14

slide-68
SLIDE 68

Object Per Page (OPP) Memory Page (MP) Data Entity Application Defined Objects 4KB objects (like pages) Memory Manager Pool Allocator Coalescing Allocator Virtual Memory

  • No. of objects * page_size
  • No. of pages *

page_size Physical Memory Separate Page Buffer & RAM Object Cache No such separation

SSDAlloc Options

14

slide-69
SLIDE 69

Object Per Page (OPP) Memory Page (MP) Data Entity Application Defined Objects 4KB objects (like pages) Memory Manager Pool Allocator Coalescing Allocator Virtual Memory

  • No. of objects * page_size
  • No. of pages *

page_size Physical Memory Separate Page Buffer & RAM Object Cache No such separation SSD Usage Log-structured Object Store Log-structured Page Store

SSDAlloc Options

14

slide-70
SLIDE 70

Object Per Page (OPP) Memory Page (MP) Data Entity Application Defined Objects 4KB objects (like pages) Memory Manager Pool Allocator Coalescing Allocator Virtual Memory

  • No. of objects * page_size
  • No. of pages *

page_size Physical Memory Separate Page Buffer & RAM Object Cache No such separation SSD Usage Log-structured Object Store Log-structured Page Store Code Changes Minimal changes restricted to memory allocation No changes needed

SSDAlloc Options

14

slide-71
SLIDE 71

SSDAlloc Overview

15

slide-72
SLIDE 72

SSDAlloc Overview

Application Virtual Memory SSD

RAM Object Cache

15

slide-73
SLIDE 73
  • A small set of pages in core

SSDAlloc Overview

Application Virtual Memory SSD

RAM Object Cache

Page Buffer

15

slide-74
SLIDE 74
  • A small set of pages in core
  • Pages materialized on demand

from RAM object cache/SSD

  • Restricted in size to minimize

RAM wastage (from OPP)

SSDAlloc Overview

Application Virtual Memory SSD

RAM Object Cache

Page Buffer Demand Fetching

15

slide-75
SLIDE 75
  • A small set of pages in core
  • Pages materialized on demand

from RAM object cache/SSD

  • Restricted in size to minimize

RAM wastage (from OPP)

  • Implemented using mprotect

SSDAlloc Overview

Application Virtual Memory SSD

RAM Object Cache

Page Buffer Demand Fetching

15

slide-76
SLIDE 76
  • A small set of pages in core
  • Pages materialized on demand

from RAM object cache/SSD

  • Restricted in size to minimize

RAM wastage (from OPP)

  • Implemented using mprotect

SSDAlloc Overview

Application Virtual Memory SSD

RAM Object Cache

Page Buffer Demand Fetching

15

slide-77
SLIDE 77
  • A small set of pages in core
  • Pages materialized on demand

from RAM object cache/SSD

  • Restricted in size to minimize

RAM wastage (from OPP)

  • Implemented using mprotect

SSDAlloc Overview

Application Virtual Memory SSD

RAM Object Cache

Page Buffer Demand Fetching

15

slide-78
SLIDE 78
  • A small set of pages in core
  • Pages materialized on demand

from RAM object cache/SSD

  • Restricted in size to minimize

RAM wastage (from OPP)

  • Implemented using mprotect
  • Page materialized in seg-fault

handler

SSDAlloc Overview

Application Virtual Memory SSD

RAM Object Cache

Page Buffer

X

Demand Fetching

15

slide-79
SLIDE 79
  • A small set of pages in core
  • Pages materialized on demand

from RAM object cache/SSD

  • Restricted in size to minimize

RAM wastage (from OPP)

  • Implemented using mprotect
  • Page materialized in seg-fault

handler

  • RAM Object Cache continuously

flushes dirty objects to the SSD in LRU order

SSDAlloc Overview

Application Virtual Memory SSD

RAM Object Cache

Page Buffer Dirty Objects

X

Demand Fetching

15

slide-80
SLIDE 80

SSD Maintenance

16

slide-81
SLIDE 81

SSD Maintenance

Virtual Memory SSD RAM Object Cache

Dirty Objects

Object Tables

16

slide-82
SLIDE 82

SSD Maintenance

16

slide-83
SLIDE 83
  • Copy-and-compact garbage-collector/log-writer
  • Seek optimizations not needed

SSD Maintenance

16

slide-84
SLIDE 84
  • Copy-and-compact garbage-collector/log-writer
  • Seek optimizations not needed
  • Read at the head and write live and dirty objects
  • Use Object Tables to determine liveness

SSD Maintenance

16

slide-85
SLIDE 85
  • Copy-and-compact garbage-collector/log-writer
  • Seek optimizations not needed
  • Read at the head and write live and dirty objects
  • Use Object Tables to determine liveness
  • Garbage is disposed
  • Objects written elsewhere are garbage
  • OPP object which is “free” is garbage

SSD Maintenance

16

slide-86
SLIDE 86

Implementation

17

slide-87
SLIDE 87

Implementation

  • 11,000 lines of C++ code (runtime library)
  • Implemented using mprotect, mmap, and madvise
  • SSDAlloc-OPP pool and array allocator
  • SSDAlloc-MP coalescing allocator (array allocations)
  • SSDFree frees the allocated data
  • Can coexist with malloc pointers

17

slide-88
SLIDE 88

SSD Usage Techniques

18

slide-89
SLIDE 89

SSD Usage Techniques

Technique Write Logging Access < 4KB Finegrained GC Avoid DRAM Pollution High Performance Programming Ease SSD Swap

SSD Swap (Write Logged) ✔

Application Rewrite

✔ ✔ ✔ ✔ ✔

18

slide-90
SLIDE 90

SSD Usage Techniques

Technique Write Logging Access < 4KB Finegrained GC Avoid DRAM Pollution High Performance Programming Ease SSD Swap

SSD Swap (Write Logged) ✔

Application Rewrite

✔ ✔ ✔ ✔ ✔

SSDAlloc

✔ ✔ ✔ ✔ ✔ ✔

18

slide-91
SLIDE 91

SSDAlloc Runtime Overhead

19

slide-92
SLIDE 92

SSDAlloc Runtime Overhead

Overhead Source Max Latency TLB Miss (DRAM Read) 0.014 μSec Object Table Lookup 0.046 μSec Page Materialization 0.138 μSec Page Dematerialization 0.172 μSec Signal Handling 0.666 μSec Combined Overhead 0.833 μSec

  • Overhead for SSDAlloc runtime intervention

19

slide-93
SLIDE 93

SSDAlloc Runtime Overhead

Overhead Source Max Latency TLB Miss (DRAM Read) 0.014 μSec Object Table Lookup 0.046 μSec Page Materialization 0.138 μSec Page Dematerialization 0.172 μSec Signal Handling 0.666 μSec Combined Overhead 0.833 μSec

  • Overhead for SSDAlloc runtime intervention
  • NAND Flash latency ~ 30-50 μSec

19

slide-94
SLIDE 94

SSDAlloc Runtime Overhead

Overhead Source Max Latency TLB Miss (DRAM Read) 0.014 μSec Object Table Lookup 0.046 μSec Page Materialization 0.138 μSec Page Dematerialization 0.172 μSec Signal Handling 0.666 μSec Combined Overhead 0.833 μSec

  • Overhead for SSDAlloc runtime intervention
  • NAND Flash latency ~ 30-50 μSec
  • Can reach 1 Million IOPS

19

slide-95
SLIDE 95

Experiments

20

slide-96
SLIDE 96
  • Comparing three allocation methods
  • malloc replaced with SSDAlloc-OPP
  • malloc replaced with SSDAlloc-MP
  • Swap

Experiments

20

slide-97
SLIDE 97
  • Comparing three allocation methods
  • malloc replaced with SSDAlloc-OPP
  • malloc replaced with SSDAlloc-MP
  • Swap
  • 2.4Ghz Quadcore CPU with 16GB RAM
  • RiData, Kingston, Intel X25-E, Intel X25-V and

Intel X25-M

Experiments

20

slide-98
SLIDE 98

Original Modified SSDAlloc-OP c-OPP’s gain vs Application Original LOC Modified LOC Swap SSDAlloc-MP Memcached 11,193 21 5.5 - 17.4x 1.4 - 3.5x B+Tree Index 477 15 4.3 - 12.7x 1.4 - 3.2x Packet Cache 1,540 9 4.8 - 10.1x 1.3 - 2.3x HashCache 20,096 36 5.3 - 17.1x 1.3 - 3.3x

Results Overview

21

slide-99
SLIDE 99

Original Modified SSDAlloc-OP c-OPP’s gain vs Application Original LOC Modified LOC Swap SSDAlloc-MP Memcached 11,193 21 5.5 - 17.4x 1.4 - 3.5x B+Tree Index 477 15 4.3 - 12.7x 1.4 - 3.2x Packet Cache 1,540 9 4.8 - 10.1x 1.3 - 2.3x HashCache 20,096 36 5.3 - 17.1x 1.3 - 3.3x

  • SSDAlloc applications write up to 32 times less data to the SSD

than when compared to the traditional VM style applications

Results Overview

21

slide-100
SLIDE 100

Microbenchmarks

22

slide-101
SLIDE 101

Microbenchmarks

  • 32GB array of 128 byte objects (32GB SSD, 2GB RAM)

22

slide-102
SLIDE 102

Microbenchmarks

  • 32GB array of 128 byte objects (32GB SSD, 2GB RAM)

3.75 7.5 11.25 15 SSDAlloc-OPP over Swap SSDAlloc-OPP over SSDAlloc-MP Throughput Gain Factor All Reads 25% Reads 50% Reads 75% Reads All Writes

22

slide-103
SLIDE 103

Memcached Benchmarks

23

slide-104
SLIDE 104

Memcached Benchmarks

  • 30GB SSD, 4GB RAM, 4 memcache clients
  • Memcache server slab allocator modified to use SSDAlloc

23

slide-105
SLIDE 105

Memcached Benchmarks

  • 30GB SSD, 4GB RAM, 4 memcache clients
  • Memcache server slab allocator modified to use SSDAlloc

23

12500 25000 37500 50000 128 256 512 1024 2048 4096 8192 Throughput (req/sec)

SSDAlloc-OPP SSDAlloc-MP Swap

Average Object Size

slide-106
SLIDE 106

Memcached Benchmarks

24

slide-107
SLIDE 107

Memcached Benchmarks

  • Performance for 50% reads and 50% writes

24

slide-108
SLIDE 108

Memcached Benchmarks

  • Performance for 50% reads and 50% writes

7500 15000 22500 30000 SSD-swap SSDAlloc-MP SSDAlloc-OPP Thtoughput (req/sec)

RiData Kingston Intel X25-E Intel X25-V Intel X25-M

24

slide-109
SLIDE 109

Summary

25

slide-110
SLIDE 110
  • SSDAlloc migrates SSD naturally into

VM system

  • RAM as a compact object cache
  • Virtual memory addresses are used
  • Only memory allocation code changes (9 to 36 LOC)
  • Other approaches need intrusive modifications

Summary

25

slide-111
SLIDE 111
  • SSDAlloc migrates SSD naturally into

VM system

  • RAM as a compact object cache
  • Virtual memory addresses are used
  • Only memory allocation code changes (9 to 36 LOC)
  • Other approaches need intrusive modifications
  • SSD as log-structured object store
  • Can obtain 90% raw SSD random read performance
  • Other transparent approaches deliver only 10--30%
  • Reduce write traffic by up to 32 times

Summary

25

slide-112
SLIDE 112

Thanks

Anirudh Badam abadam@cs.princeton.edu http://tinyurl.com/ssdalloc

26