Unified Address Translation for Memory Mapped SSDs with FlashMap - - PowerPoint PPT Presentation
Unified Address Translation for Memory Mapped SSDs with FlashMap Jian Huang Anirudh Badam Moinuddin K. Qureshi Karsten Schwan Bridging the DRAM-Disk Gap DRAM Disk Low Performance High Performance Large Capacity Small Capacity
Jian Huang
Anirudh Badam† Moinuddin K. Qureshi Karsten Schwan †
2
DRAM High Performance Small Capacity Application Memory Component Disk Low Performance Large Capacity Application Storage Component
2
DRAM High Performance Small Capacity Application Memory Component Disk Low Performance Large Capacity Application Storage Component SSD Good Performance Good Capacity
3
Flash behaves more like memory than disk
3
Flash behaves more like memory than disk
3
Flash behaves more like memory than disk
No Seek Latency
3
Flash behaves more like memory than disk
No Seek Latency + Internal Parallelism
3
Flash behaves more like memory than disk
No Seek Latency + Internal Parallelism + High IOPS
3
Flash behaves more like memory than disk
No Seek Latency + Internal Parallelism + High IOPS Use Flash as Memory [Badam et al., NSDI’11]
DRAM High Performance Small Capacity Application Memory Component SSD Good Performance Good Capacity Disk Low Performance Large Capacity Application Storage Component
3
Flash behaves more like memory than disk
No Seek Latency + Internal Parallelism + High IOPS Use Flash as Memory [Badam et al., NSDI’11]
DRAM High Performance Small Capacity Application Memory Component SSD Good Performance Good Capacity Disk Low Performance Large Capacity Application Storage Component
4
Application SSD
4
Application SSD Virtual Memory File Filesystem
4
Application SSD
mmap() munmap() msync()
Virtual Memory File Filesystem
4
Application SSD
mmap() munmap() msync()
Virtual Memory File Filesystem Extended Memory
YAY! MORE MEM!!!
4
Minimal Code Modifications
Application SSD
mmap() munmap() msync()
Virtual Memory File Filesystem Extended Memory
YAY! MORE MEM!!!
4
Minimal Code Modifications
Application SSD
mmap() munmap() msync()
Virtual Memory File Filesystem Extended Memory
YAY! MORE MEM!!!
Data Durability
5
Virtual Memory System Application Flash Translation Layer Flash File System
5
Virtual Memory System Application Flash Translation Layer Flash File System
Page Table & Memory Manager
Virtual Address
Physical Address/File Offset
page fault
5
Virtual Memory System Application Flash Translation Layer Flash File System
Page Table & Memory Manager
Virtual Address
Physical Address/File Offset
File Index File Offset Logical Block Address
page fault
5
Virtual Memory System Application Flash Translation Layer Flash File System
Page Table & Memory Manager
Virtual Address
Physical Address/File Offset
File Index File Offset Logical Block Address FTL Logical Block Address Physical Block Address
page fault
6
Virtual Memory System Application Flash Translation Layer Flash File System 3 address translations + 2 boundary checks + 2 permission checks
+ Increased Metadata Overhead
7
Application Flash
Reduced Storage, only 1 mapping table Reduced Latency, only 1 address translation + 1 permission check +1 boundary check
Virtual Memory System Flash Translation Layer File System
7
Application Flash
Reduced Storage, only 1 mapping table Reduced Latency, only 1 address translation + 1 permission check +1 boundary check
Unified Address Translation
8
Flash Translation Layer Flash Process A File System File
8
Flash Translation Layer Flash Process A File System File VM Region
8
Flash Translation Layer Flash Process A File System File VM Region Page Table
PGD PUD PMD PTE Offset
…
8
Flash Translation Layer Flash Process A File System File VM Region Page Table
PGD PUD PMD PTE Offset
…
Process-specific, private
8
Flash Translation Layer Flash Process A File System File VM Region Page Table
PGD PUD PMD PTE Offset
…
Only for mapped file
9
Flash Translation Layer Flash Process A File System Mapped File VM Region Shared Page Table Process B VM Region
PGD PUD PMD PTE Offset
…
9
Flash Translation Layer Flash Process A File System Mapped File VM Region Shared Page Table Process B VM Region
READ_ONLY READ_WRITE
PGD PUD PMD PTE Offset
…
9
Flash Translation Layer Flash Process A File System Mapped File VM Region Shared Page Table Process B VM Region
READ_ONLY READ_WRITE
Permission Conflict !!!
PGD PUD PMD PTE Offset
…
9
Flash Translation Layer Flash Process A File System Mapped File VM Region Shared Page Table Process B VM Region
READ_ONLY READ_WRITE
Only share the leaf-level page table pages !
PGD PUD PMD PTE Offset
…
9
Flash Translation Layer Flash Process A File System Mapped File VM Region Shared Page Table Process B VM Region
READ_ONLY READ_WRITE
Only share the leaf-level page table pages !
PGD PUD PMD PTE Offset
…
9
Flash Translation Layer Flash File System Mapped File Process A
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
Process B
PGD PUD PMD PTE Offset
…
9
Flash Translation Layer Flash File System Mapped File Process A
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
Process B
PGD PUD PMD PTE Offset
… Private Private Shared
10
Before Mapping a File Process’s Private Virtual Memory + File Backed Memo
File Process Page Directory Private Virtual Memory Regions Private Leaf-Level Page Table Pages Virtual Memory Regions Backed by File Shared Leaf-Level Page Table Pages
10
After Mapping a File Process’s Private Virtual Memory + File Backed Memory
File Process Page Directory Private Virtual Memory Regions Private Leaf-Level Page Table Pages Virtual Memory Regions Backed by File Shared Leaf-Level Page Table Pages Only for mapped file
11
Flash Translation Layer Flash Process A File System Mapped File
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
Process B
PGD PUD PMD PTE Offset
…
11
Flash Translation Layer Flash Process A File System Mapped File
Shared Leaf-level Page Table Pages What if I require custom memory protection for a single page ???
PGD PUD PMD PTE Offset
Process B
PGD PUD PMD PTE Offset
…
11
Flash Translation Layer Flash Process A File System Mapped File
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
Process B
PGD PUD PMD PTE Offset
Private Leaf-level Page Table
…
12
Flash Translation Layer Flash Process A Mapped File
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
Process B
PGD PUD PMD PTE Offset
Mapping Table GC ECC Wear Leveling
…
12
Flash Translation Layer Flash Process A Mapped File
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
Process B
PGD PUD PMD PTE Offset
GC ECC Wear Leveling
Overloaded PTE …
12
Flash Translation Layer Flash Process A Mapped File
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
Process B
PGD PUD PMD PTE Offset
GC ECC Wear Leveling
Overloaded PTE …
13
Flash Process A Mapped File
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
Read DRAM …
FlashMap
13
Flash Process A Mapped File
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
Read DRAM …
FlashMap
13
Flash Process A Mapped File
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
Read DRAM … page fault
FlashMap
13
Flash Process A Mapped File
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
Read DRAM … update PTE
FlashMap
13
Flash Process A Mapped File
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
DRAM Write …
FlashMap
13
Flash Process A Mapped File
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
DRAM Write … DRAM hit
FlashMap
13
Flash Process A Mapped File
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
DRAM Write … DRAM miss
FlashMap
13
Flash Process A Mapped File
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
DRAM Write … update PTE
FlashMap
13
Flash Process A Mapped File
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
DRAM Write …
FlashMap
13
Flash Process A Mapped File
Shared Leaf-level Page Table Pages
PGD PUD PMD PTE Offset
DRAM GC …
FlashMap
14
FlashMap: Implementation in Real System
SSD Emulator Real SSD File System (EXT4) Mapped File … File Index RAMDisk Linux Memory Manager
14
FlashMap: Implementation in Real System
SSD Emulator Real SSD File System (EXT4) Mapped File … File Index RAMDisk Linux Memory Manager
14
FlashMap: Implementation in Real System
SSD Emulator Real SSD File System (EXT4) Mapped File … File Index RAMDisk Linux Memory Manager
14
FlashMap: Implementation in Real System
SSD Emulator Real SSD File System (EXT4) Mapped File … File Index RAMDisk Linux Memory Manager
15
Baseline
unmodified Linux: mmap + EXT4 + FTL with page-level mapping
FTL+FS★
mmap + combined FTL & file system
FlashMap
unified address translation
Intel Xeon processors + 64 GB DRAM + 2 TB SSD
★similar to Nameless Writes [Zhang et al., FAST’12] and
DFS [Josephson et al., FAST’10]
16
NoSQL Store SQL Database
Shore-MT TPCC, TPCB, TATP
Graph Analytics
YCSB
PageRank
17
2 4 6 8 10 12 14 Metadata Size (GB) Baseline FTL+FS FlashMap
50%
1 2 3 4 16 32 64 128 256 Speedup SSD size : DRAM size
Baseline FTL+FS FlashMap
18
Benefits from Reduced Mapping Overhead
FlashMap: 1.7x performance improvement over FTL+FS
1 2 3 4 16 32 64 128 256 Speedup SSD size : DRAM size
Baseline FTL+FS FlashMap
18
Benefits from Reduced Mapping Overhead
FlashMap: 1.7x performance improvement over FTL+FS
1 2 3 4 16 32 64 128 256 Speedup SSD size : DRAM size
Baseline FTL+FS FlashMap
18
Benefits from Reduced Mapping Overhead
25 50 75 100
16 32 64 128 256
DRAM Hit Rate (%)
Reducing the mapping overhead improves the DRAM caching efficiency
19
5 10 15 20 25 30 35 40 45 50 5 10 15 20 25 Latency (us) Device latency (microseconds)
Baseline FTL+FS FlashMap
Benefit (up to 53% latency reduction) mainly comes from the combination of page table and file system
20
5 10 15 20 25 5 10 15 20 25 Throughput (K TPS) Device Latency (us)
Baseline FTL+FS FlashMap FlashMap: 1.8x more TPS than baseline and FTL+FS
21
Unified Address Translation Application Flash as Memory
Reduced Storage
3.3x performance improvement for data-intensive applications
Reduced Latency
53% latency reduction for high-end SSDs, 1.8x more TPS for latency-sensitive applications, e.g., database systems
Jian Huang jian.huang@gatech.edu
Anirudh Badam† Moinuddin K. Qureshi Karsten Schwan
†