SLIDE 1
1
2
Is Virtual Memory Relevant?
- There is never enough DRAM
– Price, power and DIMM slots limit amount – Application memory footprints are ever‐increasing
- VM is no longer DRAM+Disk
FlashVM: Virtual Memory Management on Flash Mohit Saxena Michael M. - - PowerPoint PPT Presentation
FlashVM: Virtual Memory Management on Flash Mohit Saxena Michael M. - - PowerPoint PPT Presentation
FlashVM: Virtual Memory Management on Flash Mohit Saxena Michael M. Swift University of Wisconsin Madison 1 Is Virtual Memory Relevant? There is never enough DRAM Price, power and DIMM slots limit amount Application memory
SLIDE 2
SLIDE 3
3
Flash and Virtual Memory
DRAM
DRAM is expensive Disk is slow Flash is cheap and fast Flash for Virtual Memory
Storage + VM Disk VM Storage Flash
SLIDE 4
4
In this talk
- Flash for Virtual Memory
– Does it improve system price/performance? – What OS changes are required?
- FlashVM
– System architecture using dedicated flash for VM – Extension to core VM subsystem in the Linux kernel – Improved performance, reliability and garbage collection
SLIDE 5
5
Outline
- Introduction
- Background
– Flash and VM
- Design
- Evaluation
- Conclusions
SLIDE 6
6
Flash 101
- Flash is not disk
– Faster random access performance: 0.1 vs. 2‐3 ms for disk – No in‐place modify: write only to erased location
- Flash blocks wear out
– Erasures limited to 10,000‐100,000 per block – Reliability dropping with increasing MLC flash density
- Flash devices age
– Log‐structured writes leave few clean blocks after extensive use – Performance drops by up to 85% on some SSDs – Requires garbage collection of free blocks
SLIDE 7
7
Virtual Memory 101
DiskVM FlashVM
Memory Size M Execution Time T
M
Reduced DRAM Same performance Lower system price
T
Faster execution No additional DRAM Similar system price
No Locality Unused Memory
SLIDE 8
8
Outline
- Introduction
- Background
- Design
– Performance – Reliability – Garbage Collection
- Evaluation
- Conclusions
SLIDE 9
9
FlashVM Hierarchy
Block Device Driver Page Swapping DRAM
FlashVM Manager: Performance, Reliability and Garbage Collection
Dedicated Flash Cost‐effective for VM Reduced FS interference
Disk
Disk Scheduler Block Layer
VM Memory Manager
Dedicated Flash MLC NAND
SLIDE 10
10
VM Performance
- Challenge
– VM systems optimized for disk performance – Slow random reads, high access and seek costs, symmetrical read/write performance
- FlashVM de‐diskifies VM:
– Page write back – Page scanning – Disk scheduling – Page prefetching
Parameter Tuning
SLIDE 11
11
Page Prefetching
Linux swap map FREE FREE BAD FlashVM VM assumption
Seek and rotational delays are longer than the transfer cost of extra blocks
Linux sequential prefetching
Minimize costly disk seeks Delimited by free and bad blocks
FlashVM prefetching
Exploit fast flash random reads and spatial locality in reference pattern Seek over free and bad blocks
Request
SLIDE 12
12
Stride Prefetching
- FlashVM uses stride
prefetching
– Exploit temporal locality in the reference pattern – Exploit cheap seeks for fast random access – Fetch two extra blocks in the stride
Request Request Request Request Request
swap map
SLIDE 13
The Reliability Problem
- Challenge: Reduce the number of writes
– Flash chips lose durability after 10,000 – 100,000 writes – Actual write‐lifetime can be two orders of magnitude less – Past solutions:
- Disk‐based write caches for streamed I/O
- De‐duplication and compression for storage
- FlashVM uses knowledge of page content and state
– Dirty Page sampling – Zero Page sharing
SLIDE 14
14
Page Sampling
Dirty? Dirty Clean
Inactive LRU Page List
free_pages sample 1‐sR Disk Linux Write‐back all evicted dirty pages Flash FlashVM Prioritize young clean
- ver old dirty pages
Free Page List
sR
SLIDE 15
15
Adaptive Sampling
- Challenge: Reference pattern variations
– Write‐mostly: Many dirty pages – Read‐mostly: Many clean pages
- FlashVM adapts sampling rate
– Maintain a moving average for the write rate – Low write rate Increase sR
- Aggressively skip dirty pages
– High write rate Converge to native Linux
- Evict dirty pages to relieve memory pressure
SLIDE 16
16
Outline
- Introduction
- Why FlashVM?
- Design
– Performance – Reliability – Garbage Collection
- Evaluation
- Conclusions
SLIDE 17
17
write ‘cluster A’ at block 0 write ‘cluster B’ at block 100
Flash Cleaning
- All writes to flash go to
a new location
- Discard command
notifies SSD that blocks are unused
- Benefits:
– More free blocks for writing – Avoids copying data for partial over‐writes
free ‘cluster A’ & discard write ‘cluster D’ at block 0 write ‘cluster A’ at block 0 free ‘cluster A’ write ‘cluster B’ at block 100 free ‘cluster B’ write ‘cluster C’ at block 200
SLIDE 18
18
100 200 300 400 500 read 4KB write 4KB erase 128KB discard 4KB discard 1M B discard 10M B discard 100M B discard 1GB
Operation
Latency (ms)
Discard is Expensive
OCZ‐Vertex, Indilinx controller
Operation Latency
< 0.5 ms 2 ms 55 ms 417 ms
SLIDE 19
19
Discard and VM
- Native Linux VM has limited discard support
– Invokes discard before reusing free page clusters – Pays high fixed cost for small sets of pages
- FlashVM optimizes to reduce discard cost
– Avoid unnecessary discards: dummy discard – Discard larger sizes to amortize cost: merged discard
SLIDE 20
20
Dummy Discard
- Observation: Overwriting a
block
– notifies SSD it is empty – after discarding it, uses the free space made available by discard
- FlashVM implements dummy
discard
– Monitors rate of allocation – Virtualize discard by reusing blocks likely to be overwritten soon
Overwrite Discard Overwrite
SLIDE 21
21
Merged Discard
- Native Linux invokes
discard once per page cluster
– Result: 55 ms latency for freeing 32 pages (128K)
- FlashVM batch many
free pages
– Defer discard until 100 MB of free pages available – Pages discarded may be non‐contiguous
Discard Discard Discard Discard
SLIDE 22
22
Design Summary
- Performance improvements
– Parameter Tuning: page write back, page scanning, disk scheduling – Improved/stride prefetching
- Reliability improvements
– Reduced writes: page sampling and sharing
- Garbage collection improvements
– Merged and Dummy discard
SLIDE 23
23
Outline
- Introduction
- Motivation
- Design
- Evaluation
– Performance and memory savings – Reliability and garbage collection
- Conclusions
SLIDE 24
24
Methodology
- System and Devices
– 2.5 GHz Intel Core 2 Quad, Linux 2.6.28 kernel – IBM, Intel X‐25M, OCZ‐Vertex trim‐capable SSDs
- Application Workloads
– ImageMagick ‐ resizing a large JPEG image by 500% – Spin – model checking for 10 million states – SpecJBB – 16 concurrent warehouses – memcached server – key‐value store for 1 million keys
SLIDE 25
25
Application Performance and Memory Savings
10 20 30 40 50 60 70 ImageMagick Spin SpecJBB memcached- store memcached- lookup
Applications
Performance/Memory Savings Runtime Memory Use
Const Memory 94% less execution time Const Performance 84% memory savings
SLIDE 26
26
Write Reduction
20 40 60 80 100 120 Uniform Page Sampling Adaptive Page Sampling Page Sharing
Write Reduction Technique
Performance/Writes Performance Writes
7% overhead, 12% reduction 14% reduction 93% reduction
ImageMagick Spin
SLIDE 27
27
Garbage Collection
1 10 100 1000 10000 ImageMagick Spin memcached
Application
Elapsed Time (s) Linux/Discard FlashVM Linux/No Discard
10X faster 15% slower
SLIDE 28
28
Conclusions
- FlashVM: Virtual Memory Management on
Flash
– Dedicated flash for paging – Improved performance, reliability and garbage collection
- More opportunities and challenges for OS
design
– Scaling FlashVM to massive memory capacities (terabytes!) – Future memory technologies: PCM and Memristors
SLIDE 29
29