and Uniform Lifetime for Virtualized SSDs Jian Huang Anirudh Badam - PowerPoint PPT Presentation

FlashBlox: Achieving Both Performance Isolation and Uniform Lifetime for Virtualized SSDs Jian Huang † Anirudh Badam Laura Caulfield Bikash Sharma Moinuddin K. Qureshi † Suman Nath Sudipta Sengupta †

Flash Has Changed Over the Last Decade Performance Improvement 100 x lower latency 5,000 x higher throughput 2

Flash Has Changed Over the Last Decade Performance Increased Improvement Parallelism Dozens of 100 x lower latency 5,000 x higher throughput parallel chips 2

Flash Has Changed Over the Last Decade Performance Increased Became Improvement Parallelism Commodity Dozens of 100 x lower latency Less than $ 0.3 /GB 5,000 x higher throughput parallel chips 2

Flash Has Changed Over the Last Decade Performance Increased Became Improvement Parallelism Commodity Dozens of 100 x lower latency Less than $ 0.3 /GB 5,000 x higher throughput parallel chips Significant improvements on Flash 2

Shared Flash-Based Solid State Disk (SSD) in the Cloud ……. 3

Shared Flash-Based Solid State Disk (SSD) in the Cloud ……. SSDs are virtualized and shared in data centers 3

Performance Interference in Shared SSD ……. Write Read Flash Translation Layer Flash-based SSD: A Black Box …… …… …… …… … … … … … … Chip Chip Chip Chip Chip Chip Channel Channel Channel 4

Performance Interference in Shared SSD ……. Read/write interferences cause long ( 3 x) tail latency! Write Read Flash Translation Layer Flash-based SSD: A Black Box …… …… …… …… … … … … … … Chip Chip Chip Chip Chip Chip Channel Channel Channel 4

Performance Interference in Shared SSD ……. Write Read Flash Translation Layer …… …… …… …… … … … … … … Chip Chip Chip Chip Chip Chip Channel Channel Channel 4

Performance Interference in Shared SSD ……. Write Read Flash Translation Layer …… …… …… …… … … … … … … Chip Chip Chip Chip Chip Chip Channel Channel Channel 4

FlashBlox: Hardware Isolation in Cloud Storage ……. Flash Translation Layer …… …… …… …… … … … … … … Chip Chip Chip Chip Chip Chip Channel Channel Channel 5

FlashBlox: Hardware Isolation in Cloud Storage ……. Leveraging parallel chips for hardware isolation Flash Translation Layer …… …… …… …… … … … … … … Chip Chip Chip Chip Chip Chip Channel Channel Channel 5

Internal Parallelism Enables Hardware Isolation …… …… …… …… … … … … … … plane plane plane plane plane plane plane plane plane plane plane plane Chip Chip Chip Chip Chip Chip Channel Channel Channel 6

Internal Parallelism Enables Hardware Isolation Channel-Level Parallelism …… …… …… …… … … … … … … plane plane plane plane plane plane plane plane plane plane plane plane Chip Chip Chip Chip Chip Chip Channel Channel Channel 6

Internal Parallelism Enables Hardware Isolation Channel-Level Parallelism Chip-Level Parallelism …… …… …… …… … … … … … … plane plane plane plane plane plane plane plane plane plane plane plane Chip Chip Chip Chip Chip Chip Channel Channel Channel 6

Internal Parallelism Enables Hardware Isolation Channel-Level Parallelism Plane-level parallelism is constrained as each chip contains only one address buffer Chip-Level Parallelism Plane-Level Parallelism …… …… …… …… … … … … … … plane plane plane plane plane plane plane plane plane plane plane plane Chip Chip Chip Chip Chip Chip Channel Channel Channel 6

Internal Parallelism Enables Hardware Isolation Channel-Level Parallelism Chip-Level Parallelism Plane-Level Parallelism …… …… …… …… … … … … … … plane plane plane plane plane plane plane plane plane plane plane plane Chip Chip Chip Chip Chip Chip Channel Channel Channel Different parallelism level provides different isolation guarantee 6

New Abstractions for Hardware Isolation …… …… …… …… … … … … … … plane plane plane plane plane plane plane plane plane plane plane plane Chip Chip Chip Chip Chip Chip Channel Channel Channel 7

New Abstractions for Hardware Isolation Virtual SSD Virtual SSD Virtual SSD (Channel Level) (Chip Level) (Plane Level) High Medium Low …… …… …… …… … … … … … … plane plane plane plane plane plane plane plane plane plane plane plane Chip Chip Chip Chip Chip Chip Channel Channel Channel 7

New Abstractions for Hardware Isolation Virtual SSD Virtual SSD Virtual SSD (Channel Level) (Chip Level) (Plane Level) Software-based High Medium Low …… …… …… …… … … … … … … plane plane plane plane plane plane plane plane plane plane plane plane Chip Chip Chip Chip Chip Chip Channel Channel Channel 7

Hardware Isolation Meets the Pay-As-You-Go Model in Cloud vSSD (Channel) vSSD (Chip) vSSD (Software) …… …… …… …… … … … … … … plane plane plane plane plane plane plane plane plane plane plane plane Chip Chip Chip Chip Chip Chip Channel Channel Channel 8

Hardware Isolation Meets the Pay-As-You-Go Model in Cloud Azure Azure Amazon DocumentDB SQL Database DynamoDB vSSD (Channel) vSSD (Chip) vSSD (Software) …… …… …… …… … … … … … … plane plane plane plane plane plane plane plane plane plane plane plane Chip Chip Chip Chip Chip Chip Channel Channel Channel 8

Hardware Isolation Meets the Pay-As-You-Go Model in Cloud Throughput Single Partition Size Azure Azure Amazon DocumentDB SQL Database DynamoDB Price vSSD (Channel) vSSD (Chip) vSSD (Software) …… …… …… …… … … … … … … plane plane plane plane plane plane plane plane plane plane plane plane Chip Chip Chip Chip Chip Chip Channel Channel Channel 8

Hardware Isolation Meets the Pay-As-You-Go Model in Cloud Hundreds of vSSDs can be supported in a single server vSSD (Channel) vSSD (Chip) vSSD (Software) …… …… …… …… … … … … … … plane plane plane plane plane plane plane plane plane plane plane plane Chip Chip Chip Chip Chip Chip Channel Channel Channel 8

Impact of Hardware Isolation on SSD Lifetime ……. Flash Translation Layer …… …… …… …… … … … … … … Chip Chip Chip Chip Chip Chip Channel Channel Channel 9

Impact of Hardware Isolation on SSD Lifetime The average rate at which flash blocks are erased ……. 4.5 4 3.5 #Blocks Erased/sec 3 Average 2.5 2 1.5 Flash Translation Layer 1 0.5 0 …… …… …… …… … … … … … … Chip Chip Chip Chip Chip Chip Channel Channel Channel 9

Impact of Hardware Isolation on SSD Lifetime The average rate at which flash blocks are erased ……. 4.5 4 3.5 #Blocks Erased/sec 3 Average 2.5 2 1.5 Flash Translation Layer 1 0.5 0 …… …… …… …… … … … … … … Chip Chip Chip Chip Chip Chip Channel Channel Channel 9

Impact of Hardware Isolation on SSD Lifetime The average rate at which flash blocks are erased ……. 4.5 4 3.5 #Blocks Erased/sec 3 Average 2.5 2 1.5 Flash Translation Layer 1 0.5 0 …… …… …… …… … … … … … … Flash blocks wear out at different rate with different workload Chip Chip Chip Chip Chip Chip Channel Channel Channel 9

Impact of Hardware Isolation on SSD Lifetime ……. Write Intensive Flash Translation Layer …… …… …… …… … … … … … … Chip Chip Chip Chip Chip Chip Channel Channel Channel 9

App App App … … … Chip Chip Chip FlashBlox SSD Lifetime Performance Isolation Challenges 10

App App App … … … Chip Chip Chip FlashBlox SSD Lifetime Performance Isolation Challenges App App App … … … Chip Chip Chip SSD Lifetime Performance Isolation 10

App App App FlashBlox … … … Challenges Chip Chip Chip SSD Lifetime SSD Lifetime Performance Isolation 10

FlashBlox: Swapping Channels for Wear Balance Used Erase Cycles Channel 1 Channel 2 Channel 3 Channel 4 Adjusting the wear imbalance at a more coarse time granularity can achieve near-ideal SSD lifetime 11

FlashBlox: Swapping Channels for Wear Balance Used Erase Cycles Channel 1 Channel 2 Channel 3 Channel 4 The channel that has incurred the maximum wearout The channel that has the minimum rate of wearout 11

FlashBlox: Swapping Channels for Wear Balance Used Erase Cycles Channel 1 Channel 2 Channel 3 Channel 4 Channel migration takes 15 minutes, once per 19 days Overall performance drops only for 0.04% of all the time 11

How Frequently Should We Swap ? Imbalance = MaxWear / AvgWear Used Erase Cycles Channel 1 Channel 2 Channel 3 Channel 4 12

How Frequently Should We Swap ? Imbalance = MaxWear / AvgWear 4 Used Erase Cycles App M Channel 1 Channel 2 Channel 3 Channel 4 12