VAT: Asymptotic Cost Analysis for Multi-Level Key-Value Stores Graduate Students Conference Nikos Batsaras 25 October 2019 Computer Science Department, University of Crete
Key-Value (KV) Store Application data Client • KV store : Dictionary of KV pairs. Server • API: put, get, scan, delete. • Use cases: web indexing, social networks, data analytics. (k,v) • Storage engine for many large scale KV store storage systems. kv kv kv kv · · · • Google (LevelDB) • Facebook, Yahoo! (RocksDB) Main memory • Microsoft (FASTER) kv kv kv kv · · · • Apple (FoundationDB) • . . . Persistent device nikbats@ics.forth.gr 1 of 8
Persistent Multi-Level KV Stores Definitions: • SST: Sorted String Table. • Si: Capacity of level L i . • f: Growth factor. KV · · · KV · · · L 0 S 0 SST Main memory Persistent device Compaction KV · · · KV KV · · · KV · · · L 1 S 1 = f · S 0 SST SST . . . KV · · · KV KV · · · KV KV · · · KV · · · S ℓ = f · S ℓ − 1 L ℓ SST SST SST nikbats@ics.forth.gr 2 of 8
Persistent Multi-Level KV Stores Design choices: Definitions: • What growth factor to pick ? • SST: Sorted String Table. • What should the SST size be ? • Si: Capacity of level L i . • What compaction algorithm to use ? • f: Growth factor. KV · · · KV · · · L 0 S 0 SST Main memory Persistent device Compaction KV · · · KV KV · · · KV · · · L 1 S 1 = f · S 0 SST SST . . . KV · · · KV KV · · · KV KV · · · KV · · · S ℓ = f · S ℓ − 1 L ℓ SST SST SST nikbats@ics.forth.gr 2 of 8
Problem Statement & Challenge Designer Analysis • Goal: Optimize KV-store configuration. Growth factor • Problem: Complexity of design space. 1. Many different designs that optimize for different metrics. I/O 2. New device technology further complicates Amplification the design space. • Challenge: An analysis that guides towards Compaction SST size optimal configuration. Optimal KV store nikbats@ics.forth.gr 3 of 8
Landscape I/O Amplification SSD/NVMe HDD NVM Storage Evolution nikbats@ics.forth.gr 4 of 8
Landscape RocksDB I/O Amplification Amplification with Dostoevsky technology trend Monkey PebblesDB SifrDB Atlas WiscKey HashKV Kreon ? SSD/NVMe HDD NVM Storage Evolution nikbats@ics.forth.gr 4 of 8
Landscape RocksDB I/O Amplification VAT Amplification with Dostoevsky technology trend Monkey PebblesDB SifrDB Atlas WiscKey HashKV Kreon ? SSD/NVMe HDD NVM Storage Evolution nikbats@ics.forth.gr 4 of 8
Experimental Results RocksDB approximation Kreon approximation 60 2.0 RocksDB VAT 50 SST-VAT Kreon 1.5 VAT 40 Cost Cost 30 1.0 20 0.5 10 0 0.0 2 4 8 64 2 4 8 64 Growth factor (f) Growth factor (f) BlobDB approximation PebblesDB approximation 2.0 VAT 10 BlobDB 1.5 8 Cost Cost 6 1.0 4 0.5 VAT 2 PebblesDB 0.0 0 2 4 8 64 2 4 8 64 Growth factor (f) Growth factor (f) nikbats@ics.forth.gr 5 of 8
Single Tier for Future Fast Storage Devices • Paper: Basic performance measurements of the intel optane DC persistent memory module. CoRR, abs/1903.05714, 2019. • Maximum throughput with 256-byte request size in NVM. • VAT says that NVM-based KV stores will use 1 device resident level. KV · · · KV · · · L 0 S 0 Compaction SST Main memory NVM device KV · · · KV KV · · · KV · · · L 1 Dataset SST SST nikbats@ics.forth.gr 6 of 8
Conclusions In this presentation, we talked about the VAT analysis which: 1. Expresses I/O cost for different KV-store designs. 2. Captures the technology impact on KV-store design. 3. Quantifies tradeoffs between different design decisions. 4. Guides towards optimal configuration. 5. Makes future projections based on technology trends. nikbats@ics.forth.gr 7 of 8
Questions ? Thank you. Institute of Computer Science, FORTH – Heraklion, Greece Computer Science Department, University of Crete – Heraklion, Greece • Email: nikbats@ics.forth.gr • Web: http://www.ics.forth.gr/carv nikbats@ics.forth.gr 8 of 8
Recommend
More recommend
