rocksteady fast migration for low latency in memory
play

Rocksteady: Fast Migration for Low-Latency In-memory Storage Chinmay - PowerPoint PPT Presentation

Oct 28, 2022 •242 likes •757 views

Rocksteady: Fast Migration for Low-Latency In-memory Storage Chinmay Kulkarni , Aniraj Kesavan, Tian Zhang, Robert Ricci, Ryan Stutsman 1 Introduction Distributed low-latency in-memory key-value stores are emerging Predictable response

  1. Rocksteady: Fast Migration for Low-Latency In-memory Storage Chinmay Kulkarni , Aniraj Kesavan, Tian Zhang, Robert Ricci, Ryan Stutsman 1

  2. Introduction • Distributed low-latency in-memory key-value stores are emerging • Predictable response times ~ 10 µs median, ~60 µs 99.9 th -tile • Problem: Must migrate data between servers • Minimize performance impact of migration → go slow? • Quickly respond to hot spots, skew shifts, load spikes → go fast? • Solution: Fast data migration with low impact • Early ownership transfer of data, leverage workload skew • Low priority, parallel and adaptive migration • Result: Migration protocol for RAMCloud in-memory key-value store • Migrates 256 GB in 6 minutes, 99.9 th -tile latency less than 250 µs • Median latency recovers from 40 µs to 20 µs in 14 s 2

  3. Why Migrate Data? Client 1 Client 2 multiGet( ) multiGet( ) A B C D 6 Million No Locality Fanout=7 A C B D Server 1 Server 2 Poor spatial locality → High multiGet() fan-out → More RPCs 3

  4. Migrate To Improve Spatial Locality Client 1 Client 2 multiGet( ) multiGet( ) A B C D 6 Million No Locality Fanout=7 B A C B D Server 1 Server 2 C 4

  5. Spatial Locality Improves Throughput Client 1 Client 2 25 Million multiGet( ) multiGet( ) A B C D 6 Million Full Locality No Locality Fanout=1 Fanout=7 A B C D Server 1 Server 2 Better spatial locality → Fewer RPCs → Higher throughput Benefits multiGet(), range scans 5

  6. The RAMCloud Key-Value Store Client Client Client Client All Data in RAM Kernel Bypass/ DPDK Data Center Fabric Coordinator 10 µs reads Master Master Master Master Backup Backup Backup Backup 6

  7. The RAMCloud Key-Value Store Client Client Client Client Write RPC Data Center Fabric Coordinator Master Master Master Master Backup Backup Backup Backup 7

  8. The RAMCloud Key-Value Store Client Client Client Client Write RPC 1x in DRAM Data Center Fabric Coordinator 3x on Disk Master Master Master Master Backup Backup Backup Backup 8

  9. Fault-toler eranc nce & e & Rec ecover ery I In n RAM AMCl Cloud ud Client Client Client Client Data Center Fabric Coordinator Master Master Master Master Backup Backup Backup Backup 9

  10. Fault-toler eranc nce & e & Rec ecover ery I In n RAM AMCl Cloud ud Client Client Client Client 2 seconds to Data Center Fabric Coordinator recover Master Master Master Master Backup Backup Backup Backup 10

  11. Performance Goals For Migration • Maintain low access latency • 10 µsec median latency → System extremely sensitive • Tail latency matters at scale → Even more sensitive • Migrate data fast • Workloads dynamic → Respond quickly • Growing DRAM storage: 512 GB per server • Slow data migration → Entire day to scale cluster 11

  12. Rocksteady Overview: Early Ownership Transfer Problem: Loaded source can bottleneck migration Solution: Instantly shift ownership and all load to target Client 1 Client 2 Client 3 Client 4 Reads and Writes Source Server Target Server 12

  13. Rocksteady Overview: Early Ownership Transfer Problem: Loaded source can bottleneck migration Solution: Instantly shift ownership and all load to target Client 1 Client 2 Client 3 Client 4 Instantly All future operations Redirected serviced at Target Creates “headroom” to speed migration Source Server Target Server 13

  14. Rocksteady Overview: Leverage Skew Problem: Data has not arrived at source yet Solution: On demand migration of unavailable data Client 1 Client 2 Client 3 Client 4 Read On-demand Pull Source Server Target Server 14

  15. Rocksteady Overview: Leverage Skew Problem: Data has not arrived at source yet Solution: On demand migration of unavailable data Client 1 Client 2 Client 3 Client 4 Read Hot keys move early Median Latency recovers to 20 µs in 14 s Source Server Target Server 15

  16. Rocksteady Overview: Adaptive and Parallel Problem: Old single-threaded protocol limited to 130 MB/s Solution: Pipelined and parallel at source and target Client 1 Client 2 Client 3 Client 4 On-demand Pull Parallel Pulls Source Server Target Server 16

  17. Rocksteady Overview: Adaptive and Parallel Problem: Old single-threaded protocol limited to 130 MB/s Solution: Pipelined and parallel at source and target Client 1 Client 2 Client 3 Client 4 On-demand Pull Target Driven Yields to On-demand Pulls Parallel Pulls Moves 758 MB/s Source Server Target Server 17

  18. Rocksteady Overview: Eliminate Sync Replication Problem: Synchronous replication bottleneck at target Solution: Safely defer replication until after migration Client 1 Client 2 Client 3 Client 4 Replication On-demand Pull Parallel Pulls Source Server Target Server 18

  19. Rocksteady Overview: Eliminate Sync Replication Problem: Synchronous replication bottleneck at target Solution: Safely defer replication until after migration Client 1 Client 2 Client 3 Client 4 Replication Source Server Target Server 19

  20. Rocksteady: Putting it all together • Instantaneous ownership transfer • Immediate load reduction at overloaded source • Creates “headroom” for migration work • Leverage skew to rapidly migrate hot data • Target comes up to speed with little data movement • Adaptive parallel, pipelined at source and target • All cores avoid stalls, but yield to client-facing operations • Safely defer replication at target • Eliminates replication bottleneck and contention 20

  21. Rocksteady • Instantaneous ownership transfer • Leverage skew to rapidly migrate hot data • Adaptive parallel, pipelined at source and target • Safely defer synchronous replication at target 21

  22. Evaluation Setup Client Client Client Client YCSB-B (95/5) YCSB-B (95/5) YCSB-B (95/5) YCSB-B (95/5) Skew=0.99 Skew=0.99 Skew=0.99 Skew=0.99 300 Million Records 45 GB Source Server Target Server 22

  23. Evaluation Setup Client Client Client Client YCSB-B (95/5) YCSB-B (95/5) YCSB-B (95/5) YCSB-B (95/5) Skew=0.99 Skew=0.99 Skew=0.99 Skew=0.99 150 Million Records 150 Million 22.5 GB Records 150 Million 22.5 GB Records 22.5 GB Target Server 23

  24. Instantaneous Ownership Transfer Source CPU Utilization 80% Created 55% Source CPU Headroom 25% Before Ownership Immediately After Transfer Transfer Before migration: Source over-loaded, Target under-loaded Ownership transfer creates Source headroom for migration 24

  25. Rocksteady • Instantaneous ownership transfer • Leverage skew to rapidly migrate hot data • Adaptive parallel, pipelined at source and target • Safely defer synchronous replication at target 25

  26. Leverage Skew To Move Hot Data Before Migration: 99.9th Latency Median Latency 245µs 240µs Median=10 µs 99.9 th = 60 µs 155µs 75µs 28µs 17µs Uniform (Low) Skew=0.99 Skew=1.5 (High) After ownership transfer, hot keys pulled on-demand More skew → Median restored faster (migrate fewer hot keys) 26

  27. Rocksteady • Instantaneous ownership transfer • Leverage skew to rapidly migrate hot data • Adaptive parallel, pipelined at source and target • Safely defer synchronous replication at target 27

  28. Parallel, Pipelined, & Adaptive Pulls 0 8 16 24 Target Hash Table Per-Core Buffers Worker Cores replay replay read(B) Dispatch Migration Core Manager Pull Buffers NIC Polling pulling • Target driven, migration manager • Co-partitioned hash tables, pull from partitions in parallel • Replay pulled data into per-core buffers 28

  29. Parallel, Pipelined, & Adaptive Pulls 0 8 16 24 Source Hash Table Copy Addresses read(A) pull(11) pull(17) Dispatch Core Gather Gather List List NIC Polling • Stateless passive Source • Granular 20 KB pulls 29

  30. Parallel, Pipelined, & Adaptive Pulls • Redirect any idle CPU for migration • Migration yields to regular requests, on-demand pulls 30

  31. Rocksteady • Instantaneous ownership transfer • Leverage skew to rapidly migrate hot data • Adaptive parallel, pipelined at source and target • Safely defer synchronous replication at target 31

  32. Naïve Fault Tolerance During Migration Each server has a recovery log distributed across the cluster Source Target A A B C Backup Backup Backup Backup Backup Source C A B C A A C B B Recovery Log Target Recovery Log 32

  33. Naïve Fault Tolerance During Migration Migrated data needs to be triplicated to target’s recovery log Source Target A B C A Backup Backup Backup Backup Backup Source C A B C A A C B B Recovery Log Target Recovery Log 33

  34. Naïve Fault Tolerance During Migration Migrated data needs to be triplicated to target’s recovery log Source Target A B C A Backup Backup Backup Backup Backup Source C A B C A A C B B Recovery Log Target A A A Recovery Log 34

  35. Synchronous Replication Bottlenecks Migration Synchronous replication hits migration speed by 34% Source Target A B C A B Backup Backup Backup Backup Backup Source C A B C A A C B B Recovery Log Target B A B A B A Recovery Log 35

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Low-Latency Communication for Fast DBMS Using RDMA and Shared Memory Philipp Fent, Alexander van

Low-Latency Communication for Fast DBMS Using RDMA and Shared Memory Philipp Fent, Alexander van

Low-Latency Communication for Fast DBMS Using RDMA and Shared Memory Philipp Fent, Alexander van Renen, Andreas Kipf, Technische Universitt Mnchen, April 23, 2020 Viktor Leis , Thomas Neumann, Alfons Kemper

857 views • 23 slides
Why memory hierarchy (3 rd Ed: p.468-487, 4 th Ed: p. 452-470) users want unlimited fast

Why memory hierarchy (3 rd Ed: p.468-487, 4 th Ed: p. 452-470) users want unlimited fast

Why memory hierarchy (3 rd Ed: p.468-487, 4 th Ed: p. 452-470) users want unlimited fast memory fast memory expensive, slow memory cheap cache: small, fast memory near CPU large, slow memory (main memory, disk, )

387 views • 23 slides
CAME AME L Lab ab Emerging Non-Volatile Memory for SSDs 450 us Latency (reads) 150 us 25

CAME AME L Lab ab Emerging Non-Volatile Memory for SSDs 450 us Latency (reads) 150 us 25

ATC 2020 Fully Hardware Automated Open Research Framework for Future Fast NVMe Device Myoungsoo Jung Computer Architecture and Memory systems Laboratory Sponsored by CAME AME L Lab ab Emerging Non-Volatile Memory for SSDs 450 us

538 views • 49 slides
Main Memory Moving further away from the CPU .. 95 Main Memory Performance measurement

Main Memory Moving further away from the CPU .. 95 Main Memory Performance measurement

Main Memory Moving further away from the CPU .. 95 Main Memory Performance measurement Latency - cache miss penalty Bandwidth - large block sizes of L2 argue for B/W Memory latency Access time : Time between when a read is

652 views • 14 slides
Cache Management Improving Memory Locality and Reducing Memory Latency Introduction Memory

Cache Management Improving Memory Locality and Reducing Memory Latency Introduction Memory

Cache Management Improving Memory Locality and Reducing Memory Latency Introduction Memory system performance is critical in modern architectures DO I = 1, M Accessing memory takes much longer than DO J = 1, N accessing cache

364 views • 19 slides
Fast Track DC Migration with Cloud 3.0 Reap the benefits of the local cloud and Fast-Track Data

Fast Track DC Migration with Cloud 3.0 Reap the benefits of the local cloud and Fast-Track Data

MINT MANAGEMENT TECHNOLOGIES Fast Track DC Migration with Cloud 3.0 Reap the benefits of the local cloud and Fast-Track Data Center Migration with Cloud 3.0 Challenges Ideal Solution Desired Outcomes This public cloud infrastructure is

341 views • 5 slides
What makes a fast processor? 1. Instructions required per program ISA design: RISC vs. CISC

What makes a fast processor? 1. Instructions required per program ISA design: RISC vs. CISC

What makes a fast processor? 1. Instructions required per program ISA design: RISC vs. CISC 2. Memory bandwidth and latency Memory hierarchy Cache parameterisation 3. Instructions executed per second Internal CPU

260 views • 22 slides
Memory Management Ideally programmers want memory that is large fast non

Memory Management Ideally programmers want memory that is large fast non

Memory Management Virtual Memory Background; key issues Memory allocation schemes Virtual memory Memory management design and implementation issues 1 Memory Management Ideally programmers want memory that is large fast non

410 views • 30 slides
VERSION GALORE: ROCKSTEADY AND REGGAE FROM JA TO L.A. Nina L. Cole Postdoctoral Scholar

VERSION GALORE: ROCKSTEADY AND REGGAE FROM JA TO L.A. Nina L. Cole Postdoctoral Scholar

VERSION GALORE: ROCKSTEADY AND REGGAE FROM JA TO L.A. Nina L. Cole Postdoctoral Scholar Department of Gender, Sexuality, and Womens Studies University of California, Davis Version/Versioning Version: Another cut of a popular rhythm.

147 views • 10 slides
LECTURE 12 Virtual Memory VIRTUAL MEMORY Just as a cache can provide fast, easy access to

LECTURE 12 Virtual Memory VIRTUAL MEMORY Just as a cache can provide fast, easy access to

LECTURE 12 Virtual Memory VIRTUAL MEMORY Just as a cache can provide fast, easy access to recently-used code and data, main memory acts as a cache for magnetic disk. The mechanism by which this is accomplished is known as virtual memory

974 views • 34 slides
Memory Refresh Kate Nguyen, Kehan Lyu, Xianze Meng , Vilas Sridharan, Xun Jian History of DRAM 2

Memory Refresh Kate Nguyen, Kehan Lyu, Xianze Meng , Vilas Sridharan, Xun Jian History of DRAM 2

Nonblocking Memory Refresh Kate Nguyen, Kehan Lyu, Xianze Meng , Vilas Sridharan, Xun Jian History of DRAM 2 Refresh Latency Bus Cycle Time Min. Read Latency 550 512 Latency (ns) 16 13.5 0.75 0.5 2014 2007 2000 2003 1968 2018

703 views • 22 slides
Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads Sadjad

Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads Sadjad

Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads Sadjad Fouladi , Riad S. Wahby , Brennan Shacklett , Karthikeyan Vasuki Balasubramaniam , William Zeng , Rahul Bhalerao , Anirudh Sivaraman

767 views • 27 slides
Systems I The Memory Hierarchy Topics Topics Storage technologies Capacity and latency

Systems I The Memory Hierarchy Topics Topics Storage technologies Capacity and latency

Systems I The Memory Hierarchy Topics Topics Storage technologies Capacity and latency trends The hierarchy Random-Access Memory (RAM) Key features Key features RAM is packaged as a chip. Basic storage unit is a cell (one

370 views • 34 slides
Controller Architecture for Low-latency Access to Phase-Change Memory in OpenPOWER Systems A.

Controller Architecture for Low-latency Access to Phase-Change Memory in OpenPOWER Systems A.

Controller Architecture for Low-latency Access to Phase-Change Memory in OpenPOWER Systems A. Prodromakis 1 , N. Papandreou 2 , E. Bougioukou 1 , U. Egger 2 , N. Toulgaridis 1 , T. Antonakopoulos 1 , H. Pozidis 2 , E. Eleftheriou 2 1 University of

459 views • 7 slides
Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads Sadjad

Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads Sadjad

Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads Sadjad Fouladi, Riad S. Wahby, and Brennan Shacklett, Stanford University; Karthikeyan Vasuki Balasubramaniam, University of California, San Diego; William Zeng,

317 views • 19 slides
Exploiting Latency Variation for Access Conflict Reduction of NAND Flash Memory Jinhua Cui,

Exploiting Latency Variation for Access Conflict Reduction of NAND Flash Memory Jinhua Cui,

Exploiting Latency Variation for Access Conflict Reduction of NAND Flash Memory Jinhua Cui, Weiguo Wu, Xingjun Zhang, Jianhang Huang, Yinfeng Wang * Xian Jiaotong University, *ShenZhen Institute of Information Technology 1 OUTLINE 1.

509 views • 21 slides
Ou Out of the blue That day when evening came, he said to his disciples, Let us go over to

Ou Out of the blue That day when evening came, he said to his disciples, Let us go over to

Ou Out of the blue That day when evening came, he said to his disciples, Let us go over to the other side. Leaving the crowd behind, they took him along, just as he was, in the boat. There were also other boats with him. A furious M a

315 views • 13 slides
ITEC 1630 Yves Lesp erance Exception Handling Exception handling is a mechanism for making

ITEC 1630 Yves Lesp erance Exception Handling Exception handling is a mechanism for making

ITEC 1630 Yves Lesp erance Exception Handling Exception handling is a mechanism for making programs more robust, i.e. have then continue executing when errors, failures, or exceptional conditions occur, rather than crash. Lecture Notes

500 views • 7 slides
Objectives Continuing Java Fundamentals User Input Control Structures Arrays

Objectives Continuing Java Fundamentals User Input Control Structures Arrays

8/31/20 Objectives Continuing Java Fundamentals User Input Control Structures Arrays Command-line Arguments Aug 31, 2020 Sprenkle - CSCI209 1 1 Review What are some of the primitive data types of Java? What is the

701 views • 22 slides
Strings and File I/O Strings Java String objects are immutable Common methods include:

Strings and File I/O Strings Java String objects are immutable Common methods include:

Strings and File I/O Strings Java String objects are immutable Common methods include: boolean equalsIgnoreCase(String str) String toLowerCase() String substring(int offset, int endIndex) String replace(char oldChar,

638 views • 11 slides
fu next Brae Man Gradient estimate n Duane e wtwnfgg.eu rfure0BrCxl Hel E M If Brad the

fu next Brae Man Gradient estimate n Duane e wtwnfgg.eu rfure0BrCxl Hel E M If Brad the

harmonic section of Peter Li Harmonie funetriers polynomial growth Math Res left TheEuclidese 1 97 e Mean value formula fee fu next Brae Man Gradient estimate n Duane e wtwnfgg.eu rfure0BrCxl Hel E M If Brad the or E FM Mahal

378 views • 6 slides
Engineering Application OpenSees on GPU platform Ms Gouri Kadam 1 Ms Shweta Nayak 2 1 CDAC, Pune

Engineering Application OpenSees on GPU platform Ms Gouri Kadam 1 Ms Shweta Nayak 2 1 CDAC, Pune

Implementation of OpenSource Structural Engineering Application OpenSees on GPU platform Ms Gouri Kadam 1 Ms Shweta Nayak 2 1 CDAC, Pune University Campus Pune, India. Email: gourik@cdac.in 2 Department of Computer Engineering PICT, Pune, India.

397 views • 10 slides
EGSinduced seismicity Remarks by L. Rybach, GEOWATT AG Zurich (ENGINE Mid-term Conference, GFZ

EGSinduced seismicity Remarks by L. Rybach, GEOWATT AG Zurich (ENGINE Mid-term Conference, GFZ

EGSinduced seismicity Remarks by L. Rybach, GEOWATT AG Zurich (ENGINE Mid-term Conference, GFZ Postdam, 11 January 2007) Should EGS reach its full potential the issue of MMS (= man-made seismicity) must be addressed to the point of public

267 views • 12 slides
Densmap Area(Si) Area(Ni) where, Wi = the

Densmap Area(Si) Area(Ni) where, Wi = the

9. Statistical landslide hazard assessment In this exercise we will generate a landslide susceptibility map, using a basic, but useful, statistical method, called hazard index method. This method is based upon the following formula:

336 views • 10 slides

More recommend