Storm: a fast transactional dataplane for remote data structures - - PowerPoint PPT Presentation

▶
storm a fast transactional dataplane for remote data
SMART_READER_LITE
LIVE PREVIEW

Storm: a fast transactional dataplane for remote data structures - - PowerPoint PPT Presentation

Sep 29, 2023 431 likes •668 views

Storm: a fast transactional dataplane for remote data structures Stanko Novakovic Yizhou Shan Aasheesh Kolli Michael Cui Yiying Zhang Haggai Eran Boris Pismenny Liran Liss Michael Wei Dan Tsafrir Marcos Aguilera 12 th ACM

slide-1
SLIDE 1

Storm: a fast transactional dataplane for remote data structures

12th ACM International Systems and Storage Conference (SYSTOR)

Stanko Novakovic Yizhou Shan Aasheesh Kolli Michael Cui Yiying Zhang Haggai Eran Boris Pismenny Liran Liss Michael Wei Dan Tsafrir Marcos Aguilera

slide-2
SLIDE 2
  • Initiate transfer, hardware executes, async. poll for completions
  • Infiniband (IB): specialized network stack for RDMA
  • Fully implemented in hardware (PCIe-based adapters) →
  • Also: IB transport on top of IP and lossless Ethernet
  • Key benefits:
  • 1. one-sided access
  • 2. user-level w/ minimal instr. footprint

What is Remote Direct Memory Access (RDMA)?

2

slide-3
SLIDE 3

Remote data structures

  • Hash tables, graphs, trees, queues, etc
  • Fine-grain accesses
  • High fan-out
  • Pointer-linked
  • Transactional access
  • Throughput (IOPS) bound
  • Latency Service Level Objective (SLO)
  • Other (perhaps less interesting) use cases: analytics, VM migration
  • Bulk transfers, bandwidth-bound

3

slide-4
SLIDE 4

What are common concerns?

  • 1. Scalability: network state kept in limited hardware resources
  • 2. Round-trips: pointer-linked data structures

4

core

Cache CP U

R Q S Q

DR AM

rNIC

PCI/DMA

CQ

cache Infiniba nd or ETH

Protection

  • Addr. translation

DDIO

Connection state WQEs

slide-5
SLIDE 5

What are common concerns?

  • 1. Scalability: network state kept in limited hardware resources
  • FARM: Use locks to share QP connections (Dragojevic’14)
  • FaSST/eRPC: Don’t use connections (Kalia’19)
  • LITE: Enforce protection in kernel (Tsai’17)
  • 2. Round-trips: pointer-linked data structures
  • FARM: Use Hopscotch algorithm, one RTT common case
  • FaSST/eRPC: Leverage RPCs rather than one-sided reads

5

slide-6
SLIDE 6

Outline

  • Problem statement
  • Key insights
  • Storm design
  • Results

6

slide-7
SLIDE 7

Key insights (1/2)

  • Hardware has gotten much better!!!
  • ConnectX-4/5 (CX4/5) vs. ConnectX-3 (CX3)
  • 40M IOPS on CX4 → 4x higher than CX3
  • Scales up to 64 machines → on CX3 IOPS collapses for >10 machines
  • CX4 achieves 10M IOPS when zero cache hits → max IOPS for uncontended CX3
  • Break-even point with datagram send/recv currently at ~4k connections
  • Possible further improvements with ConnectX-6
  • How is HW getting better?
  • More concurrency, better prefetching, larger caches, etc

7

slide-8
SLIDE 8

Key insights (2/2)

  • FARM:
  • Locks degrade throughput unnecessarily
  • Large buckets (due to larger keys) wastes throughput
  • FaSST/eRPC:
  • Two-sided doesn’t allow for maximum full-duplex throughput
  • Especially for requests larger than a cache line (no inlining)
  • Onloaded congestion control adds overhead
  • LITE:
  • Kernel adds overhead (fine-grain accesses)
  • No support for async. operations

8

slide-9
SLIDE 9

Our approach / Storm design principles

  • 1. Use connections but minimal count
  • Lock-free QP sharing if really necessary
  • Offloaded congestion control and retransmissions
  • 2. Use one-sided reads whenever possible
  • First one-sided, then RPC (one-two-sided)
  • RPC also implemented using one-sided writes
  • 3. Leverage abundant memory
  • Cache metadata and/or reduce collisions in hash tables
  • 4. Minimize translation & protection state
  • Use contiguous physical allocation
  • 5. And don’t forget to deploy on new hardware!!!

9

slide-10
SLIDE 10

10

Storm dataplane Data structure

  • impl. & metadata

MEM

RR Division of responsibilities:

  • Storm DP only understands RDMA connections and memory regions
  • Data structure understands data layout and implements metadata caching

Event loop RPC

CPU

Storm dataplane Data structure

  • impl. & metadata

MEM CPU

rNIC rNIC HW SW

Storm design

QP & buffer mngmnt RR Event loop RPC QP & buffer mngmnt

slide-11
SLIDE 11

11

Storm dataplane

MEM CPU

Storm dataplane

MEM CPU

rNIC rNIC fail success HW SW Data structure

  • impl. & metadata

Data structure

  • impl. & metadata
  • p()

ev_loop() ev_loop()

Two-sided operations

RR Event loop RPC QP & buffer mngmnt RR Event loop RPC QP & buffer mngmnt success

1 2 3

slide-12
SLIDE 12

12

Storm dataplane

MEM CPU

Storm dataplane

MEM CPU

rNIC rNIC success HW SW Data structure

  • impl. & metadata

Data structure

  • impl. & metadata
  • p()

ev_loop()

One-sided operations

RR Event loop RPC QP & buffer mngmnt RR Event loop RPC QP & buffer mngmnt success

1 2 3

slide-13
SLIDE 13

13

Storm dataplane

MEM CPU

Storm dataplane

MEM CPU

rNIC rNIC success HW SW Data structure

  • impl. & metadata

Data structure

  • impl. & metadata
  • p()

ev_loop()

One-two-sided operations

RR Event loop RPC QP & buffer mngmnt RR Event loop RPC QP & buffer mngmnt fail

1 2 3

slide-14
SLIDE 14

14

Storm dataplane

MEM CPU

Storm dataplane

MEM CPU

rNIC rNIC fail success HW SW Data structure

  • impl. & metadata

Data structure

  • impl. & metadata
  • p()

ev_loop() ev_loop()

One-two-sided operations

RR Event loop RPC QP & buffer mngmnt RR Event loop RPC QP & buffer mngmnt success

3 4 5

slide-15
SLIDE 15

15

Storm dataplane

MEM CPU

Storm dataplane

MEM CPU

rNIC rNIC HW SW

TX TX

Data structure

  • impl. & metadata

Data structure

  • impl. & metadata

Distributed transactions

Support for concurrent data structures using transactions RR Event loop RPC QP & buffer mngmnt RR Event loop RPC QP & buffer mngmnt

slide-16
SLIDE 16

Data structure API (three callbacks)

  • RPC handler
  • Processing two-sided communication
  • Implements complex paths, such as acquiring locks and commits
  • Lookup start
  • Check if address is known (cached) or we can guess
  • If yes, leverage RDMA read
  • Lookup end
  • Check if data is valid and cache for future use

16

slide-17
SLIDE 17

Storm implementation & exp. setup

  • 13k LOC of C++, w/o MICA modifications [Lim’14]
  • HPC cluster w/ 32 Dell machines
  • High-speed Infiniband network (100Gbps)
  • Mellanox ConnectX-4 – similar in perf to CX5
  • Emulation of 3-4x larger clusters possible on Storm
  • Benchmarks:
  • Key-value transactional micro-benchmark
  • Telecommunication Application Transaction Processing (TATP)

17

slide-18
SLIDE 18

Outline

  • Problem statement
  • Key insights
  • Storm design
  • Results

18

slide-19
SLIDE 19

Baselines

  • Emulated FARM (modified: Lock-free_FaRM)
  • No connection sharing, 1KB “neighborhoods”
  • eRPC
  • With and without active congestion control
  • LITE (modified: Async_LITE)
  • Added support for asynchronous operations

19

slide-20
SLIDE 20

Storm results

  • Single-lookup workload
  • 128B KV pairs, 100M items, 20 threads per mn

10 20 30 40 50 4 8 12 16 20 24 28 32 Per-mn lookups / usec Number of machines

Storm (cache)

20

slide-21
SLIDE 21

Storm results

  • Single-lookup workload
  • 128B KV pairs, 100M items, 20 threads per mn

10 20 30 40 50 4 8 12 16 20 24 28 32 Per-mn lookups / usec Number of machines

Storm (cache) Storm (oversub)

10 20 30 40 50 4 8 12 16 Per-mn lookups / usec Number of physical machines

Storm(oversub) eRPC (w/o CC) eRPC Lock-free FARM Async_LITE (projected)

21

  • ne-two-sided operations
  • TATP: 11.8 million per node with Storm (oversub)
slide-22
SLIDE 22

Does Storm scale well?

  • Storm scales well up to 64mn
  • Reduce thread count by 2x
  • 2x fewer threads → 2x fewer QPs
  • Do we need more than 10 threads?
  • Lock-free QP sharing

10 20 30 40 50 32 64 96 128 Per-mn lookups / usec Number of emulated machines

Storm(cache)-20x Storm(cache)-10x

22

slide-23
SLIDE 23

Conclusion & future work

23

  • RDMA datacenter users should get a hardware upgrade
  • More scalable hardware available
  • Take advantage of one-sided primitives
  • Leverage caching and oversubscription (in hash tables)
  • One-sided read in the common case
  • Ongoing research threads:
  • Designing “far” memory data structures (HotOS’19)
  • Memory allocator for repurposing unused memory
  • Lock-free mechanisms for QP sharing