Securing RDMA for High-Performance Datacenter Storage Systems Anna - - PowerPoint PPT Presentation

▶
securing rdma for high performance
SMART_READER_LITE
LIVE PREVIEW

Securing RDMA for High-Performance Datacenter Storage Systems Anna - - PowerPoint PPT Presentation

Mar 19, 2023 444 likes •576 views

Securing RDMA for High-Performance Datacenter Storage Systems Anna Kornfeld Simpson, Adriana Szekeres (Paul G. Allen School of Computer Science & Engineering, University of Washington), Jacob Nelson, Irene Zhang (Microsoft Research) 1

slide-1
SLIDE 1

Securing RDMA for High-Performance Datacenter Storage Systems

Anna Kornfeld Simpson, Adriana Szekeres (Paul G. Allen School of Computer Science & Engineering, University of Washington), Jacob Nelson, Irene Zhang (Microsoft Research)

1

slide-2
SLIDE 2

Remote Direct Memory Access (RDMA) does CPU-bypass over the datacenter network with only a few microseconds of latency

RDMA over Converged Ethernet (RoCEv2) packet.

Source: RoCEv2 spec, Infiniband Trade Association, 2014

Ethernet Head. IP Head. UDP Head. RDMA Head. & Data Queue Pair Info. Remote Memory Addr. and r_key Payload

Abstracted RDMA portion of RoCEv2 packet.

2

slide-3
SLIDE 3

Example RDMA System: Pilaf (2013): Put (SEND)

Clients CPU Memory NIC Server 1 3 5 2 4

3

slide-4
SLIDE 4

Pilaf (2013): Unlike Put, Get is CPU-bypassing

4

slide-5
SLIDE 5

RDMA not designed for datacenter security needs

Security weaknesses discovered over past 2 decades (see Section 2 of paper for citations):

  • Confidentiality:

packet in plaintext

  • Integrity:

no packet integrity check or authentication

  • Availability:

denial of service

  • Side channels:

non-random r_keys and more

5

slide-6
SLIDE 6

We analyzed recent distributed storage systems built on RDMA and discovered additional systems design challenges even after security fundamentals are fixed.

  • Can RDMA-based storage systems provide security at least as good

as pre-RDMA datacenter security best practices?

  • We analyzed:

Pilaf, FaRM, HERD, DrTM, FaSST, Octopus, Hyperloop, DrTM+H

6

slide-7
SLIDE 7

Threat Model = Compromised Storage Client

7

VLANs/virtualization does not help! Compromised client

  • nly needs to see its own network traffic to spoof RDMA.

CPU Memory NIC Server Bad() Clients

slide-8
SLIDE 8

Challenge 1: no auditability/logging on reads

Adversary does CPU-bypassing READ

Clients CPU Memory NIC Server

1 2 3

What data was exfiltrated?

8

slide-9
SLIDE 9

Challenge 2: Design Implications of Storage Logic Location: RPC and Concurrency

9 4 5 6

DrTM (2016) Put

slide-10
SLIDE 10

Challenge 3: Separating different users’ data

  • Single big remote memory registration -> attacker access to

all user data

  • Vendor suggested solution (protection domains) a poor

performance fit for storage systems with multiple storage clients who all want to access same data

10

slide-11
SLIDE 11

Ingredients for more secure CPU-bypass systems

Security Fundamentals

  • High throughput AEAD

cryptography for datapath (e.g. DTLS)

  • Centralized key

management

Source: Zookeeper Project

System Design Challenges

  • Logging strategy that does not

rely on client

  • Alternatives to unreliable RPC
  • Finer-grained permissions on

remote data access

11

slide-12
SLIDE 12

Lots of big open questions for future research!

  • Wishlist for features to help support application security

when building systems that use CPU-bypassing RDMA?

  • Wishlist for securing non-user-facing datacenter tasks?
  • How do we get these better features baked in? Changing the

RDMA standard?

Thank you for watching! Questions? Email Anna: aksimpso@cs.washington.edu

12