Prism: A Proxy Architecture for Datacenter Networks Yutaro Hayakawa - - PowerPoint PPT Presentation

▶
prism a proxy architecture for datacenter networks
SMART_READER_LITE
LIVE PREVIEW

Prism: A Proxy Architecture for Datacenter Networks Yutaro Hayakawa - - PowerPoint PPT Presentation

Mar 13, 2023 780 likes •1.22k views

Prism: A Proxy Architecture for Datacenter Networks Yutaro Hayakawa (Keio University) Lars Eggert (NetApp) Michio Honda (NEC Laboratories Europe) Douglas Santry (NetApp) 1 Background TCP proxy plays important roles in modern cloud

slide-1
SLIDE 1

Yutaro Hayakawa (Keio University) Lars Eggert (NetApp) Michio Honda (NEC Laboratories Europe) Douglas Santry (NetApp)

Prism: A Proxy Architecture for Datacenter Networks

1

slide-2
SLIDE 2

Background

  • TCP proxy plays important roles in modern cloud services
  • Web cache
  • Application level firewall
  • Application level load balancer

2

slide-3
SLIDE 3

Problem

  • Aggregated bandwidth of backends are always

constrained by proxy bandwidth

3

slide-4
SLIDE 4

Problem

  • Aggregated bandwidth of backends are always

constrained by proxy bandwidth

4

Backend1 Backend2 Backend3

slide-5
SLIDE 5

Problem

  • Aggregated bandwidth of backends are always

constrained by proxy bandwidth

5

Proxy Backend1 Backend2 Backend3

slide-6
SLIDE 6

Problem

  • Aggregated bandwidth of backends are always

constrained by proxy bandwidth

6

10G 10G 10G Proxy Backend1 Backend2 Backend3

slide-7
SLIDE 7

Problem

  • Aggregated bandwidth of backends are always

constrained by proxy bandwidth

7

10G 10G 10G 10G Proxy Backend1 Backend2 Backend3

slide-8
SLIDE 8

Problem

  • Aggregated bandwidth of backends are always

constrained by proxy bandwidth

8

10G 10G 10G 10G Proxy Backend1 Backend2 Backend3

Total Bandwidth for Backends == 10G

slide-9
SLIDE 9

Why does it matter?

  • This problem is significant especially at the edge
  • f today’s datacenter topology

9

slide-10
SLIDE 10

Example : HTTP/1.1

  • Simple web system
  • One frontend proxy and multiple backend server

40Gbps Links Backend Frontend Proxy Backend Backend 10Gbps Links

・・・ ・・・

10

slide-11
SLIDE 11

Example : HTTP/1.1

40Gbps Links Backend Frontend Proxy Backend Backend 10Gbps Links

・・・ ・・・

Frontend Proxy

11

Client’s HTTP GET

slide-12
SLIDE 12

Example : HTTP/1.1

40Gbps Links Backend Frontend Proxy Backend Backend 10Gbps Links

・・・ ・・・

Frontend Proxy

12

Forward To backend

slide-13
SLIDE 13

Example : HTTP/1.1

40Gbps Links Backend Frontend Proxy Backend Backend 10Gbps Links

・・・ ・・・

Frontend Proxy

13

Return HTTP OK

slide-14
SLIDE 14

Example : HTTP/1.1

40Gbps Links Backend Frontend Proxy Backend Backend 10Gbps Links

・・・ ・・・

Frontend Proxy

14

Forward it to client

slide-15
SLIDE 15

Example : HTTP/1.1

40Gbps Links Backend Backend Backend 10Gbps Links

・・・ ・・・

Frontend Proxy

15

slide-16
SLIDE 16

Example : HTTP/1.1

40Gbps Links Backend Backend Backend 10Gbps Links

Problem: All incoming/outgoing traffic go through here

・・・ ・・・

Frontend Proxy

16

slide-17
SLIDE 17

How can we solve this problem?

40Gbps Links Backend Frontend Proxy Backend Backend 10Gbps Links

・・・ ・・・

Frontend Proxy

17

Outgoing traffic from backend bypass frontend proxy

slide-18
SLIDE 18

Related work

  • L4 load balancer
  • Maglev[1], Ananta[2], Duet[3]

18

[1] D. E. Eisenbud et al. Maglev: A Fast and Reliable Software Network Load Balancer. Proc. usenix nsdi, 2016. [2] P. Patel et al. Ananta: Cloud Scale Load Balancing. Proc. acm sigcomm, 2013. [3] R. Gandhi et al. Duet: Cloud Scale Load Balancing with Hardware and Software. Proc. acm sigcomm, 2014.

Back end L4 LB Back end Back end Client Balance TCP connect request

[4] M. Alizadeh et al. CONGA: distributed congestion-aware load balancing for datacenters.

  • Proc. Acm sigcomm. Chicago, IL, USA, 2014, pp. 503–514.

Talk directly

slide-19
SLIDE 19

Prism Architecture

19

40Gbps Links Back end Proxy Back end Back end 10Gbps Links Client L2 Switch

Legacy Proxy

slide-20
SLIDE 20

Prism Architecture

20

40Gbps Links Back end Proxy Back end Back end 10Gbps Links Client L2 Switch 40Gbps Links

Custom Backend

Switch Controller (Proxy)

10Gbps Links Client Programmable Switch(e.g.P4)

Custom Backend Custom Backend

Legacy Proxy Prism

slide-21
SLIDE 21

Prism Architecture

21

40Gbps Links Back end Proxy Back end Back end 10Gbps Links Client L2 Switch 40Gbps Links

Custom Backend

Switch Controller (Proxy)

10Gbps Links Client Programmable Switch

Custom Backend Custom Backend

Legacy Proxy Prism

Basic idea: Offloading forwarding of application protocol payload to a switch

slide-22
SLIDE 22

Prism Architecture

22

40Gbps Links Back end Proxy Back end Back end 10Gbps Links Client L2 Switch 40Gbps Links

Custom Backend

Switch Controller (Proxy)

10Gbps Links Client Programmable Switch

Custom Backend Custom Backend

Legacy Proxy Prism

Basic idea: Offloading application protocol payload forwarding to a switch => the Switch Controller (Proxy) only handles header exchange

slide-23
SLIDE 23

How does it work?

23

40Gbps Links Custom Backend Switch Controller (Proxy) Custom Backend Custom Backend 10Gbps Links

・・・ ・・・

Programmable Switch Client Example Behavior HTTP/1.1

slide-24
SLIDE 24

How does it work?

24

40Gbps Links Custom Backend Switch Controller (Proxy) Custom Backend Custom Backend 10Gbps Links

・・・ ・・・

Programmable Switch Client Client and Switch Controller establish TCP connection

slide-25
SLIDE 25

How does it work?

25

40Gbps Links Custom Backend Switch Controller (Proxy) Custom Backend Custom Backend 10Gbps Links

・・・ ・・・

Programmable Switch Client Client sends request to switch controller GET /example/index.html

slide-26
SLIDE 26

How does it work?

26

40Gbps Links Custom Backend Switch Controller (Proxy) Custom Backend Custom Backend 10Gbps Links

・・・ ・・・

Programmable Switch Client Switch controller selects backend

slide-27
SLIDE 27

How does it work?

27

40Gbps Links Custom Backend Switch Controller (Proxy) Custom Backend Custom Backend 10Gbps Links

・・・ ・・・

Programmable Switch Client Switch controller injects a rule to the switch which rewrites fields of packets from backend Rule

slide-28
SLIDE 28

How does it work?

28

40Gbps Links Custom Backend Switch Controller (Proxy) Custom Backend Custom Backend 10Gbps Links

・・・ ・・・

Programmable Switch Client Switch controller sends client request to backend with some TCP metadata

Sequence Number ACK Number Client’s request TCP options…

Rule

slide-29
SLIDE 29

How does it work?

29

40Gbps Links Custom Backend Switch Controller (Proxy) Custom Backend Custom Backend 10Gbps Links

・・・ ・・・

Programmable Switch Client Backend migrate TCP connection from information given by Switch controller

Sequence Number ACK Number Client’s request TCP options…

Rule

slide-30
SLIDE 30

How does it work?

30

40Gbps Links Custom Backend Switch Controller (Proxy) Custom Backend Custom Backend 10Gbps Links

・・・ ・・・

Programmable Switch Client Backend and client communicate directly via migrated TCP connection

HTTP OK and Payload data

Rule

slide-31
SLIDE 31

How does it work?

31

40Gbps Links Custom Backend Switch Controller (Proxy) Custom Backend Custom Backend 10Gbps Links

・・・ ・・・

Programmable Switch Client Backend send number

  • f bytes sent to client to

switch controller I sent N bytes Rule

slide-32
SLIDE 32

How does it work?

32

40Gbps Links Custom Backend Switch Controller (Proxy) Custom Backend Custom Backend 10Gbps Links

・・・ ・・・

Programmable Switch Client Switch controller removes the rule from the switch Rule

slide-33
SLIDE 33

How does it work?

33

40Gbps Links Custom Backend Switch Controller (Proxy) Custom Backend Custom Backend 10Gbps Links

・・・ ・・・

Programmable Switch Client Switch controller waits for next request or FIN packet Waiting for FIN or next request

slide-34
SLIDE 34

Challenge

  • Why do we need to use programmable switch like P4?
  • How can we stop pipelined HTTP request from client

during TCP connection is migrated in backend?

  • What happen when the client resets the connection?
  • More details are in paper!

34

slide-35
SLIDE 35

Evaluation

  • We setup two experiments
  • 1. End-to-End throughput comparison between legacy

proxy and Prism for HTTP/1.1

  • 2. Packet transformation performance measurement for

Prism Switch

35

slide-36
SLIDE 36

Evaluation: End-to-End Throughput

  • 2 clients and 2 servers

36

HTTP Benchmarking App HTTP Benchmarking App L2 Switch (Software) Proxy HTTP Server HTTP Server

slide-37
SLIDE 37

Evaluation: End-to-End Throughput

  • 2 clients and 2 servers

37

HTTP Benchmarking App HTTP Benchmarking App Prism Switch (Software) Prism Switch Controller Prism HTTP Backend Prism HTTP Backend

slide-38
SLIDE 38

Evaluation: End-to-End Throughput

38

5 10 15 1 10 100 1000 2000 5000 10000 50000 75000 100000 Object Size [KB] Throughput [Gbps] Prism Proxy

  • Total throughput of proxy’s clients

are limited to about 9Gbps in maximum

  • Total throughput of Prism’s

clients exceeds 10Gbps in 2MB data transfer

  • Finally reached to about

18.7Gbps in maximum Prism can use bandwidth of uplinks which proxy’s clients can’t use

slide-39
SLIDE 39

Evaluation: End-to-End Throughput

39

5 10 15 1 10 100 1000 2000 5000 10000 50000 75000 100000 Object Size [KB] Throughput [Gbps] Prism Proxy

  • Total throughput of proxy’s clients

are limited to about 9Gbps in maximum

  • Total throughput of Prism’s

clients exceeds 10Gbps in 2MB data transfer

  • Finally reached to about

18.7Gbps in maximum Prism can use bandwidth of uplinks which proxy’s clients can’t use 10G

slide-40
SLIDE 40

Evaluation: Prism Switch Performance

  • Measure performance of

the Prism switch packet transformation

  • Prism switch achieves
  • 63Gbps in 1514byte packet

using 1 core

  • 127Gbps in 2 cores

40

20 40 60 80 100 120 60 (1 core) 1514 (1 core) 60 (2 cores) 1514 (2 cores) Packet size [Bytes] Throughput [Gbps] No Logic L2 Learning Prism pkt-gen tx pkt-gen rx

Virtual NIC Virtual NIC

Switching Logic

mSwitch

The Prism switch can achieve high throughput even if it is implemented as software

slide-41
SLIDE 41

Conclusion

  • Legacy proxy architecture can’t utilize full bandwidth of

switch uplinks in datacenters

  • To solve this problem, we designed and implemented

Prism which connects clients and backend servers directly during payload transaction of application protocol

  • Prism improves bandwidth utilization

41

slide-42
SLIDE 42

Question?

42