Reactive Cloud-Native Networking info@netifi.com www.netifi.com - - PowerPoint PPT Presentation

▶
reactive cloud native networking
SMART_READER_LITE
LIVE PREVIEW

Reactive Cloud-Native Networking info@netifi.com www.netifi.com - - PowerPoint PPT Presentation

Sep 05, 2023 188 likes •796 views

Reactive Cloud-Native Networking info@netifi.com www.netifi.com Speakers Arsalan Farooq CEO Robert Roeser CINO Founder/CEO of Convirture Netflix Edge Platform team (acquired by Accelerite) RSocket core contributor

slide-1
SLIDE 1

info@netifi.com www.netifi.com

Reactive Cloud-Native Networking

slide-2
SLIDE 2
  • Arsalan Farooq CEO
  • Founder/CEO of Convirture

(acquired by Accelerite)

  • Founded Oracle’s ASLM

division and grew it to a multi-national organization

  • Robert Roeser CINO
  • Netflix Edge Platform team
  • RSocket core contributor
  • Principal Architect at Nike
  • n realtime analytics

system serving 30M+ users

Speakers

slide-3
SLIDE 3

Built by leaders in microservices and cloud computing

  • Open Source Layer 5/6

communication protocol

  • Reactive streams semantics
  • Application-level flow control
  • Binary Encoding
  • Asynchronous Message-Passing
slide-4
SLIDE 4

What’s needed 
 for high performance
 Microservice Networking?

slide-5
SLIDE 5
  • Temporal Decoupling
  • Spacial Decoupling
  • Binary

  • Application-Flow control

5

High Performance Microservices

slide-6
SLIDE 6

Spacial Decoupling

A message’s sender should not directly call a destination The execution time processing a call should not affect the caller

Loose Coupling

Temporal Decoupling

slide-7
SLIDE 7

RSocket is loosely coupled 
 using message-passing.

slide-8
SLIDE 8

What’s the difference between Message-Passing and Event-Driven?

slide-9
SLIDE 9

9

Message-Passing vs Event-Driven

Message-Passing Event-Driven

A message is a payload sent to a specific destination An event is signal emitted by a component reaching a specific stage Message-driven focuses on addressable recipients Event drive focuses on listen for events Errors passed as messages - can be easy sent to caller Error handling in Event driven very complex - think dead letter queues Bi-directional communication is easy Bi-directional communication is very hard in Event- driven

slide-10
SLIDE 10

10

Event-Driven

Emits Events Listens for Events Queue

slide-11
SLIDE 11

11

Event-Driven

Emits Events Listens for Events Queue

slide-12
SLIDE 12

12

Event-Driven

Emits Events Listens for Events Queue

slide-13
SLIDE 13

13

Event-Driven

Emits Events Listens for Events Queue

slide-14
SLIDE 14

14

Event-Driven

Emits Events Listens for Events Queue

slide-15
SLIDE 15

15

Event-Driven

Emits Events Listens for Events Queue

response

???

slide-16
SLIDE 16

16

Event-Driven

Emits Events Listens for Events Queue

exception

???

slide-17
SLIDE 17

17

Event-Driven

Emits Events Listens for Events Queue ???

backpressure

slide-18
SLIDE 18

18

Message-Passing

Dest 1 Dest 2 Dest 3 Dest 4 Dest 5 Dest 6

slide-19
SLIDE 19

19

Message-Passing

Dest 1 Dest 2 Dest 3 Dest 4 Dest 5 Dest 6

slide-20
SLIDE 20

20

Message-Passing

Dest 1 Dest 2 Dest 3 Dest 4 Dest 5 Dest 6

slide-21
SLIDE 21

21

Message-Passing

Dest 1 Dest 2 Dest 3 Dest 4 Dest 5 Dest 6

slide-22
SLIDE 22

22

RSocket - Message-Passing

Dest 1 Dest 2

slide-23
SLIDE 23

23

RSocket - Message-Passing

Dest 1 Dest 2

requestChannel

slide-24
SLIDE 24

24

RSocket - Message-Passing

Dest 1 Dest 2

request(3) requestChannel

slide-25
SLIDE 25

25

RSocket - Message-Passing

Dest 1 Dest 2

Payload request(3) requestChannel

slide-26
SLIDE 26

26

RSocket - Message-Passing

Dest 1 Dest 2

Payload request(3) requestChannel Payload

slide-27
SLIDE 27

27

RSocket - Message-Passing

Dest 1 Dest 2

Payload request(3) requestChannel Payload Payload

slide-28
SLIDE 28

28

RSocket - Message-Passing

Dest 1 Dest 2

Payload request(3) requestChannel Payload Payload request(2)

slide-29
SLIDE 29

29

RSocket - Message-Passing

Dest 1 Dest 2

Payload request(3) requestChannel Payload Payload request(2) complete

slide-30
SLIDE 30

30

RSocket - Message-Passing

Dest 1 Dest 2

Payload request(3) requestChannel Payload Payload request(2)

slide-31
SLIDE 31

31

RSocket - Message-Passing

Dest 1 Dest 2

Payload request(3) requestChannel Payload Payload exception request(2)

slide-32
SLIDE 32

Spacial Decoupling

RSocket messages are binary frames over dedicated streams RSocket’s APIs are non-blocking

RSocket - Loose Coupling

Temporal Decoupling

slide-33
SLIDE 33

What type of Messages does RSocket use?

slide-34
SLIDE 34

Binary Payloads

Efficiently encoded binary payloads Resumable lens over bytes to access messages

RSocket’s Messages

Flyweight Pattern

slide-35
SLIDE 35

How does RSocket Process Messages?

slide-36
SLIDE 36

It’s easy to get processing wrong.

slide-37
SLIDE 37

Key Insight

public class Counter1 { private int count; private int count() { //count = 0; for (int i = 0; i < Integer.MAX_VALUE; i++) { count++; } return count; } public static void main(String... args) { final Counter1 counter1 = new Counter1(); final Recorder histogram = new Recorder(3600000000000L, 3); for (int i = 0; i < 100; i++) { long start = System.nanoTime(); counter1.count(); long stop = System.nanoTime(); histogram.recordValue(stop - start); } histogram .getIntervalHistogram() .outputPercentileDistribution(System.out, 5, 1000.0, false); } }

slide-38
SLIDE 38

Key Insight

public class Counter1 { private int count; private int count() { //count = 0; for (int i = 0; i < Integer.MAX_VALUE; i++) { count++; } return count; } public static void main(String... args) { final Counter1 counter1 = new Counter1(); final Recorder histogram = new Recorder(3600000000000L, 3); for (int i = 0; i < 100; i++) { long start = System.nanoTime(); counter1.count(); long stop = System.nanoTime(); histogram.recordValue(stop - start); } histogram .getIntervalHistogram() .outputPercentileDistribution(System.out, 5, 1000.0, false); } }

Single

Count Counter Main

slide-39
SLIDE 39

Key Insight

public class Counter1 { private int count; private int count() { //count = 0; for (int i = 0; i < Integer.MAX_VALUE; i++) { count++; } return count; } public static void main(String... args) { final Counter1 counter1 = new Counter1(); final Recorder histogram = new Recorder(3600000000000L, 3); for (int i = 0; i < 100; i++) { long start = System.nanoTime(); counter1.count(); long stop = System.nanoTime(); histogram.recordValue(stop - start); } histogram .getIntervalHistogram() .outputPercentileDistribution(System.out, 5, 1000.0, false); } }

Takes about 1 microsecond.

Single

Count Counter Main

slide-40
SLIDE 40

Key Insight

public class Counter2 { private static AtomicIntegerFieldUpdater<Counter2> COUNT = AtomicIntegerFieldUpdater.newUpdater(Counter2.class, "count"); private volatile int count; private int count() throws Exception { int target = Integer.MAX_VALUE; count = 0; ExecutorService executor = Executors.newCachedThreadPool(); for (int i = 0; i < Runtime.getRuntime().availableProcessors(); i++) { executor.execute(() -> { while (COUNT.incrementAndGet(Counter2.this) < target) Thread.yield(); }); } while (COUNT.get(Counter2.this) < target) Thread.yield(); return count; } public static void main(String... args) throws Exception { final Counter2 counter = new Counter2(); final Recorder histogram = new Recorder(3600000000000L, 3); for (int i = 0; i < 100; i++) { long start = System.nanoTime(); counter.count(); long stop = System.nanoTime(); histogram.recordValue(stop - start); } histogram.getIntervalHistogram().outputPercentileDistribution(System.out, 5, 1000.0, false); } }

slide-41
SLIDE 41

Key Insight

public class Counter2 { private static AtomicIntegerFieldUpdater<Counter2> COUNT = AtomicIntegerFieldUpdater.newUpdater(Counter2.class, "count"); private volatile int count; private int count() throws Exception { int target = Integer.MAX_VALUE; count = 0; ExecutorService executor = Executors.newCachedThreadPool(); for (int i = 0; i < Runtime.getRuntime().availableProcessors(); i++) { executor.execute(() -> { while (COUNT.incrementAndGet(Counter2.this) < target) Thread.yield(); }); } while (COUNT.get(Counter2.this) < target) Thread.yield(); return count; } public static void main(String... args) throws Exception { final Counter2 counter = new Counter2(); final Recorder histogram = new Recorder(3600000000000L, 3); for (int i = 0; i < 100; i++) { long start = System.nanoTime(); counter1.count(); long stop = System.nanoTime(); histogram.recordValue(stop - start); } histogram.getIntervalHistogram().outputPercentileDistribution(System.out, 5, 1000.0, false); } }

Multi-threaded

Count Counter Main Counter Counter

slide-42
SLIDE 42

Key Insight

public class Counter2 { private static AtomicIntegerFieldUpdater<Counter2> COUNT = AtomicIntegerFieldUpdater.newUpdater(Counter2.class, "count"); private volatile int count; private int count() throws Exception { int target = Integer.MAX_VALUE; count = 0; ExecutorService executor = Executors.newCachedThreadPool(); for (int i = 0; i < Runtime.getRuntime().availableProcessors(); i++) { executor.execute(() -> { while (COUNT.incrementAndGet(Counter2.this) < target) Thread.yield(); }); } while (COUNT.get(Counter2.this) < target) Thread.yield(); return count; } public static void main(String... args) throws Exception { final Counter2 counter = new Counter2(); final Recorder histogram = new Recorder(3600000000000L, 3); for (int i = 0; i < 100; i++) { long start = System.nanoTime(); counter1.count(); long stop = System.nanoTime(); histogram.recordValue(stop - start); } histogram.getIntervalHistogram().outputPercentileDistribution(System.out, 5, 1000.0, false); } }

Didn’t wait for it to finish.

Multi-threaded

Count Counter Main Counter Counter

slide-43
SLIDE 43

Key Insight

public class Counter3 { private static AtomicIntegerFieldUpdater<Counter3> WIP = AtomicIntegerFieldUpdater.newUpdater(Counter3.class, "wip"); private volatile int wip; private volatile int count; private int count() throws Exception { count = 0; int parties = Runtime.getRuntime().availableProcessors(); for (int i = 0; i < parties; i++) { Thread thread = new Thread( () -> { while (count < Integer.MAX_VALUE) { // Use work in progress flag to let only one thread count at a time if (WIP.compareAndSet(Counter3.this, 0, 1)) { // Update the volatile variable count = doCount(); // No more work - set WIP back to zero WIP.set(Counter3.this, 0); } else Thread.yield(); } }); thread.start(); } while (count < Integer.MAX_VALUE) Thread.yield(); return count; } private int doCount() { int c = 0; for (int i = 0; i < Integer.MAX_VALUE; i++) c++; return c; } public static void main(String... args) throws Exception { final Counter3 counter1 = new Counter3(); final Recorder histogram = new Recorder(3600000000000L, 3); for (int i = 0; i < 100; i++) { long start = System.nanoTime(); counter1.count(); long stop = System.nanoTime(); histogram.recordValue(stop - start); } histogram.getIntervalHistogram().outputPercentileDistribution(System.out, 5, 1000.0, false); } }

slide-44
SLIDE 44

Key Insight

public class Counter3 { private static AtomicIntegerFieldUpdater<Counter3> WIP = AtomicIntegerFieldUpdater.newUpdater(Counter3.class, "wip"); private volatile int wip; private volatile int count; private int count() throws Exception { count = 0; int parties = Runtime.getRuntime().availableProcessors(); for (int i = 0; i < parties; i++) { Thread thread = new Thread( () -> { while (count < Integer.MAX_VALUE) { // Use work in progress flag to let only one thread count at a time if (WIP.compareAndSet(Counter3.this, 0, 1)) { // Update the volatile variable count = doCount(); // No more work - set WIP back to zero WIP.set(Counter3.this, 0); } else Thread.yield(); } }); thread.start(); } while (count < Integer.MAX_VALUE) Thread.yield(); return count; } private int doCount() { int c = 0; for (int i = 0; i < Integer.MAX_VALUE; i++) c++; return c; } public static void main(String... args) throws Exception { final Counter3 counter1 = new Counter3(); final Recorder histogram = new Recorder(3600000000000L, 3); for (int i = 0; i < 100; i++) { long start = System.nanoTime(); counter1.count(); long stop = System.nanoTime(); histogram.recordValue(stop - start); } histogram.getIntervalHistogram().outputPercentileDistribution(System.out, 5, 1000.0, false); } }

Drain Loop

Count Counter Main Counter Count

… …

slide-45
SLIDE 45

Key Insight

Less than a millisecond.

public class Counter3 { private static AtomicIntegerFieldUpdater<Counter3> WIP = AtomicIntegerFieldUpdater.newUpdater(Counter3.class, "wip"); private volatile int wip; private volatile int count; private int count() throws Exception { count = 0; int parties = Runtime.getRuntime().availableProcessors(); for (int i = 0; i < parties; i++) { Thread thread = new Thread( () -> { while (count < Integer.MAX_VALUE) { // Use work in progress flag to let only one thread count at a time if (WIP.compareAndSet(Counter3.this, 0, 1)) { // Update the volatile variable count = doCount(); // No more work - set WIP back to zero WIP.set(Counter3.this, 0); } else Thread.yield(); } }); thread.start(); } while (count < Integer.MAX_VALUE) Thread.yield(); return count; } private int doCount() { int c = 0; for (int i = 0; i < Integer.MAX_VALUE; i++) c++; return c; } public static void main(String... args) throws Exception { final Counter3 counter1 = new Counter3(); final Recorder histogram = new Recorder(3600000000000L, 3); for (int i = 0; i < 100; i++) { long start = System.nanoTime(); counter1.count(); long stop = System.nanoTime(); histogram.recordValue(stop - start); } histogram.getIntervalHistogram().outputPercentileDistribution(System.out, 5, 1000.0, false); } }

Drain Loop

Count Counter Main Counter Count

… …

slide-46
SLIDE 46

Key Insight

public class Counter4 { private volatile int count; private Mono<Integer> count() { return Mono.fromSupplier( () -> { int c = count; for (int i = 0; i < Integer.MAX_VALUE; i++) { c++; } count = c; return c; }); } public static void main(String... args) throws Exception { Counter4 counter = new Counter4(); final Recorder histogram = new Recorder(3600000000000L, 3); Flux.range(1, 100) .concatMap( ignore -> { long start = System.nanoTime(); return counter .count() .doFinally(s -> histogram.recordValue(System.nanoTime() - start)); }) .then() .block(); histogram.getIntervalHistogram().outputPercentileDistribution(System.out, 5, 1000.0, false); } }

slide-47
SLIDE 47

Key Insight

public class Counter4 { private volatile int count; private Mono<Integer> count() { return Mono.fromSupplier( () -> { int c = count; for (int i = 0; i < Integer.MAX_VALUE; i++) { c++; } count = c; return c; }); } public static void main(String... args) throws Exception { Counter4 counter = new Counter4(); final Recorder histogram = new Recorder(3600000000000L, 3); Flux.range(1, 100) .concatMap( ignore -> { long start = System.nanoTime(); return counter .count() .doFinally(s -> histogram.recordValue(System.nanoTime() - start)); }) .then() .block(); histogram.getIntervalHistogram().outputPercentileDistribution(System.out, 5, 1000.0, false); } }

Drain Loop

Count Counter Main Counter Count

… …

slide-48
SLIDE 48

Key Insight

Less than a millisecond.

public class Counter4 { private volatile int count; private Mono<Integer> count() { return Mono.fromSupplier( () -> { int c = count; for (int i = 0; i < Integer.MAX_VALUE; i++) { c++; } count = c; return c; }); } public static void main(String... args) throws Exception { Counter4 counter = new Counter4(); final Recorder histogram = new Recorder(3600000000000L, 3); Flux.range(1, 100) .concatMap( ignore -> { long start = System.nanoTime(); return counter .count() .doFinally(s -> histogram.recordValue(System.nanoTime() - start)); }) .then() .block(); histogram.getIntervalHistogram().outputPercentileDistribution(System.out, 5, 1000.0, false); } }

Drain Loop

Count Counter Main Counter Count

… …

slide-49
SLIDE 49
  • Reactive Streams abstracts async message passing
  • No difference in message passing within and across a binary boundary

to the caller

  • RSocket formally defines a protocol to make Reactive Streams work

across a binary boundary

  • Can swap RSocket in place of Reactive Streams calls

49

Reactive Streams / RSocket Connection

slide-50
SLIDE 50

Reactive Stream / RSocket Connection

public class Counter4 { private volatile int count; private Mono<Integer> count() { return Mono.fromSupplier( () -> { int c = count; for (int i = 0; i < Integer.MAX_VALUE; i++) { c++; } count = c; return c; }); } public static void main(String... args) throws Exception { Counter4 counter = new Counter4(); final Recorder histogram = new Recorder(3600000000000L, 3); Flux.range(1, 100) .concatMap( ignore -> { long start = System.nanoTime(); return counter .count() .doFinally(s -> histogram.recordValue(System.nanoTime() - start)); }) .then() .block(); histogram.getIntervalHistogram().outputPercentileDistribution(System.out, 5, 1000.0, false); } }

slide-51
SLIDE 51

Reactive Stream / RSocket Connection

public class Counter4 { private RSocket rSocket; public Counter4() { this.rSocket = RSocketFactory.connect().transport(TcpClientTransport.create(9090)).start().block(); } @Override public Mono<Integer> count() { return rSocket .requestResponse(ByteBufPayload.create(new byte[0])) .map( payload -> { int i = payload.data().readInt(); payload.release(); return i; }); } public static void main(String... args) throws Exception { Counter4 counter = new Counter4(); final Recorder histogram = new Recorder(3600000000000L, 3); Flux.range(1, 100) .concatMap( ignore -> { long start = System.nanoTime(); return counter .count() .doFinally(s -> histogram.recordValue(System.nanoTime() - start)); }) .then() .block(); histogram.getIntervalHistogram().outputPercentileDistribution(System.out, 5, 1000.0, false); }

slide-52
SLIDE 52

Reactive Stream / RSocket Connection

public class Counter4 { private RSocket rSocket; public Counter4() { this.rSocket = RSocketFactory.connect().transport(TcpClientTransport.create(9090)).start().block(); } @Override public Mono<Integer> count() { return rSocket .requestResponse(ByteBufPayload.create(new byte[0])) .map( payload -> { int i = payload.data().readInt(); payload.release(); return i; }); } public static void main(String... args) throws Exception { Counter4 counter = new Counter4(); final Recorder histogram = new Recorder(3600000000000L, 3); Flux.range(1, 100) .concatMap( ignore -> { long start = System.nanoTime(); return counter .count() .doFinally(s -> histogram.recordValue(System.nanoTime() - start)); }) .then() .block(); histogram.getIntervalHistogram().outputPercentileDistribution(System.out, 5, 1000.0, false); }

slide-53
SLIDE 53

Reactive Stream / RSocket Connection

public class Counter4 { private RSocket rSocket; public Counter4() { this.rSocket = RSocketFactory.connect().transport(TcpClientTransport.create(9090)).start().block(); } @Override public Mono<Integer> count() { return rSocket .requestResponse(ByteBufPayload.create(new byte[0])) .map( payload -> { int i = payload.data().readInt(); payload.release(); return i; }); } public static void main(String... args) throws Exception { Counter4 counter = new Counter4(); final Recorder histogram = new Recorder(3600000000000L, 3); Flux.range(1, 100) .concatMap( ignore -> { long start = System.nanoTime(); return counter .count() .doFinally(s -> histogram.recordValue(System.nanoTime() - start)); }) .then() .block(); histogram.getIntervalHistogram().outputPercentileDistribution(System.out, 5, 1000.0, false); }

RSocket

Count Counter Main Counter Count

… …

slide-54
SLIDE 54
  • Temporal Decoupling
  • Spacial Decoupling
  • Binary

  • Application-Flow control

54

High Performance Microservices

slide-55
SLIDE 55
  • Temporal Decoupling - non-blocking api
  • Spacial Decoupling - binary frames sent over dedicated stream
  • Binary - binary payloads access with flyweights 

  • Application-Flow control - flow control information passed via messages

55

RSocket Performance

slide-56
SLIDE 56

Real-world performance?

slide-57
SLIDE 57
  • Acme Air is a a realistic application

benchmark that runs as part of Istio's nightly release pipeline

  • Tests run on a standard 4-node

GCP Kubernetes cluster

  • n1-standard-4 (4 vCPUs, 15

GB)

  • https://ibmcloud-perf.istio.io

Source: IBM Acme Air Key Performance Indicator

slide-58
SLIDE 58
  • Acme Air is a a realistic application

benchmark that runs as part of Istio's nightly release pipeline

  • Tests run on a standard 4-node

GCP Kubernetes cluster

  • n1-standard-4 (4 vCPUs, 15

GB)

  • https://ibmcloud-perf.istio.io

Enterprise

Source: IBM Acme Air Key Performance Indicator Source: Evaluating Critical Performance Needs for Microservices and Cloud-Native

slide-59
SLIDE 59
  • Acme Air is a a realistic application

benchmark that runs as part of Istio's nightly release pipeline

  • Tests run on a standard 4-node

GCP Kubernetes cluster

  • n1-standard-4 (4 vCPUs, 15

GB)

  • https://ibmcloud-perf.istio.io

Enterprise

Source: IBM Acme Air Key Performance Indicator Source: Evaluating Critical Performance Needs for Microservices and Cloud-Native

slide-60
SLIDE 60

info@netifi.com www.netifi.com

Bringing Microservices to the Enterprise