High Performance Actors Kiki, principal enterprise architect @ - - PowerPoint PPT Presentation

▶
high performance actors
SMART_READER_LITE
LIVE PREVIEW

High Performance Actors Kiki, principal enterprise architect @ - - PowerPoint PPT Presentation

Aug 25, 2023 334 likes •988 views

High Performance Actors Kiki, principal enterprise architect @ Lightbend What we mean by Reactive React to React to React to Users Failures Load Variance Low latency Graceful, Non-catastrophic Responsive in the face of Recovery

slide-1
SLIDE 1

High Performance Actors

Kiki, principal enterprise architect @ Lightbend

slide-2
SLIDE 2

The enabler of these characteristics is a Message Driven Model. Responsive Resilient Elastic

React to Users React to Failures React to Load Variance

Low latency Real-time / NRT Graceful, Non-catastrophic Recovery Self-Healing Responsive in the face of changing loads

What we mean by “Reactive”

slide-3
SLIDE 3
  • Fast decisions
  • Fast failures
  • Fast progress with Concurrency &

Parallelism

Message Driven Benefits for Speed and Performance

slide-4
SLIDE 4

Actor B Actor A

Make a local decision

state message handler Ignore This? message

slide-5
SLIDE 5

Actor B Actor A

Make a local decision

state: 1 message handler Do This +2 +2

asynchronous

message

slide-6
SLIDE 6

Actor B Actor A

Make a local decision

state: 3 message handler Do This message +2 +2

asynchronous

slide-7
SLIDE 7

Local decision code

slide-8
SLIDE 8

Actor B Actor A

Change Behavior

state: 3 message handler OK subtract

  • n

message

  • 1

message St Start su subtracting

slide-9
SLIDE 9

Actor B Actor A

Change Behavior

state: 2 message handler Did that… message

  • 1
slide-10
SLIDE 10

Change behavior code

slide-11
SLIDE 11

Change behavior code (cont’d)

slide-12
SLIDE 12

Actor B Actor A

Send more messages

state: 2 message handler

Tell “A” I did that…

message

slide-13
SLIDE 13

Send more messages code

slide-14
SLIDE 14

Actor B Actor A

Or chose not to

state: 3 message handler

…Or not…

¯\_()_/¯

“don’t care”

slide-15
SLIDE 15

Trust but verify

slide-16
SLIDE 16

Actor B Actor A

Create More Actors

state: 3 message handler Actor B.1 Actor B.2 Actor B.3

Go forth!

slide-17
SLIDE 17

Create more actors code

Java

slide-18
SLIDE 18

Create Actors Liberally

Akka actors have a small memory footprint; ~2.5 million actors per GB of heap.

slide-19
SLIDE 19

Akka Abstractions

Akka Streams Akka Cluster Akka Http

slide-20
SLIDE 20

Scalability Cube: 3 dimensions of scalability

Y-axis: scale via functional compositi

  • n

x-axis: horizontal scaling via duplication and load balancing

z-axis: split similar things via data partitioning/sharding

slide-21
SLIDE 21

*never* block *always* asynchronous

slide-22
SLIDE 22

Non-blocking

Actor messaging, default mailbox, work scheduling is non blocking.

slide-23
SLIDE 23

Asynchronous

Actors are asynchronous by nature: an actor can progress after a message send without waiting for the actual delivery to happen.

slide-24
SLIDE 24

Dispatcher

  • Selfless: Doesn’t hog resources
  • Responsible for assigning threads
  • Only assigns when necessary
  • Idle actors do not use threads
  • Dispatcher always paired up with an executor which

will define what kind of thread pool model is used to support the actors

  • CPU caches likely each time actors are assigned

threads and warmed caches are one of your best friend for high performance.

slide-25
SLIDE 25

Fork Join Executor - from 1.4 million to 20 million messages per second

slide-26
SLIDE 26

…and then from 20 million to 50 million messages per second

slide-27
SLIDE 27

throughput vs fairness

slide-28
SLIDE 28
  • CAS compare and swap
  • Atomic values, adders and accumulators

Even faster in actors

slide-29
SLIDE 29

What kinds of machines, with how many cores, will run this application? How CPU- bound are the tasks being performed by my actors? How many threads can I realistically expect to run concurrently for this application on those machines?

Mechanical Sympathy

slide-30
SLIDE 30

Akka Streams

Implementation of Reactive Streams, concerned with applying backpressure when producers are faster than consumers.

slide-31
SLIDE 31

Artery is designed from the ground up to support high-throughput in the magnitude of 1 million messages per second and low- latency in the magnitude of 100 microseconds.

slide-32
SLIDE 32
  • 630,239 messages/s with message payload of 100 bytes
  • 8,245 messages/s with messages payload of 10,000 bytes
  • Round trip latency at a message rate of 10,000 messages/s:

– 50%ile: 155 µs, – 90%ile: 173 µs – 99%ile: 196 µs

Artery Aeron Performance

*two m4.4xlarge EC2 instances (1 Gbit/s bandwidth)

slide-33
SLIDE 33

Aeron efficient, guaranteed transport.

Designed to work with low-latency, high- throughput systems.

slide-34
SLIDE 34

Aeron “Busy Spinning”

Message offers or polls are retried until successful.

slide-35
SLIDE 35

Tuning: CPU Usage vs Latency

# Values can be from 1 to 10, where 10 strongly prefers low latency # and 1 strongly prefers less CPU usage akka.remote.artery.advanced.idle-cpu-level = 1

slide-36
SLIDE 36

Aeron Interaction Pattern

The unidirectional nature of Aeron channels fits nicely with the Akka peer-to-peer communication model.

slide-37
SLIDE 37

ByteBuffer Based Serialization

slide-38
SLIDE 38

ByteBuffer Serializer Interface

slide-39
SLIDE 39

Aeron Message Channels

User Messages Internal Control Messages

slide-40
SLIDE 40

Control stream

slide-41
SLIDE 41

All messages are not created equally

User Messages Internal Control Messages Optional: Large Messages

slide-42
SLIDE 42

Actor Path Compression

slide-43
SLIDE 43

Actor Path Compression

Message!

slide-44
SLIDE 44

Receiver initiates algorithm Advertises compression table Remote sender uses advertised table

Compression Algorithm

slide-45
SLIDE 45

Built-in Flight-Recorder

slide-46
SLIDE 46
  • Fixed size file
  • This file is crash resistant
  • Very low overhead

Flight Recorder

slide-47
SLIDE 47

Akka Cluster

slide-48
SLIDE 48
  • Gossip Protocol
  • Adaptive load balancing

Akka Cluster

slide-49
SLIDE 49

Akka Cluster: Gossip

The cluster membership used in Akka is based

  • n Amazon’s Dynamo system and particularly

the approach taken in Basho’s’ Riak distributed database.

slide-50
SLIDE 50

Akka Cluster: Gossip

The cluster membership used in Akka is based

  • n Amazon’s Dynamo system and particularly

the approach taken in Basho’s’ Riak distributed database.

slide-51
SLIDE 51

Akka Cluster: Gossip Convergence

Information about the cluster converges locally at a node at certain points in time.

slide-52
SLIDE 52

Akka Cluster: Gossip Protocol

A variation of push-pull gossip is used to reduce the amount of gossip information sent around the cluster.

slide-53
SLIDE 53

Akka Cluster: Sharding performance

Cluster Sharding can be scaled in a nearly linear

  • fashion. It can easily handle millions of Actors without

much regression in messaging speed or message loss.

slide-54
SLIDE 54

20 cluster nodes and 10,000 clustered actors. First we measured the latency given a message throughput of 100 pings per second, and then given a throughput of 500 pings per second.

slide-55
SLIDE 55

Akka Cluster: Adaptive Load Balancing

The AdaptiveLoadBalancingPool / AdaptiveLoadBalancingGro up performs load balancing of messages to cluster nodes based on the cluster metrics data.

slide-56
SLIDE 56

Akka Cluster: Adaptive Load Balancing

http://blog.kamkor.me/Akka-Cluster-Load-Balancing/

slide-57
SLIDE 57

A user’s story

slide-58
SLIDE 58

"Powered by Akka and Scala, squbs has already provided very high-scale results with a low infrastructure footprint:

  • ur applications are able to serve over a billion hits a day with as

little as 8 VMs and 2 vCPU each."

PayPal

slide-59
SLIDE 59

"Akka helps our systems stay responsive even at 90% CPU, very uncharacteristic for our older architectures, and provides for transaction densities never seen before. Batches or micro-batches do their jobs in one-tenth of the time it took before. With wider adoption, we will see this kind of technology being able to reduce cost and allow for much better organizational growth without growing the compute infrastructure accordingly."

PayPal

slide-60
SLIDE 60

"Not only do we achieve much larger transaction rates with the same hardware infrastructure, but we also often cut down code by 80% for the same functionality when compared to equivalent imperative code."

PayPal

slide-61
SLIDE 61

LinkedIn

Presence Platform display real-time presence status for a member’s connections across LinkedIn messaging, feed, notifications, etc. on both mobile and web for hundreds of millions of LinkedIn members across the globe.

slide-62
SLIDE 62

Is it really so Difficult?

Akka makes the difficult possible

slide-63
SLIDE 63

How can I be confident?

  • The Actor model is proven
  • Akka is designed from ground up for

performance on the JVM

  • Akka scales on all axis X, Y, Z
  • Other companies have benefited from Akka
  • You can confidently focus on your core

competencies while using Akka.

slide-64
SLIDE 64
  • Akka.io / akka guides
  • Talk to users, gitter chat
  • Tweet the akka team: @akkateam
  • For fun tweet me: @kikisworldrace

Getting started with Akka