Async or Bust!? Todd L. Montgomery @toddlmontgomery About me The - - PowerPoint PPT Presentation

▶
async or bust
SMART_READER_LITE
LIVE PREVIEW

Async or Bust!? Todd L. Montgomery @toddlmontgomery About me The - - PowerPoint PPT Presentation

Apr 24, 2023 156 likes •1.17k views

Async or Bust!? Todd L. Montgomery @toddlmontgomery About me The Myth(s) Illusion & Cognitive Dissonance Impact of the Myth Subjectivity of the Myth Do you remember. https://www.youtube.com/watch?v=bzkRVzciAZg The

slide-1
SLIDE 1

Async or Bust!?

Todd L. Montgomery @toddlmontgomery

slide-2
SLIDE 2

About me…

slide-3
SLIDE 3

✓ The Myth(s) ✓ Illusion & Cognitive Dissonance ✓ Impact of the Myth ✓ Subjectivity of the Myth

slide-4
SLIDE 4

Do you remember….

slide-5
SLIDE 5

https://www.youtube.com/watch?v=bzkRVzciAZg

slide-6
SLIDE 6

The Myth Sequential is good enough

slide-7
SLIDE 7

The Myth Sequential is good enough …. Async is complicated & error prone

slide-8
SLIDE 8

A Right Way & MANY Wrong Ways

slide-9
SLIDE 9

MANY Right Ways & MANY Wrong Ways In Reality…

slide-10
SLIDE 10

Sequential Synchronous Blocking Asynchronous Non-Blocking Wording

slide-11
SLIDE 11

What is Sync?

Request Response Processing

slide-12
SLIDE 12

What is Async?

Request Response Processing

slide-13
SLIDE 13

What is Async?

Request Response Processing Cross Thread/Core/Node

slide-14
SLIDE 14

What is Async?

Request Response Processing

?

slide-15
SLIDE 15

What is Event-Driven?

Request Response

?

slide-16
SLIDE 16

Illusion of Sequentiality

slide-17
SLIDE 17

Ordering is an Illusion

slide-18
SLIDE 18

Compiler can re-order Runtime can re-order CPU can re-order

slide-19
SLIDE 19

Ordering has to be imposed!

slide-20
SLIDE 20

Illusion of Sequentiality

slide-21
SLIDE 21
  • CPUs - Load/Store Buffers

Illusion of Sequentiality

slide-22
SLIDE 22
  • CPUs - Load/Store Buffers
  • Storage - Caches

Illusion of Sequentiality

slide-23
SLIDE 23
  • CPUs - Load/Store Buffers
  • Storage - Caches
  • OS - VM & Caches

Illusion of Sequentiality

slide-24
SLIDE 24
  • CPUs - Load/Store Buffers
  • Storage - Caches
  • OS - VM & Caches
  • Library - Promises / Futures

Illusion of Sequentiality

slide-25
SLIDE 25

@toddlmontgomery

Sync Requests & Responses Request Request Request Response Response Response

As easy* as…

* - for some definition of

slide-26
SLIDE 26

@toddlmontgomery

Async Requests & Responses Request Request Request Response Response Response

But with efficiency of…

slide-27
SLIDE 27

Do something while waiting

slide-28
SLIDE 28

@toddlmontgomery

Async Requests & Responses Correlation! Request 0 Request 1 Request 2 Response 0 Response 1 Response 2

slide-29
SLIDE 29

@toddlmontgomery

Correlation! Request 0 Request 1 Request 2 Response 0 Response 1 Response 2 Ordering

slide-30
SLIDE 30

@toddlmontgomery

Correlation! Request 0 Request 1 Request 2 Response 0 Response 1 Response 2 (Valid) Re-Ordering (one of many)

slide-31
SLIDE 31

The key is to wait… That has a price!

slide-32
SLIDE 32
  • Opportunity to De-Schedule

Price of Illusion

slide-33
SLIDE 33
  • Opportunity to De-Schedule
  • Locks + Signaling

Price of Illusion

slide-34
SLIDE 34
  • Opportunity to De-Schedule
  • Locks + Signaling
  • Semaphores
  • Condition Variables

Price of Illusion

slide-35
SLIDE 35

Cognitive Dissonance

slide-36
SLIDE 36
  • Completed Operation Fallacy

Cognitive Dissonance

slide-37
SLIDE 37
  • Completed Operation Fallacy
  • Caching

Cognitive Dissonance

slide-38
SLIDE 38

@toddlmontgomery

Flush … Request Response Block

Caches

slide-39
SLIDE 39

@toddlmontgomery

Flush … Request Response Block

Caches

…?

slide-40
SLIDE 40

@toddlmontgomery

Flush … Request Response Block

Caches

Stable… ? …?

slide-41
SLIDE 41
  • Completed Operation Fallacy
  • Caching
  • Blocking ACK Spiral

Cognitive Dissonance

slide-42
SLIDE 42

@toddlmontgomery

Block Request Response ACK Response Block

Blocking ACK

slide-43
SLIDE 43

@toddlmontgomery

Block Request Response ACK Response Block

Blocking ACK

X

slide-44
SLIDE 44

@toddlmontgomery

Block Request Response ACK Response Block

Blocking ACK

X

Receiver Blocks Forever

slide-45
SLIDE 45

@toddlmontgomery

Block Request Response ACK Response Spiral!!! Block

Blocking ACK

Response ACK-ACK

slide-46
SLIDE 46
  • Completed Operation Fallacy
  • Caching
  • Blocking ACK Spiral
  • Wrong Abstraction
  • Remote Procedure Call

Cognitive Dissonance

slide-47
SLIDE 47

“The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise” — Edsger W. Dijkstra (The Humble Programmer) Abstraction

slide-48
SLIDE 48
  • Hiding precision
  • Inherent asynchronous nature
  • Error handling

Remote Procedure Call

slide-49
SLIDE 49

Don’t assume the network is reliable

https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

Remote Procedure Call

slide-50
SLIDE 50

“Yeah, yeah, but your scientists were so preoccupied with whether or not they could that they didn't stop to think if they should.” — Jurassic Park Remote Procedure Call

slide-51
SLIDE 51

Works sooo poorly, we took it one step further…

slide-52
SLIDE 52

REST via HTTP/1.1

  • Custom Methods
  • Custom Response Codes
  • No Pipelining
  • Everything Request/Response
slide-53
SLIDE 53

REST via HTTP/2

  • Custom Methods
  • Custom Error Codes
  • Custom Frame Types
  • No Pipelining
  • Mostly Request/Response
slide-54
SLIDE 54
  • Completed Operation Fallacy
  • Caching
  • Blocking ACK Spiral
  • Wrong Abstraction
  • Remote Procedure Call
  • Coupling

Cognitive Dissonance

slide-55
SLIDE 55

Sequential function calls can and do create Coupling

slide-56
SLIDE 56

Impact

slide-57
SLIDE 57

@toddlmontgomery

Sync Requests & Responses Request Request Request Response Response Response

slide-58
SLIDE 58

@toddlmontgomery

Sync Requests & Responses Request Request Request Response Response Response Throughput limited by Round-Trip Time (RTT)

slide-59
SLIDE 59

Speed of Light isn’t only a good idea, it’s the Law

slide-60
SLIDE 60

Accumulated Improvement Time Network Bandwidth Response Time Storage Capacity CPU Cores Memory Capacity

slide-61
SLIDE 61

@toddlmontgomery

Data Data Data ACK ACK ACK Throughput = Data Length / RTT RTT Stop-And-Wait Flow Control

slide-62
SLIDE 62
slide-63
SLIDE 63

Delay Bandwidth BDP = (Byte / sec) * sec = Bytes BDP (Buffer)

slide-64
SLIDE 64

@toddlmontgomery

Data ACK RTT Throughput = N * Data Length / RTT … N Data “Blobs”

slide-65
SLIDE 65

So… How big is N?

slide-66
SLIDE 66

N = Number of Cores Thread-Per-Request

slide-67
SLIDE 67

How big is N? TCP Flow & Congestion Control

slide-68
SLIDE 68

How big is N? It depends… TCP Flow & Congestion Control

slide-69
SLIDE 69

Big… but Don’t overflow receiver Don’t overflow “network”

slide-70
SLIDE 70

TCP Flow Control Receiver advertises N

slide-71
SLIDE 71

TCP Congestion Control Sender probes for network N

slide-72
SLIDE 72

TCP BBR Congestion Control Bottleneck Bandwidth vs. Round-Trip Time

http://queue.acm.org/detail.cfm?id=3022184

slide-73
SLIDE 73

TCP Sender min(Receiver N, Network N)

Only go as fast as Network & Receiver

slide-74
SLIDE 74

Static N? Based on number of cores(threads)? REALLY?!

slide-75
SLIDE 75

But that isn’t the worst…

slide-76
SLIDE 76

Locks & Signaling

  • Introduces Serialization
slide-77
SLIDE 77

"AmdahlsLaw" by Daniels220 at English Wikipedia - Own work based on: File:AmdahlsLaw.png. Licensed under CC BY-SA 3.0 via Wikimedia Commons

slide-78
SLIDE 78

Locks & Signaling

  • Introduces Serialization
  • Introduces Coherence Penalty
slide-79
SLIDE 79

Universal Scalability Law

2 4 6 8 10 12 14 16 18 20 1 2 4 8 16 32 64 128 256 512 1024

Speedup Processors

Amdahl USL

slide-80
SLIDE 80

Locks & Signaling

  • Introduces Serialization
  • Introduces Coherence Penalty

Limits Scaling!

slide-81
SLIDE 81

1 thread of awesome > 128 cores of so-so

http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.html http://blog.acolyer.org/2015/06/05/scalability-but-at-what-cost/

slide-82
SLIDE 82

Async is HARD!!!

slide-83
SLIDE 83
  • Callback Hell
  • Back Pressure!

Async is HARD!!

slide-84
SLIDE 84

Composition is hard

slide-85
SLIDE 85

ReactiveX

http://reactivex.io/

slide-86
SLIDE 86

Observables

slide-87
SLIDE 87

JavaScript

  • RxJS
  • ECMAScript Observables

https://github.com/ReactiveX/RxJS https://github.com/zenparsing/es-observable

slide-88
SLIDE 88

Challenges?

slide-89
SLIDE 89
  • Non-Blocking Back Pressure
  • Heterogeneous Connectivity

Challenges

slide-90
SLIDE 90

Dealing with Back Pressure

  • ReactiveStreams
  • RxJava 2.0

http://www.reactive-streams.org/

slide-91
SLIDE 91

Rx Heterogenous Connectivity

  • ReactiveSocket

http://reactivesocket.io/

slide-92
SLIDE 92
  • Callback Hell
  • Back Pressure!
  • Breaking up work units?

Async is HARD!!

slide-93
SLIDE 93

Threaded Work Units

  • Work between System Calls
slide-94
SLIDE 94

Threaded Work Units

  • Work between System Calls
  • Time between System Calls

High Variance

slide-95
SLIDE 95

Async Duty Cycle

  • Work within a single cycle

First Class Concern

slide-96
SLIDE 96
  • Callback Hell
  • Back Pressure!
  • Duty Cycle
  • Error Handling

Async is HARD!!

slide-97
SLIDE 97

Error Handling

  • Errors are events

No real difference!!

slide-98
SLIDE 98

Error Handling Be Honest…

slide-99
SLIDE 99

Takeaways!

slide-100
SLIDE 100

Still Think… Sequential is good enough? …. Async is complicated & error prone?

slide-101
SLIDE 101

@toddlmontgomery

Questions?

  • Twitter @toddlmontgomery

Thank You!