Bimodal Multicast And Cache Invalidation Who/What/Where Bruce - - PowerPoint PPT Presentation

▶
bimodal multicast
SMART_READER_LITE
LIVE PREVIEW

Bimodal Multicast And Cache Invalidation Who/What/Where Bruce - - PowerPoint PPT Presentation

Apr 24, 2023 154 likes •1.01k views

Bimodal Multicast And Cache Invalidation Who/What/Where Bruce Spang Software Engineer/Student Fastly ` Powderhorn Outline Frame the problem The papers we looked at Bimodal Multicast What we built Content Delivery

slide-1
SLIDE 1

Bimodal Multicast

And Cache Invalidation

slide-2
SLIDE 2

Who/What/Where

  • Bruce Spang
  • Software Engineer/Student
  • Fastly
slide-3
SLIDE 3

`

slide-4
SLIDE 4

Powderhorn

slide-5
SLIDE 5

Outline

  • Frame the problem
  • The papers we looked at
  • Bimodal Multicast
  • What we built
slide-6
SLIDE 6

Content Delivery Networks

slide-7
SLIDE 7
slide-8
SLIDE 8
slide-9
SLIDE 9
slide-10
SLIDE 10
slide-11
SLIDE 11
slide-12
SLIDE 12
slide-13
SLIDE 13
slide-14
SLIDE 14

Cache Invalidation

We would like to be able to update a piece of content globally

slide-15
SLIDE 15

Cache Invalidation

slide-16
SLIDE 16

Cache Invalidation

slide-17
SLIDE 17

Cache Invalidation

slide-18
SLIDE 18

Approach

Notify all servers to remove a piece of content

slide-19
SLIDE 19

Naïve Approach

Central Service.

slide-20
SLIDE 20

Central Service

M e s s a g e

slide-21
SLIDE 21

Central Service

A c k

slide-22
SLIDE 22

Central Service

slide-23
SLIDE 23

Central Service

M e s s a g e

slide-24
SLIDE 24

Central Service

A c k

slide-25
SLIDE 25

Central Service

Works

slide-26
SLIDE 26

Problems

  • Unreliable
  • High Latency
slide-27
SLIDE 27

Another Idea

Cache servers can send purges themselves

slide-28
SLIDE 28

Fixes

  • More Reliable
  • Lower Latency
slide-29
SLIDE 29

Another Idea

M e s s a g e

slide-30
SLIDE 30

Another Idea

A c k

slide-31
SLIDE 31

Problem

Every server sends an ack to the sender

slide-32
SLIDE 32

Conclusion

This is hard.

slide-33
SLIDE 33

Reliable Broadcast

slide-34
SLIDE 34

Basic Problem

Send a message to a set of servers

slide-35
SLIDE 35

Applications

  • Stock Exchanges
  • Control Systems
  • Configuration
  • Cache Invalidation
slide-36
SLIDE 36

Atomic Broadcast

  • Paxos
  • ZooKeeper Atomic Broadcast
  • etc…
slide-37
SLIDE 37

Strong Guarantees

  • Guaranteed Delivery
  • Total Order
slide-38
SLIDE 38

Too Many Guarantees

  • Don’t need Ordering
  • Don’t need Atomicity
slide-39
SLIDE 39

“Best-Effort” Broadcast

“Try very hard” to deliver a message

slide-40
SLIDE 40

Algorithms

  • Scalable Reliable Multicast
  • Bimodal Multicast
  • Plumtree
  • Sprinkler
  • etc…
slide-41
SLIDE 41

`

`

slide-42
SLIDE 42

Goals

  • “Best-Effort” Delivery
  • Predictable Performance
slide-43
SLIDE 43

Algorithm

  • Dissemination
  • Anti-Entropy (Gossip)
slide-44
SLIDE 44

Dissemination

  • Unreliably broadcast a message to all other servers
  • IP multicast, UDP in a for loop, etc…
slide-45
SLIDE 45

Anti-Entropy

  • Each server sends a random server a digest of the

messages it knows about

  • If the other server hasn’t received some of those

messages, it requests that they be retransmitted

slide-46
SLIDE 46

Example

slide-47
SLIDE 47

Example

slide-48
SLIDE 48

Example

slide-49
SLIDE 49
slide-50
SLIDE 50

Gossip

slide-51
SLIDE 51

Convergence

slide-52
SLIDE 52

Goals

  • “Best-Effort” Delivery
  • Predictable Performance
slide-53
SLIDE 53

Stable Throughput

M e s s a g e

slide-54
SLIDE 54

Stable Throughput

Ack

slide-55
SLIDE 55

Stable Throughput

M e s s a g e

slide-56
SLIDE 56

Stable Throughput

R e s e n d

slide-57
SLIDE 57

Independence

A server that’s behind will recover from many servers in the cluster.

slide-58
SLIDE 58

Retransmission Limits

Don’t DDoS servers that are trying to recover.

slide-59
SLIDE 59

Soft Failure Detection

Ignore servers that are running behind.

slide-60
SLIDE 60
slide-61
SLIDE 61

“Best effort” is ok!

  • Messages are delivered
  • Predicable Performance
slide-62
SLIDE 62

Powderhorn

Bimodal Multicast in the Wild

slide-63
SLIDE 63

Development

All the logic is in failure handling.

slide-64
SLIDE 64

Jepsen

  • Five nodes
  • Simulated partitions, packet loss
  • http://github.com/aphyr/jepsen
slide-65
SLIDE 65

“Scale” Testing

  • EC2
  • 200 m1.mediums
slide-66
SLIDE 66

Avalanche

  • Random, repeatable network fault injection
  • https://github.com/fastly/avalanche
slide-67
SLIDE 67

Garbage Collection

“I have messages {1,2,3,4,5,6,7,8,9,…}”

slide-68
SLIDE 68

Garbage Collection

slide-69
SLIDE 69

Computers are Horrible

We see high packet loss and network partitions all the time.

slide-70
SLIDE 70

Convergence

< 40%

slide-71
SLIDE 71

The Digest

List of Message IDs

slide-72
SLIDE 72

The Digest

Doesn’t Have to be a List

slide-73
SLIDE 73

The Digest

  • send ranges of ids of known messages
  • “messages 1 to 1,000,000 from server 1"
  • can represent large numbers of messages
slide-74
SLIDE 74

The Digest

slide-75
SLIDE 75

Behavior

slide-76
SLIDE 76

End-to-End Latency

42ms 74ms 83ms 133ms

New York London San Jose Tokyo

0.00 0.05 0.10 0.00 0.05 0.10 0.00 0.05 0.10 0.00 0.05 0.10 50 100 150

Latency (ms) Density

Density plot and 95th percentile of purge latency by server location

slide-77
SLIDE 77

Firewall Partition

30 60 90 120 02:30 03:00 03:30 04:00

Time Throughput (messages/s)

0.1 1 10 60 02:30 03:00 03:30 04:00

Time 95th percentile latency (s)

Cache server A B C D

Purge performance under network partition

slide-78
SLIDE 78

NYC to London Partition

30 60 90 120 06:00 06:10 06:20 06:30

Time Throughput (messages/s)

5 10 15 06:00 06:10 06:20 06:30

Time Recovered purges (messages/s)

Cache server NYC London

Purge performance

slide-79
SLIDE 79

APAC Packet Loss

100 200 300 400 16:30 17:00 17:30 18:00

Time Throughput (messages/s)

25 50 75 100 125 16:30 17:00 17:30 18:00

Time Recovered purges (messages/s)

Cache server Affected Unaffected

Purge performance

slide-80
SLIDE 80

DDoS

  • `

50 100 150 23:40 23:50 00:00 00:10 00:20

Time Throughput (messages/s)

0.1 1 10 60 23:40 23:50 00:00 00:10 00:20

Time 95th percentile latency (s)

Cache server Victim Unaffected

Purge performance under denial−of−service attack

slide-81
SLIDE 81

Bimodal Multicast is Great

We generally don’t have to worry about purging failing, even when the network does.

slide-82
SLIDE 82

Fin.

brucespang.com/bimodal

slide-83
SLIDE 83

Questions?

brucespang.com/bimodal

slide-84
SLIDE 84

We're Hiring

www.fastly.com/about/jobs

slide-85
SLIDE 85

Decent Hash Table

5 10 May 07 May 08 May 09 May 10 May 11 May 12 May 13

Date 95th percentile latency (ms)

Purge performance with linear probing hash−table