Shadow: The Case for Determinism on the Edge Matthew Furlong , - - PowerPoint PPT Presentation

▶
shadow the case for determinism on the edge
SMART_READER_LITE
LIVE PREVIEW

Shadow: The Case for Determinism on the Edge Matthew Furlong , - - PowerPoint PPT Presentation

Apr 30, 2023 252 likes •879 views

Slide Title Shadow: The Case for Determinism on the Edge Matthew Furlong , Andrew Quinn, and Jason Flinn Concerns for the Edge Edge Concern Effect Problem Concerns for the Edge Edge Concern Effect Problem Provisioning More jitter

slide-1
SLIDE 1

Slide Title

Shadow: The Case for Determinism on the Edge

Matthew Furlong, Andrew Quinn, and Jason Flinn

slide-2
SLIDE 2

Edge Concern Effect Problem

Concerns for the Edge

slide-3
SLIDE 3

Concerns for the Edge

Edge Concern Effect Problem Provisioning More jitter

slide-4
SLIDE 4

Edge Concern Effect Problem Provisioning More jitter

  • Limited edge resources

Concerns for the Edge

slide-5
SLIDE 5

[1] Lee, H., Flinn, J., And Tonshal, B. Raven: Improving interactive latency for the connected car.

Edge Concern Effect Problem Provisioning More jitter

  • Limited edge resources
  • Mobile network variability

High tail latency!

Concerns for the Edge

slide-6
SLIDE 6

Edge Concern Effect Problem Provisioning More jitter

  • Limited edge resources
  • Mobile network variability

Mobility Performance variability

Concerns for the Edge

slide-7
SLIDE 7

Edge Concern Effect Problem Provisioning More jitter

  • Limited edge resources
  • Mobile network variability

Mobility Performance variability

  • Low-latency connections are lost

Concerns for the Edge

slide-8
SLIDE 8

Edge Concern Effect Problem Provisioning More jitter

  • Limited edge resources
  • Mobile network variability

Mobility Performance variability

  • Low-latency connections are lost

Reliability More faults

Concerns for the Edge

slide-9
SLIDE 9

Edge Concern Effect Problem Provisioning More jitter

  • Limited edge resources
  • Mobile network variability

Mobility Performance variability

  • Low-latency connections are lost

Reliability More faults

  • Numerous providers

Concerns for the Edge

slide-10
SLIDE 10

Edge Concern Effect Problem Provisioning More jitter

  • Limited edge resources
  • Mobile network variability

Mobility Performance variability

  • Low-latency connections are lost

Reliability More faults

  • Numerous providers
  • Distributed

Concerns for the Edge

slide-11
SLIDE 11

Edge Concern Effect Problem Provisioning More jitter

  • Limited edge resources
  • Mobile network variability

Mobility Performance variability

  • Low-latency connections are lost

Reliability More faults

  • Numerous providers
  • Distributed

More data sources Privacy concerns

Concerns for the Edge

slide-12
SLIDE 12

Edge Concern Effect Problem Provisioning More jitter

  • Limited edge resources
  • Mobile network variability

Mobility Performance variability

  • Low-latency connections are lost

Reliability More faults

  • Numerous providers
  • Distributed

More data sources Privacy concerns

  • More IoT devices = more sensitive user data

Concerns for the Edge

slide-13
SLIDE 13
  • Applications are typically latency-sensitive or require

high bandwidth

Moving From the Cloud to Edge

slide-14
SLIDE 14
  • Applications are typically latency-sensitive or require

high bandwidth

Moving From the Cloud to Edge

AR App Torch OpenCV

Edge

AR App Torch OpenCV AR App Torch OpenCV AR App Torch OpenCV

slide-15
SLIDE 15

Outline

  • Motivation
  • Determinism to the rescue
  • Shadow: Deterministic containers
  • Evaluation
slide-16
SLIDE 16
  • A deterministic execution of an application

provides the necessary support for edge systems

Determinism to the Rescue

S0 S1 1 0/1

Deterministic Non-Deterministic

S0 S1 1 0/1 1 S2 0/1

slide-17
SLIDE 17
  • Determinism allows applications to be

replicated across multiple servers

Determinism to the Rescue

slide-18
SLIDE 18
  • Determinism allows applications to be

replicated across multiple servers

  • Determinism enables migration with minimal

downtime

Determinism to the Rescue

slide-19
SLIDE 19

Migration

Low-latency connection

1 Server inputs: a Server Responses:

slide-20
SLIDE 20

Migration

Better connection discovered

1 Server inputs: a Server Responses:

slide-21
SLIDE 21

Migration

Initiate migration!

1 Server Responses: Server inputs: a

slide-22
SLIDE 22

Migration

1 Server inputs: a b

Checkpoint Application

Server inputs: a b ? Server Responses:

slide-23
SLIDE 23

Migration

1 Server inputs: a

Multicast client input

b b

Transfer Checkpoint

Server inputs: a b b 2 Server Responses:

slide-24
SLIDE 24

Migration

1 Server inputs: a

Multicast client input

c c

Transfer Checkpoint

Server inputs: a b b 2 c c 3 Server Responses:

slide-25
SLIDE 25

Migration

1 Server inputs: a

Multicast client input

d d Server inputs: a b b 2 c c 3

Restore Application

d d Server Responses: 4

slide-26
SLIDE 26

e

Migration

1 Server inputs: a

Multicast client input

e e Server inputs: a b b 2 c c 3 d d 4 5 e Server Responses:

slide-27
SLIDE 27

e

Migration

1 Server inputs: a

Multicast client input

f f Server inputs: a b b 2 c c 3 d d 4 5 e f f 6 Server Responses:

slide-28
SLIDE 28

e

Migration

1 Client inputs: Server inputs: a

Migration complete!

g Server inputs: a b b 2 c c 3 d d 4 5 e f f 6

slide-29
SLIDE 29

Determinism to the Rescue

Edge Concern Problem Solution Provisioning

  • Limited edge resources
  • Mobile network variability

Determinism Mobility

  • Low-latency connections are lost

Determinism Reliability

  • Numerous providers
  • Distributed

Determinism More data sources • More IoT devices = more sensitive user data Determinism

slide-30
SLIDE 30

Determinism to the Rescue

Edge Concern Problem Solution Provisioning

  • Limited edge resources
  • Mobile network variability

Determinism Mobility

  • Low-latency connections are lost

Determinism Reliability

  • Numerous providers
  • Distributed

Determinism More data sources • More IoT devices = more sensitive user data Determinism

Challenge: Applications are inherently non- deterministic!

slide-31
SLIDE 31
  • Non-determinism in applications
  • Randomness
  • Time
  • Multi-threading
  • Network communication
  • Inter-process communication (IPC)

Determinism to the Rescue

slide-32
SLIDE 32
  • Non-determinism in applications
  • Randomness
  • Time
  • Multi-threading
  • Network communication
  • Inter-process communication (IPC)
  • Challenge: How can we make non-deterministic

applications behave deterministically??

Determinism to the Rescue

slide-33
SLIDE 33

Outline

  • Motivation
  • Determinism to the rescue
  • Shadow: Deterministic containers
  • Evaluation
slide-34
SLIDE 34
  • Goal: Eliminate sources of non-determinism

Shadow

slide-35
SLIDE 35
  • Goal: Eliminate sources of non-determinism
  • Minimize application modifications

Shadow

slide-36
SLIDE 36
  • Goal: Eliminate sources of non-determinism
  • Minimize application modifications
  • Maintain similar performance to non-deterministic

executions

Shadow

slide-37
SLIDE 37
  • Shadow removes non-determinism from

applications deployed within containers

  • Multi-threading
  • OS Events
  • Client inputs
  • External Sources of Non-Determinism

Deterministic Containers

slide-38
SLIDE 38
  • Shadow uses a deterministic thread scheduler

based on weak determinism1

Deterministic Thread Scheduler

[1] Olseweski, M., Ansel, J., And Amarsinghe, S. Kendo: Efficient deterministic multithreading in software.

slide-39
SLIDE 39
  • Shadow uses a deterministic thread scheduler

based on weak determinism1

  • Weak determinism globally orders all

synchronization operations

Deterministic Thread Scheduler

[1] Olseweski, M., Ansel, J., And Amarsinghe, S. Kendo: Efficient deterministic multithreading in software.

slide-40
SLIDE 40
  • Shadow removes non-determinism from

applications deployed within containers

  • Multi-threading
  • OS Events
  • Client inputs
  • External Sources of Non-Determinism

Deterministic Containers

slide-41
SLIDE 41
  • System Calls
  • getrandom()
  • gettimeofday()
  • getpid()
  • …

Operating System Events

slide-42
SLIDE 42
  • Shadow removes non-determinism from

applications deployed within containers

  • Multi-threading
  • OS Events
  • Client inputs
  • External Sources of Non-Determinism

Deterministic Containers

slide-43
SLIDE 43
  • Determinism is only useful if replicas receive the

same sequence of inputs

Client Inputs

slide-44
SLIDE 44
  • Determinism is only useful if replicas receive the

same sequence of inputs

Client Inputs

Client multicasts inputs

slide-45
SLIDE 45
  • Determinism is only useful if replicas receive the

same sequence of inputs

Client Inputs

Inputs are delivered in epochs to each server

slide-46
SLIDE 46
  • Determinism is only useful if replicas receive the

same sequence of inputs

External Sources

A B

slide-47
SLIDE 47
  • Determinism is only useful if replicas receive the

same sequence of inputs

External Sources

3rd party service responses are not identical

A B

slide-48
SLIDE 48
  • Determinism is only useful if replicas receive the

same sequence of inputs

External Sources

slide-49
SLIDE 49
  • Determinism is only useful if replicas receive the

same sequence of inputs

External Sources

Forward request from client a a

slide-50
SLIDE 50
  • Determinism is only useful if replicas receive the

same sequence of inputs

External Sources

a

A

a

slide-51
SLIDE 51
  • Determinism is only useful if replicas receive the

same sequence of inputs

External Sources

a

A A

Client caches response

a

slide-52
SLIDE 52
  • Determinism is only useful if replicas receive the

same sequence of inputs

External Sources

a

A

slide-53
SLIDE 53
  • Determinism is only useful if replicas receive the

same sequence of inputs

External Sources

a

A A

slide-54
SLIDE 54
  • Determinism is only useful if replicas receive the

same sequence of inputs

External Sources

a

A A

3rd party services can produce non- deterministic results and still work with Shadow!

slide-55
SLIDE 55

Determinism Summary

Non-Determinisim Solution Application Limitations Multi-threading Weak determinism No data races OS Events Deterministic syscalls Time skew No IPC or kernel devices Client Inputs Use client as primary None External Sources Use client as primary None

slide-56
SLIDE 56

Outline

  • Motivation
  • Determinism to the rescue
  • Shadow: Deterministic containers
  • Evaluation
slide-57
SLIDE 57

Evaluation

  • Test Application
  • Vehicle Caravan
slide-58
SLIDE 58
  • Emulate suburban network traces from VNperf1
  • Evaluate quality of experience for the client

using response time from server

Jitter Results

[1] Lee, H., Flinn, J., And Tonshal, B. Raven: Improving interactive latency for the connected car.

Non-Det Non-Det Det Det

slide-59
SLIDE 59

Jitter Results

Vehicle Caravan

slide-60
SLIDE 60

Jitter Results

Vehicle Caravan

Shadow reduces the median response by up to 25 percent

slide-61
SLIDE 61

Discussion

  • Feedback
  • Are Shadow’s limitations reasonable to place on applications?
  • How prevalent does the community think single-client

applications will be in edge deployments?

  • What other applications could benefit from a deterministic

environment?

  • Controversial points
  • Applications must fit into Shadow’s restrictions (e.g. no IPC,

clock skew, race-free)

  • Everything falls apart…
  • Performance overhead of determinism outweighs the benefits
  • If the majority of applications cannot be modified to fit into

Shadow’s restrictions