The State of Serverless Computing or, Fixing - - PowerPoint PPT Presentation

▶
the state of serverless computing
SMART_READER_LITE
LIVE PREVIEW

The State of Serverless Computing or, Fixing - - PowerPoint PPT Presentation

Jul 13, 2023 49 likes •552 views

The State of Serverless Computing or, Fixing Dysfunction-as-a-Service Chenggang Wu RISE Lab, UC Berkeley QCon New York 06/24/2019 UC Berkeley: CS 61A Fall 2018 The State of Serverless Computing QCon New York 06/24/2019 The State of

slide-1
SLIDE 1

The State of Serverless Computing

  • r, Fixing Dysfunction-as-a-Service

Chenggang Wu RISE Lab, UC Berkeley QCon New York 06/24/2019

slide-2
SLIDE 2

UC Berkeley: CS 61A Fall 2018

The State of Serverless Computing QCon New York 06/24/2019

slide-3
SLIDE 3

The State of Serverless Computing QCon New York 06/24/2019 The State of Serverless Computing QCon New York 06/24/2019

slide-4
SLIDE 4

Making Programmers Productive

Key Question: Where will code be run? In the ☁!

The State of Serverless Computing QCon New York 06/24/2019

slide-5
SLIDE 5

Background: Serverless Computing

The State of Serverless Computing QCon New York 06/24/2019

slide-6
SLIDE 6

What is serverless computing?

Serverless computing is a programming abstraction that enables users to upload programs, run them at any scale, and pay only for resources used.

The State of Serverless Computing QCon New York 06/24/2019

slide-7
SLIDE 7

Functions-as-a-Service (FaaS)

  • AWS Lambda, Google Cloud Functions, OpenWhisk (IBM), Azure

Functions, OpenLambda, OpenFaaS, kNative…

  • Optimized for simplicity – register functions, enable triggers, and scale

transparently

The State of Serverless Computing QCon New York 06/24/2019

slide-8
SLIDE 8

The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary

Academic Interest in Serverless

slide-9
SLIDE 9

Industrial Interest in Serverless

The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary

slide-10
SLIDE 10

What is FaaS good at today?

f() f() f() f()

Embarrassingly parallel tasks Workflow orchestration

The State of Serverless Computing QCon New York 06/24/2019

slide-11
SLIDE 11

Wait! What about…

  • There are other serverless services, too!
  • e.g., Google Cloud Dataflow, AWS Athena,

Snowflake…

  • Often referred to as Backend-as-a-Service

(BaaS)

We’re primarily interested in generality.

The State of Serverless Computing QCon New York 06/24/2019

slide-12
SLIDE 12

Limitations on FaaS Today

Limited execution lifetimes No inbound network connections

x

IO is a bottleneck No specialized hardware

The State of Serverless Computing QCon New York 06/24/2019

slide-13
SLIDE 13

But that’s okay: Everything is functional!

  • Functional programs don’t have side effects or mutable state!
  • And it is called AWS Lambda

The State of Serverless Computing QCon New York 06/24/2019

slide-14
SLIDE 14

Dysfunction-as-a-Service

  • FaaS is not designed for functional programming because real

applications share state

  • FaaS is poorly suited for all of these

The State of Serverless Computing QCon New York 06/24/2019

slide-15
SLIDE 15

Quantifying The Pain of FaaS

How FaaS Disappoints Famous Computer Scientists

The State of Serverless Computing QCon New York 06/24/2019

slide-16
SLIDE 16

Even Functional Programming is Slow!

Median and 99th percentile latencies for composing two arithmetic functions on AWS Lambda.

1 10 100 1000

Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step FNs

Latency (ms)

285 239 737 573

The State of Serverless Computing QCon New York 06/24/2019

slide-17
SLIDE 17

Even Functional Programming is Slow!

Median and 99th percentile latencies for composing two arithmetic functions on AWS Lambda.

1 10 100 1000

Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step FNs

Latency (ms)

285 239 79.6 737 573 178

The State of Serverless Computing QCon New York 06/24/2019

slide-18
SLIDE 18

Even Functional Programming is Slow!

Median and 99th percentile latencies for composing two arithmetic functions on AWS Lambda.

1 10 100 1000

Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step Fns

Latency (ms)

285 239 79.6 569 737 573 178 3346

The State of Serverless Computing QCon New York 06/24/2019

slide-19
SLIDE 19

Shared Mutable State

The State of Serverless Computing QCon New York 06/24/2019

slide-20
SLIDE 20

Shared Mutable Storage

The State of Serverless Computing QCon New York 06/24/2019

slide-21
SLIDE 21

Ideal Serverless Storage

Autoscaling Low Latency

The State of Serverless Computing QCon New York 06/24/2019

slide-22
SLIDE 22

(In)Consistency Guarantees

Shared Counter ?

The State of Serverless Computing QCon New York 06/24/2019

slide-23
SLIDE 23

No Inbound Network Connections

Easy Fault Tolerance Enables Process Migration

The State of Serverless Computing QCon New York 06/24/2019

slide-24
SLIDE 24

Indirect Communication

Write Read

The State of Serverless Computing QCon New York 06/24/2019

slide-25
SLIDE 25

We can fix that!

The State of Serverless Computing QCon New York 06/24/2019

slide-26
SLIDE 26

A Platform for Stateful Serverless Computing

The State of Serverless Computing QCon New York 06/24/2019

slide-27
SLIDE 27

Background: Anna

  • High performance across orders of magnitude in scale

ü10x faster than Redis/Cassandra in a geo-distributed deployment

  • Autoscaling & cost-efficient

ü500x faster than Amazon DynamoDB for the same cost

Chenggang Wu, Jose Faleiro, Yihan Lin, and Joseph M. Hellerstein. "Anna: A KVS for Any Scale." IEEE Transactions on Knowledge and Data Engineering (2019). Chenggang Wu, Vikram Sreekanti, and Joseph M. Hellerstein. "Autoscaling Tiered Cloud Storage in Anna." Proceedings of the VLDB Endowment 12, no. 6 (2019): 624-638.

The State of Serverless Computing QCon New York 06/24/2019

slide-28
SLIDE 28

Fluent: FaaS-over-Anna

  • Maintain disaggregation of compute & state
  • Make serverless a viable option for stateful applications
  • Use Anna for both storage and communication

Anna

The State of Serverless Computing QCon New York 06/24/2019

slide-29
SLIDE 29

Fluent: FaaS-over-Anna

Network Boundary

Anna

The State of Serverless Computing QCon New York 06/24/2019

slide-30
SLIDE 30

Logical disaggregation with physical colocation

The State of Serverless Computing QCon New York 06/24/2019

slide-31
SLIDE 31

Fluent: FaaS-over-Anna

Network Boundary

Anna

The State of Serverless Computing QCon New York 06/24/2019

slide-32
SLIDE 32

Key Idea: Caching

  • Enable low-latency data access by caching data close to code

execution

  • Communication (and composition) is achieved via a fast-path on top of

KVS puts and gets

The State of Serverless Computing QCon New York 06/24/2019

slide-33
SLIDE 33

Challenge: Cache Consistency

  • tl;dr: we can provide a variety of coordination-avoiding consistency

modes – which is better than S3 or DynamoDB!

  • This is done by encapsulating program state in lattices

The State of Serverless Computing QCon New York 06/24/2019

slide-34
SLIDE 34

Lattice

  • Data structure that accepts incoming update in a way that is

associative, commutative, and idempotent (ACI).

  • Achieves eventual replica convergence

The State of Serverless Computing QCon New York 06/24/2019

slide-35
SLIDE 35

Causal Consistency

  • Strongest consistency level that doesn’t require coordination
  • Causally-related updates will be revealed in an order that respects

causality

  • In addition, guarantee
  • Repeatable read
  • Atomic visibility

The State of Serverless Computing QCon New York 06/24/2019

slide-36
SLIDE 36

Function Composition, Revisited

Median and 99th percentile latencies for composing two arithmetic functions on AWS Lambda and Fluent.

1 10 100 1000

Fluent Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step Fns

Latency (ms)

3.59 285 239 79.6 569 6.37 737 573 178 3346

The State of Serverless Computing QCon New York 06/24/2019

slide-37
SLIDE 37

Case Study: Prediction Serving

The State of Serverless Computing QCon New York 06/24/2019

slide-38
SLIDE 38

Prediction Serving

  • Generate predictions from pretrained machine learning models
  • At first blush, a great fit for serverless infrastructure

Clean Input

Join with Reference Data

Model Replica Model Replica Model Replica

Combine Results

The State of Serverless Computing QCon New York 06/24/2019

slide-39
SLIDE 39

The State of Serverless Computing QCon New York 06/24/2019

slide-40
SLIDE 40

Background: SqueezeNet

  • State-of-the-art image classification model (developed at Berkeley!)

The State of Serverless Computing QCon New York 06/24/2019

slide-41
SLIDE 41

Prediction Serving

Median and 99th percentile latencies for SqueezeNet on Fluent and AWS SageMaker.

The State of Serverless Computing QCon New York 06/24/2019

50 100 150 200 250 300 350 400

Fluent Python AWS SageMaker

Latency (ms)

90.6 153.8 282.2 122.9 156.3 361.7

slide-42
SLIDE 42

The Future of Cloud Programming

The State of Serverless Computing QCon New York 06/24/2019

slide-43
SLIDE 43

Looking Back: Disappointed Computer Scientists

  • Functional programming is slow
  • Communication through slow storage
  • Poor consistency guarantees

The State of Serverless Computing QCon New York 06/24/2019

slide-44
SLIDE 44

Making FaaS Functional

  • Embrace state
  • Easy things become better
  • Hard things become easy

A step on our road towards a programmable cloud.

The State of Serverless Computing QCon New York 06/24/2019

slide-45
SLIDE 45

Our Vision

  • Serverless will change the way that we write software and the way

that programming infrastructure works

  • Cloud-native programming models
  • Enable users to take advantage of millions of cores and petabytes of RAM

The State of Serverless Computing QCon New York 06/24/2019

slide-46
SLIDE 46

Moving Forward from FaaS

The State of Serverless Computing QCon New York 06/24/2019

Building Developer Tools

slide-47
SLIDE 47

UC Berkeley: CS 61A Fall 2018

The State of Serverless Computing QCon New York 06/24/2019

slide-48
SLIDE 48

Moving Forward from FaaS

Developing Autoscaling Policy

The State of Serverless Computing QCon New York 06/24/2019

Building Developer Tools

slide-49
SLIDE 49

Moving Forward from FaaS

Developing Autoscaling Policy Designing SLOs & SLAs

The State of Serverless Computing QCon New York 06/24/2019

Building Developer Tools

slide-50
SLIDE 50

Thanks!

@cgwu0530 cgwu@berkeley.edu fluent-project/fluent

The State of Serverless Computing QCon New York 06/24/2019