An Investigation of the Impact of Language Runtime on the - - PowerPoint PPT Presentation

▶
an investigation of the impact of language runtime on the
SMART_READER_LITE
LIVE PREVIEW

An Investigation of the Impact of Language Runtime on the - - PowerPoint PPT Presentation

Nov 17, 2022 213 likes •570 views

An Investigation of the Impact of Language Runtime on the Performance and Cost of Serverless Functions David Jackson & Gary Clynch Institute of Technology, Tallaght 4th International Workshop on Serverless Computing (WoSC4) 20th December

slide-1
SLIDE 1

An Investigation of the Impact of Language Runtime on the Performance and Cost of Serverless Functions

David Jackson & Gary Clynch Institute of Technology, Tallaght

4th International Workshop on Serverless Computing (WoSC4) 20th December 2018

slide-2
SLIDE 2

Objective: To understand the impact of the choice of language runtime

  • n the performance and cost of

serverless function execution.

slide-3
SLIDE 3

Scope of Investigation

AWS Lambda

  • .NET Core 2 (C#)
  • Java 8
  • Python 3.6
  • NodeJS 6.10
  • Go

Azure Functions

  • .NET C#
  • NodeJS 6.11.2

AWS US-East-1 / Azure East US Empty Functions Cold-Start vs Warm-Start November 2018 128MB Memory Allocation Single Function Execution

Use empty test functions to measure platform startup performance.

slide-4
SLIDE 4

Serverless Billing Model

  • Individual execution cost per function invocation
  • Execution duration billed at “GB-second” rate
  • Cost rounded at 100ms increments (50ms @ edge)

Example: AWS Lambda, 128MB, 85ms Duration, 100ms “Billed” Duration

$0.21 + $0.20 = $0.41

* 1 million function invocations

Execution Time (0.1 s * 0.125 GB * $0.00001667 per function) Invocation Cost Total Cost

slide-5
SLIDE 5

Serverless Architecture

Serverless Performance Framework

https://github.com/Learnspree/Serverless-Language-Performance-Framework

slide-6
SLIDE 6

Results Summary

slide-7
SLIDE 7

2.69ms

.NET Core 2 (Average Performance)

2.70ms

Python

10.84ms

GoLang

3.77ms

Java

4.20ms

NodeJS

AWS Lambda - Warm Start

slide-8
SLIDE 8

2,643ms

.NET Core 2 (Average Performance)

4.84ms

Python

6.63ms

GoLang

412.89ms

Java

31.9ms

NodeJS

AWS Lambda - Cold Start

slide-9
SLIDE 9

0.78ms

.NET C# Script (Average Performance)

1.61ms

NodeJS

Azure Functions - Warm Start

slide-10
SLIDE 10

17.08ms

.NET C# Script (Average Performance)

424.97ms

NodeJS

Azure Functions - Cold Start

slide-11
SLIDE 11

Results Analysis - AWS Lambda

slide-12
SLIDE 12

AWS Lambda

Warm Start 9,550 Function Invocations Per Runtime Nov 2018

slide-13
SLIDE 13

AWS Lambda

Warm Start 9,550 Function Invocations Per Runtime Nov 2018

slide-14
SLIDE 14

AWS Lambda

Cold-Start 500 Function Invocations Per Runtime Nov 2018

slide-15
SLIDE 15

AWS Lambda

Cold-Start Histogram 500 Function Invocations Per Runtime Nov 2018

slide-16
SLIDE 16

AWS Lambda

Cold Start Top 3 Performers 500 Function Invocations Per Runtime Nov 2018

slide-17
SLIDE 17

Results Analysis - Azure Functions

slide-18
SLIDE 18

Azure Functions

Warm Start 9,550 Function Invocations Per Runtime Nov 2018

slide-19
SLIDE 19

Azure Functions

Cold Start 500 Function Invocations Per Runtime Nov 2018

slide-20
SLIDE 20

Azure Performance

Cold-Start Histogram 500 Function Invocations Per Runtime Nov 2018

slide-21
SLIDE 21

Results Analysis - AWS vs Azure

slide-22
SLIDE 22

AWS vs Azure

NodeJS Cold Start 500 Function Invocations Per Runtime Nov 2018

slide-23
SLIDE 23

AWS vs Azure

C# .NET Warm Start

slide-24
SLIDE 24

Cost Analysis

slide-25
SLIDE 25

TPS Cost Calculations

AWS Lambda

* Figures based on cold-start times to illustrate potential cost impact

slide-26
SLIDE 26

TPS Cost Calculations

Azure Functions

* Figures based on cold-start times to illustrate potential cost impact

slide-27
SLIDE 27

CostHat Model

Serverless Performance Framework Architecture

30k Function Invocations based on 1,000 TPS (Leitner et al. 2016)

slide-28
SLIDE 28

Conclusions & Future Work

slide-29
SLIDE 29

Conclusion

Overall Performance

For optimum performance and cost-management of serverless applications, C# .NET is the top performer for Azure Functions. Python is clear overall choice on AWS Lambda.

slide-30
SLIDE 30

Conclusion

Cold-Start Performance

The performance of NodeJS in Azure Functions in cold-start scenarios demands caution on its usage. Similarly caution is advised with Java and especially C# .NET on AWS Lambda.

slide-31
SLIDE 31

Conclusion

Pace of Change

The pace of change in serverless computing is extremely high - in features offered, performance characteristics and cost models. This constantly shifting environment requires regular review to ensure serverless applications are designed for optimum performance and cost benefit.

slide-32
SLIDE 32

Conclusion

Function Composition

The composition of functions in serverless applications is a crucial design decision, which if done in an appropriately fine-grained manner, can lead to a more flexible but also more cost-effective solution in the long term.

slide-33
SLIDE 33

Future Work

  • Additional Serverless Platform Testing

○ Google Cloud Functions ○ IBM OpenWhisk ○ OpenLambda

  • Real-Time Dashboard
  • Additional Test Variables

○ Regions / Hardware ○ Memory Allocations

  • Additional Test Scenarios

○ DynamoDB Access ○ API Access ○ Language Performance Benchmarking Tests

slide-34
SLIDE 34

Questions?

slide-35
SLIDE 35

References

  • Leitner, P., Cito, J. & Stöckli, E. (2016), Modelling and managing

deployment costs of microservice-based cloud applications, in ‘Proceedings of the 9th International Conference on Utility and Cloud Computing’, ACM, pp. 165–174.