Deploying Large Scale Webapps Thierry Sans Users respond to speed - - PowerPoint PPT Presentation

▶

Feb 11, 2023 347 likes •560 views

Deploying Large Scale Webapps Thierry Sans Users respond to speed Amazon found every 100ms of latency cost them 1% in sales Google found an extra . 5 seconds in search page generation time dropped traffic by 20% A broker could

SLIDE 1

Deploying Large Scale Webapps

Thierry Sans

SLIDE 2

Users respond to speed

“Amazon found every 100ms of latency cost them 1% in sales” “Google found an extra .5 seconds in search page generation time dropped traffic by 20%” “A broker could lose $4 million in revenues per millisecond if their electronic trading platform is 5 milliseconds behind the competition”

http://blog.gigaspaces.com/amazon-found-every-100ms-of-latency-cost-them-1-in-sales/

SLIDE 3

How to serve millions

Optimizing frontend code with HTTP/2, Web Workers and PWA (Progressive Web Apps) Optimizing backend code with web caching Optimizing the infrastructure with microservices

SLIDE 4

Current situation (dedicated or virtual server)

Web App

Static Files

Two types of content

Static content : html, css, js, images and so on
Dynamic content : database, uploaded files

SLIDE 5

Optimizing the backend code with Web Caching

SLIDE 6

How to improve response time?

Processing the request means:

1. Parse the HTTP request
2. Map the URL to the handler
3. Query the database or third-party API
4. Compute the view

DB and API accesses are expensive (time and money when your host charges you each access)

SLIDE 7

Fine-grained caching with the web application

(for dynamic content)

Memory Cache Web App

Cache controlled by the program

๏ Specific for each app ✓ Good for dynamic content ➡ Popular memory cache: Memcached

Static Files

SLIDE 8

Distributed Shared Cache : Memcached

http://memcached.org/

Store key/value pairs in memory
Throw away data that is the least recently used

SLIDE 9

A typical cache algorithm

retrieve from cache if data not in cache: # cache miss query the database or API update the cache return result

SLIDE 10

Cache Stampede (a.k.a dog piling)

Problem: Multiple concurrent requests doing the same request because cache was cleared Solution:

update the cache instead of clearing it after an insert
a page view will never query the database

➡ Requires cache warming

Web App

Cache

cache miss!

SLIDE 11

Optimizing the infrastructure with microservices

SLIDE 12

The reverse proxy architecture

Reverse Proxy Web App

static files

Cache repeated HTTP requests for a given time

๏ Bad for dynamic content (latency when the content is updated) ✓ Good for static content (Javascript, CSS, Media, static HTML)

Static Web Server

Memory Cache

Certificate Manager

certficate

SLIDE 13

Microservices via Docker Containers

Reverse Proxy Web App

Static Files Static Web Server

Memory Cache

Certificate Manager

SLIDE 14

Scaling over multiple servers

SLIDE 15

Load Balancer

Serving multiple apps with a load balancer

Web App Web App Web App

Memcached Memcached Memcached

…

This is not an efficient cache

Static Server

Cert Manager

SLIDE 16

Load Balancer

Distributed Shared Cache

Web App Web App Web App

Memcached Memcached Memcached

… …

Static Server

Cert Manager

SLIDE 17

Load Balancer

Database Sharding

Web App Web App Web App

Memcached Memcached Memcached

… …

…

Static Server

Cert Manager

SLIDE 18

Microservices over several physical machines

. . .

Container Orchestrator (like Kubernetes or Docker Swarm)

Load Balancer Web App Web App Web App

Memcached Memcached Memcached

…

Static Server

…

Cert Manager

SLIDE 19

CDN : Content Distribution Network

CDN

Example : Akamai, Cloudflare

edge server

SLIDE 20

High-Performance Software

Reverse proxy Nginx Static Web Server Load Balancer Certificate Manager Memory Cache Memcached / Redis Container Docker Container Orchestration Kubernetes / Docker Swarm