Presentation is loading The Latency Problem 100 ms Average : 9,3s - - PowerPoint PPT Presentation

Felix Gessert, Michael Schaarschmidt, Wolfram Wingerath, Steffen Friedrich, Norbert Ritter

gessert@informatik.uni-hamburg.de

The Cache Sketch: Revisiting

Expiration-based Caching in the Age of Cloud Data Management

Presentation is loading

Average: 9,3s

The Latency Problem

Loading…

• 1% Revenue

100 ms

The Latency Problem

• 1% Revenue
• 9% Visitors

400 ms

Average: 9,3s

The Latency Problem

Loading…

• 1% Revenue
• 9% Visitors

500 ms

• 20% Traffic

Average: 9,3s

The Latency Problem

Loading…

• 1% Revenue
• 9% Visitors
• 20% Traffic

1s

• 7% Conversions

If perceived speed is such an import factor

...what causes slow page load times?

State of

• f the art

Two bottlenecks: : la latency und processing

High Latency Processing Time

Network Latency

Th The underlying problem of

• f hig

igh page lo load tim times

• I. Grigorik, High performance browser networking.

O’Reilly Media, 2013.

The lo low-latency vis ision

Data is is served by by ubiquitous web-caches

Low Latency Less Processing

Expiration-based Every object has a defined Time-To-Live (TTL) Revalidations Allow clients and caches to check freshness at the server

The web‘s cachin ing model

Staleness as as a a consequence of

• f scalability

Expiration-based Every object has a defined Time-To-Live (TTL) Revalidations Allow clients and caches to check freshness at the server Stale Da Data

The web‘s cachin ing model

Staleness as as a a consequence of

• f scalability

Expiration-based Every object has a defined Time-To-Live (TTL) Revalidations Allow clients and caches to check freshness at the server Stale Da Data

The web‘s cachin ing model

Staleness as as a a consequence of

• f scalability

Research Question:

Can database services leverage the web caching infrastructure for low latency with rich consistency guarantees?

Client Expi pira ration

• n-

based Caches Invalida dation

• n-

based Caches

Request Path

Server/DB

Cache Hits

Browser Caches, Forward Proxies, ISP Caches Content Delivery Networks, Reverse Proxies

Expiration-based Caches:

 An object x is considered

fresh for TTLx seconds

 The server assigns TTLs

for each object In Invalidation-based Caches:

 Expose object eviction

• peration to the server

Web Cachin ing Concepts

In Invalidation- and and expiration-based caches

Client Expi pira ration

• n-

based Caches Invalida dation

• n-

based Caches

Request Path

Server/DB

Cache Hits

Browser Caches, Forward Proxies, ISP Caches Content Delivery Networks, Reverse Proxies

at connect Periodic every Δ seconds at transaction begin

2 3 1

Invalidations, Records Needs Invalidation?

4

Needs Revalidation?

The Cache Sketch approach

Le Letting th the clie client handle cache coherence

Staleness ss-Minimization Invalidation-Minimization Client nt Cache he Sketch

10101010

Bloom filter

Serv rver r Cache he Sketch

10201040 10101010

Counting Bloom Filter Non-expired Record Keys Report Expirations and Writes

The End to to End Path of

• f Request

Th The Caching Hie ierarchy

Client- (Browser-) Cache Proxy Caches ISP Caches CDN Caches Reverse- Proxy Cache

Miss Hit Miss Miss Miss Miss

Orestes

DB.posts.get(id) JavaScript

Updated by Cache Sketch Updated by the server

The End to to End Path of

• f Request

Th The Caching Hie ierarchy

Client- (Browser-) Cache Proxy Caches ISP Caches CDN Caches Reverse- Proxy Cache

Miss Hit Miss Miss Miss Miss

Orestes

GET /db/posts/{id} HTTP

Updated by Cache Sketch Updated by the server

The End to to End Path of

• f Request

Th The Caching Hie ierarchy

Client- (Browser-) Cache Proxy Caches ISP Caches CDN Caches Reverse- Proxy Cache

Miss Hit Miss Miss Miss Miss

Orestes

Updated by Cache Sketch Updated by the server

Ca Cache-Hit it: Return Object Ca Cache-Miss or

• r Revalid

idation: Forward Request

The End to to End Path of

• f Request

Th The Caching Hie ierarchy

Client- (Browser-) Cache Proxy Caches ISP Caches CDN Caches Reverse- Proxy Cache

Miss Hit Miss Miss Miss Miss

Orestes

Updated by Cache Sketch Updated by the server

Return record from DB with caching TTL

The End to to End Path of

• f Request

Th The Caching Hie ierarchy

Client- (Browser-) Cache Proxy Caches ISP Caches CDN Caches Reverse- Proxy Cache

Miss Hit Miss Miss Miss Miss

Orestes

Updated by Cache Sketch Updated by the server Low Latency Reduced Database Load Flash-Crowd Protection Higher Availability

 Let ct be the client Cache Sketch generated at time t, containing

the key keyx of every record x that was written before it expired in all caches, i.e. every x for which holds:

The Cli lient Cache Sketch

∃ 𝑠(𝑦, 𝑢𝑠, 𝑈𝑈𝑀), 𝑥 𝑦, 𝑢𝑥 ∶ 𝑢𝑠 + 𝑈𝑈𝑀 > 𝑢 > 𝑢𝑥 > 𝑢𝑠

k hash functions m Bloom filter bits 1 1 1 1 1

h1 hk ... key

find(key)

Cli lient nt Cache he Sketch

Bits = 1 no yes GET request Revalidation

Cache he

Hit Miss

key key

1 4 2

purge(obj) hashB(oid) hashA(oid)

3 1 1 1 1

Flat(Counting Bloomfilter) hashB(oid) hashA(oid)

Browser Cache CDN 1

 Solution: Cached In

Init itialization

• Clients load the Cache Sketch at connection
• Every non-stale cached record can be reused

without degraded consistency

Slo low in initial page lo loads

1

1 4 2

purge(obj) hashB(oid) hashA(oid)

3 1 1 1 1

Flat(Counting Bloomfilter) hashB(oid) hashA(oid)

Browser Cache CDN 1

 Solution: Cached In

Init itialization

• Clients load the Cache Sketch at connection
• Every non-stale cached record can be reused

without degraded consistency

Slo low in initial page lo loads

1

𝑔 ≈ 1 − 𝑓−𝑙𝑜

𝑛 𝑙

𝑙 = ln 2 ⋅ (

𝑜 𝑛)

False-Positive Rate: Hash- Functions: With 20.000 distinct updates and 5% error rate: 11 KByte

 Solution: Δ-Bounded Staleness

• Clients refresh the Cache Sketch so its age never exceeds Δ

→ Consistency guarantee: Δ-atomicity

Slo low CRUD performance

Client Expiration- based Caches Invalidation- based Caches Server

Cache Sketch ct Qu Query Cache Sketch fresh records Revalidate record & Refresh Cache Sketch Cache Hits Fresh record & new Cache Sketch

• time t
• time t + Δ

2

Hig igh Abort Rates in in OCC

Cache Cache Cache

REST-Server REST-Server REST-Server DB

Coordinator

Client

Begin Transaction Bloom Filter

1

Reads Writes

2

Writes (Hidden) validation 4

5

Writes (Public) Read all

prevent conflicting validations

 Solution: Conflict-Avoidant Optimistic Transactions

• Cache Sketch fetched with transaction begin
• Cac

ached reads → Sh Shorter tr transaction duration → le less ss ab aborts

3

Committed OR aborted + stale objects

Commit: read- & write-set versions

3

Costly In Invalidations

 Solution: In

Invalidation Min inimization

• The server Cache Sketch tracks TTLs
• Invalidation on
• nly

ly necessary, if there are unexpired records

Server Cache he Sketch

10201040 10101010

Counting Bloom Filter Non-expired Record Keys Report Expirations and Writes Needs Invalidation?

4

End-to to-End Example

Client Expiration- based Cache Invalidation- based Cache Server Client Cache Sketch Server Cache Sketch

b={x2} t = {(x2, t2), (x3, t3),(x1, t1)} b=

INITIALIZE

c={(x2,t2),(x3,t3)} c={(x1,t1)

b={x2}

CONNECT

bt0={x2}

READ x3 QUERY

x3

RESPONSE

false

GET

x3

RESPONSE

x3

QUERY

x2

RESPONSE

true

READ x2

REVALIDATE

x2

c={(x3,t3)}

RESPONSE

x2,t4

c={(x2,t4),(x3,t3)} c={(x2,t4)}

REPORT READ

x2,t4

b={x2} t = {(x2, t4), (x3, t3),(x1, t1)}

RESPONSE

inv=true

WRITE x1 PUT

x1=v

REPORT WRITE

x1

RESPONSE

• k

INVALIDATE

x1

b={x1,x2} t = {(x2, t4), (x3, t3),(x1, t1)}

 Problem: if TTL ≫ time to next write, then it is

contained in Cache Sketch unnecessarily long

 TTL

TL Esti timator: finds „best“ TTL

 Trade-Off:

TTL Estimation

Determining th the best TTL TL

Lo Longer TTL TLs Sh Shorter TTLs

• Higher cache-hit rates
• more invalidations
• less invalidations
• less stale reads

Id Idea:

• 1. Estimate average time to next write 𝐹[𝑈

𝑥] for each record

• 2. Weight 𝐹[𝑈

𝑥] using the cache miss rate

TTL Estimation

Determining th the best TTL TL

Client Server

Reads Misses λm: Miss Rate λw: Write Rate collect TTL per record λm λw

Caches

Writes ~ Poisson

TTL Estimator

Objective:

• maximize Cache Hits
• minimize Purges
• minimize Stale Reads
• bound Cache Sketch

false positive rate

Writes ~ Poisson

 Goal: Analysis of arbitrary caching architectures using

the standard YCSB benchmark

• Metric

ics to to evaluate: Latency, Throughput, Cache Hits, Stale Reads, Invalidations

YCSB Monte Carlo lo Cachin ing Sim imulator (YMCA)

Pluggable simulated caches, choosable topology

YCSB SB wo workload YMCA Client

Stale Read Detector Cache Miss Detector

Expiration- based Cache Invalidation- based Cache Database Ser erver er

Pluggable latency distributions

purge CRUD CRUD CRUD CRUD

Results: Sim imulation & real-world

CDN Northern California Client MongoDB Orestes Ireland

Setup:

Page load times with cach ched in initia itialization (YMCA): Average Latency for YCSB Workloads A and B (real):

Results: Sim imulation & real-world

CDN Northern California Client MongoDB Orestes Ireland

Setup:

Page load times with cach ched in initia itialization (YMCA): Average Throughput for YCSB Workloads A and B (real):

 Goal: Efficient Generation of Cache Sketch and

Invalidation Minimization

 Cou

• untin

ing Bloo loom Filt Filter and key → expir iration mapping

The Server Cache Sketch

Scalable Im Implementation

https://github.com/Baqend/Orestes-Bloomfilter

Add keyx if x unexpired

Cache Sketch for Table A … B … C

Get Cache Sketch: Union

(Bitwise OR)

The Big ig Pic icture

Im Implementation in in ORESTES

Internet Cache Sketch Reverse-Proxy Caches Orestes Servers Desktop Mobile Tablet

Content-Delivery- Network

 Cache Sketch is part of ORESTES, a database-

independent Backend-as-a-Service

The Big ig Pic icture

Im Implementation in in ORESTES

Internet Cache Sketch Reverse-Proxy Caches Orestes Servers Desktop Mobile Tablet

Content-Delivery- Network

Polyglot Storage

The Big ig Pic icture

Im Implementation in in ORESTES

Internet Cache Sketch Reverse-Proxy Caches Orestes Servers Desktop Mobile Tablet

Content-Delivery- Network

Database-as-a-Service Middleware: Caching, Transactions, Schemas, Authorization, Multi-Tenancy

The Big ig Pic icture

Im Implementation in in ORESTES

Internet Cache Sketch Reverse-Proxy Caches Orestes Servers Desktop Mobile Tablet

Content-Delivery- Network

Standard HTTP Caching

The Big ig Pic icture

Im Implementation in in ORESTES

Internet Cache Sketch Reverse-Proxy Caches Orestes Servers Desktop Mobile Tablet

Content-Delivery- Network

Unified REST API

Future Work

Query ry-Result-Caching

Cached Query

ORESTES

Create, Update, Delete

Pub-Sub

• peration &

after-image

Ca Caches Pub-Sub Ca Cache Sketch & In Invalid idator

invalidate Cache Sketch

• f queries

Changed queries

Stream-Processing: Which query result sets changed? Decision Model: When is it better to cache lists of ids vs. full results and for which TTL – or not at all?

Team: Felix Gessert, Florian Bücklers, Hannes Kuhlmann, Malte Lauenroth, Michael Schaarschmidt

• 19. August 2014

Page-Load Tim imes

What im impact does th the Cache Sketch have?

0,7s 1,8s 2,8s 3,6s 3,4s CALIFORNIEN 0,5s 1,8s 2,9s 1,5s 1,3s FRANKFURT 0,6s 3,0s 7,2s 5,0s 5,7s SYDNEY

0,5s 2,4s 4,0s 5,7s 4,7s TOKYO

 Cache Sketch: dual approach to web caching for

database services

• Consistent (Δ-atomic) expiration-based caching
• Invalidation-based caching with minimal purges

 Keys Id

Ideas:

• Maintain Blo

loom fi filt lter of potentially stale objects

• Let clients handle cache coherence through revali

lidatio ions when an object is contained in the filter

• Estimate the best TTL

TL based on access statistics

Summary ry

Cached Initialization Δ-Bounded Staleness Conflict- Avoidant Transactions Invalidation Minimization

Thank you

gessert@informatik.uni-hamburg.de Orestes.info Baqend.com