Semantics of Caching with SPOCA - A Stateless, Proportional, - - PowerPoint PPT Presentation

▶
semantics of caching with spoca a stateless proportional
SMART_READER_LITE
LIVE PREVIEW

Semantics of Caching with SPOCA - A Stateless, Proportional, - - PowerPoint PPT Presentation

Sep 24, 2022 280 likes •694 views

Semantics of Caching with SPOCA - A Stateless, Proportional, Optimally-Consistent Addressing Algorithm Ashish Chawla, Benjamin Reed, Karl Juhnke, Ghousuddin Syed Yahoo! Inc Video Platform 2 6/21/11 Video Platform 3 6/21/11 Simple Content

slide-1
SLIDE 1

Semantics of Caching with SPOCA - A Stateless, Proportional, Optimally-Consistent Addressing Algorithm

Ashish Chawla, Benjamin Reed, Karl Juhnke, Ghousuddin Syed Yahoo! Inc

slide-2
SLIDE 2

Video Platform

6/21/11 2
slide-3
SLIDE 3

Video Platform

6/21/11 3
slide-4
SLIDE 4

Simple Content Serving Architecture

6/21/11 4
slide-5
SLIDE 5

Outline

§ Introduction § Problem Definition § SPOCA and Requirements § Evaluations § Conclusion

6/21/11 5
slide-6
SLIDE 6

The Problem

§ The front-end server disks are a secondary bottleneck. § Eliminating redundant caching of content also reduces the load on the storage farm. § An intelligent request-routing policy can produce far more caching efficiency than even a perfect cache promotion policy that must labor under random request routing. § The cache promotion algorithm not enough.

6/21/11 6
slide-7
SLIDE 7

Problems from Geographic Distribution

6/21/11 7
slide-8
SLIDE 8

Problems from Geographic Distribution

6/21/11 8 Reque sts v7
slide-9
SLIDE 9

Problems from Geographic Distribution

6/21/11 9
slide-10
SLIDE 10

Outline

§ Introduction § Problem Definition § SPOCA and Requirements § Evaluations § Conclusion

6/21/11 10
slide-11
SLIDE 11

Requirements

§ Merge different delivery pools and manage the diverse requirements in an adaptive way. § Minimize caching disruptions when front-end server leaves or enters the pool - re-address as few files as possible to different servers. § Proportional distribution of files among servers does not necessarily result in a proportional distribution of requests (Power Law)

6/21/11 11
slide-12
SLIDE 12

SPOCA and Zebra

§ Used in production in a global scenario for web-scale load. § Shows real world improvements over the simple off-the- shelf solution. § Implements load balancing, fault tolerance, popular content handling, and efficient cache utilization with a single simple mechanism.

6/21/11 12
slide-13
SLIDE 13

Traditional Approach

6/21/11 13
slide-14
SLIDE 14

Complete Picture

6/21/11 14
slide-15
SLIDE 15

Complete Picture – Inside Data Center

6/21/11 15
slide-16
SLIDE 16

Zebra Algorithm

§ Handles the geographic component of request routing and content caching § Based on content popularity, Zebra decides when requests should be routed to content’s home locale and when the content should be cached in the nearest locale § We use bloom filters to determine popularity.

6/21/11 16
slide-17
SLIDE 17

Tracking popularity

6/21/11 17

add(vid1)

Bloom Filter

slide-18
SLIDE 18

Checking Popularity

6/21/11 18

contains(vid1)

Bloom Filter

slide-19
SLIDE 19

What’s the problem here? § Everything will become popular. § No way to expire content in bloom filter § We use a sequence of bloom filters to track popularity.

6/21/11 19
slide-20
SLIDE 20

Bloom Filter Representation

  • vid1
  • vid5

1

  • vid8
  • vid526

2

  • vid2
  • vid752
6/21/11 20
slide-21
SLIDE 21

Bloom Filter Representation

1

  • vid1
  • vid5

2

  • vid8
  • vid526
6/21/11 21
slide-22
SLIDE 22

Bloom Filter Representation

1

  • vid1
  • vid5

2

  • vid8
  • vid526
6/21/11 22

add(vid8)

slide-23
SLIDE 23

Bloom Filter Representation

  • vid8

1

  • vid1
  • vid5

2

  • vid8
  • vid526
6/21/11 23
slide-24
SLIDE 24

Bloom Filter Representation

  • vid8

1

  • vid1
  • vid5

2

  • vid8
  • vid526
6/21/11 24

contains(vid3)

slide-25
SLIDE 25

Bloom Filter Representation

  • vid8

1

  • vid1
  • vid5

2

  • vid8
  • vid526
6/21/11 25

contains(vid3)

Unified Filter vid1, vid5, vid8, vid526

slide-26
SLIDE 26

Key Points

§ Zebra determines which serving cluster will handle a given request based on geolocality and popularity. § SPOCA determines which front-end server within that cluster will cache and serve the request.

6/21/11 26
slide-27
SLIDE 27

SPOCA Algorithm

§ Goal: Maximize cache utilization at the front-end servers. § Simple content to server assignment function based on a sparse hash space. § Each front-end server is assigned a portion of the hash space according to its capacity.

§ The SPOCA routing function uses a hash function to map names to a point in a hash space.

› Input = the name of the requested content › Output = the server that will handle the request.

§ Re-hashing happens till the result maps to a valid hash space.

6/21/11 27
slide-28
SLIDE 28

SPOCA hash map example

  • 6/21/11
28
slide-29
SLIDE 29

Failure Handling

  • 6/21/11
29
slide-30
SLIDE 30

Elasticity

  • 6/21/11
30
slide-31
SLIDE 31

Popular Content

§ SPOCA minimizes the number of servers to maximize the aggregate number of cached objects. § For popular content we need to route requests to multiple front-end servers. § We store the hashed address of any requested content for a brief popularity window, 150 seconds in our case. § When the popularity window expires, the stored hash for each object is discarded.

6/21/11 31
slide-32
SLIDE 32
  • 6/21/11
32
slide-33
SLIDE 33
  • 6/21/11
33
slide-34
SLIDE 34

Outline

§ Introduction § Problem Definition § SPOCA and Requirements § Evaluations § Conclusion

6/21/11 34
slide-35
SLIDE 35

Scaling 5x w/o software improvements

6/21/11 35
slide-36
SLIDE 36

Scaling 5x with software improvements

6/21/11 36
slide-37
SLIDE 37

Memory cache hits

6/21/11 37
slide-38
SLIDE 38

Cache Hit and Misses*

2/26 3/1 3/5 3/7 3/10 3/14 Download Cache Miss 9.7% 7.2% 4.3% 3.7% 1.8% 0.4% Download Cache HIT 90.3% 92.8% 95.7% 96.3% 98.2% 99.6% Flash Cache Miss 21.8% 13.5% 22.0% 14.8% 2.5% 0.7% Flash RAM hit 57.2% 81.4% 66.1% 71.5% 90.0% 90.1%

6/21/11 38

* Download and Flash Pools in S1S data center

slide-39
SLIDE 39

Conclusion

§ Zebra and SPOCA do not have any hard state to maintain

  • r per object meta-data

§ Eliminates any per object storage overhead or management, simplifying operations. § Consolidate content serving into a single pool of servers that can handle files from a variety of different workloads. § Decouple serving and caching layers. § Cost savings and end user satisfaction are key success metrics.

6/21/11 39
slide-40
SLIDE 40 6/21/11 40