Semantics of Caching with SPOCA: A Stateless, Proportional, - - PowerPoint PPT Presentation

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Semantics of Caching with SPOCA: A Stateless, Proportional, Optimally-Consistent Addressing Algorithm

Piotr Skowron November 6, 2011

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Abstract

This presentation is about: how Yahoo! efficiently serves millions of videos from its video library, what architecture they use to ensure efficient caching (and so the significant improvement in the quality of service), how their new algorithm improved disk cache misses from 5% to less than 1% and increased memory cache hits from 45% to 80% (thus improving overall cache hits from 95% to 99.6%).

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

The way it works

clients access videos using web browsers, clients connect to front-end servers which serve the video content, the front-end servers cache content, but are not the permanent repository, videos are stored in a storage farm that is accessible through front-end servers.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

The way it works – diagram

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Local or remote serving

The videos are spread around the world so when the user is accessing a video we have one of the following options: retrieve the content from the storage farm , if it is cached in the disks of a front-end server, the content can be served more efficiently , in the best case the content may be cached in the memory of a front-end servers . In case of videos the difference between caching in memory and the caching in disks is small.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Local or remote serving

The videos are spread around the world so when the user is accessing a video we have one of the following options: retrieve the content from the storage farm , if it is cached in the disks of a front-end server, the content can be served more efficiently , in the best case the content may be cached in the memory of a front-end servers . In case of videos the difference between caching in memory and the caching in disks is small.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Local or remote serving

Retrieving a content from the storage farm not only causes significantly longer delivery, but also causes more load on the back-end infrastructure. This causes: higher cost of upgrading networking components, higher number of the servers in the storage farm required, in

• rder to handle more load.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Why caching is difficult in case of the video?

Why caching is difficult in case of the video? videos are large – a typical front-end server can hold 500 unique videos in memory and 100,000 on disk, the demand is high – users make over 30,000,000 requests per day for over 800,000 unique videos; there is over 20,000,000

• f unique videos in the library,

the ratio: total/unique requests in low.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

The traditional VIP (Virtual IP)

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

The traditional VIP (Virtual IP)

With VIP each piece of popular content will end up on multiple servers → this redundant caching is highly inefficient compared to caching where each piece of content is kept at the single server. On the other hand remembering which front-end server hosts which content is expensive. The question of a day is: how to increase caching efficiency through intelligent routing without remembering content location in a database.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Stateless

There should be no need of keeping a data catalog associating each content file with a particular front-end server.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Consistent

The requests for a particular content file should all be directed to the same server. For stateless addressing the inputs are a filename and a list of currently available servers; the output is a server from the list. Consistency means that the same input always produces the same output.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Proportional

The requests should be partitioned between the front-end servers proportionally to the servers weights. Example A newer server might have twice the capacity of an older one, and therefore should service twice as large portion of the content library. Remark The proportionality requirement rules out the use of a distance-based consistent hashing algorithm, although such algorithms are consistent and stateless.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Proportional

Proportionality must be ensured also in case of adding or removing a server. The requests must not be distributed only between the nearest servers, therefore the distance-based hashing algorithms do not pass their exam.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Optimally-consistent

When a front-end server enters or leaves the pool as few as possible files are redistributed to the other servers. Example Suppose that a pool has 3 front-end servers of weight 100 and two servers of weight 200. If the new server of weight 200 is added to the pool it must be assigned 2

9 of the files in the content library.

But also, more specifically, for each of the other servers it must take over 2

9 of the files that server was handling.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Optimally-consistent

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Proportionality once again

It must be taken into account that proportional distribution of the files does not necessarily means the proportional distribution of the requests.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Hashing algorithm

The routing function uses a hash function to map file names to the points in a hash space. Each front-end server is assigned a portion of hash space proportional to its capacity. Not every point in the hash space maps to a front-end server, so when the hash of the name of a requested video maps to unassigned space, the result of the hash function is hashed again until result lands in an assigned portion of the hash space.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Unassigned space hit

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Leaving server

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Entering server

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Hashing algorithm

The percentage of the assigned space is between 1%-25% dependently on the anticipation of future growth. If the percentage of the assigned space is 1% then SPOCA will have to generate on average 100 pseudo-random numbers to distribute tone request – this is efficient as linear congruence generators are very simple. By starting from 1% of the assigned space the system may grow to almost 100 times bigger, without worrying of running

• ut of room.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

How to deal with popular files

The hash of each filename, seed, is saved in the memory for a configurable length of time, T. When T elapses the seed is thrown away from the memory. If a request arrives for a file for which the request router has a saved seed, that file is deemed to be popular, and the generation of pseudo-random numbers starts from the saved seed rather than from the filename.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

So the SPOCA algorithm looks like that

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Geographic distribution

The separate problem is how to take into consideration geographic distribution of the content. The second question of a day, therefore, is: when it is profitable to serve data from the storing cluster, home locale, and when it is profitable to cache it near to the user, at nearest locale. The solution: we want to serve unpopular data from the home local, while the popular data should be fetched to the nearest

• locale. This is done by the independent Zebra protocol.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Bloom filters

Zebra tracks popularity of the files using Bloom filters. Bloom filter A Bloom filter, conceived by Burton Howard Bloom in 1970, is a space-efficient probabilistic data structure that is used to test whether an element is a member of a set. False positives are possible (but rarely), and false negatives are not. Unfortunately it is not possible to remove content from the bloom filter, therefore Zebra uses 17 Bloom filters instead of the single

• ne. Each bloom filter represents requests for a given interval, on

the order of hours. When the oldest filter expires, it is removed, and the new filter is added.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Data growth

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Back-end servers traffic without SPOCA

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Back-end servers traffic with SPOCA

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Estimations of the savings in the storage costs

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

The impact of the szie popularity window (T) on caching efficiency

Here the popularity window was decreased from 300s to 240s. Decreasing the window increases cach hits, however decreasing it too much can result in overloading the servers.

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Related work

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,

Context of the problem Requirements for routing Hashing algorithm Geographic distribution Experiments

Thank You

Thank You :)

Piotr Skowron Semantics of Caching with SPOCA: A Stateless, Proportional,