From FIFO to Predictive Cache Replacement Daniel Meint advised by - PowerPoint PPT Presentation

Chair of Network Architectures and Services Department of Informatics Technical University of Munich From FIFO to Predictive Cache Replacement Daniel Meint advised by Stefan Liebald Thursday 11 th October, 2018 Chair of Network Architectures and Services Department of Informatics Technical University of Munich

Introduction Structure of this talk Introduction Evaluation of Replacement Strategies Traditional Cache Replacement Arrival-based Replacement Recency-based Replacement Frequency-based Replacement Adaptive Cache Replacement Predictive Cache Replacement for Web-based Multimedia Content Conclusion Bibliography D. Meint — Cache Replacement 2

Introduction Caching “We are therefore forced to recognize the possibility of constructing a hierarchy of memories, each of which has greater capacity than the preceding but which is less quickly accessible.” [2] Caches offer high speed, but (relatively) low capacity. • Hardware Caches used by the CPU/GPU • Software Caches • Database systems • Operating systems • Web browsers • Web servers • ... D. Meint — Cache Replacement 3

Introduction Cache Management Two central heuristics govern the content of any cache: Fetch Policy determines when information is brought into cache. • Demand Fetching • Anticipatory Fetching or Prefetching Removal Policy determines what information leaves the cache. • widely studied • 50+ proposals from simple to complex D. Meint — Cache Replacement 4

Evaluation of Replacement Strategies Performance Metrics Definition Concern � i h i Hit Ratio (HR) general; small cache systems � i f i � i h i · s i Byte Hit Ratio (BHR) network bandwidth savings � i f i · s i � i h i · d i Delay Savings Ratio (DSR) experienced retrieval time � i f i · d i h i total number of references to object i which were satisfied from the cache f i total number of references to object i s i size of object i d i mean delay to fetch object i into the cache D. Meint — Cache Replacement 5

Traditional Cache Replacement Key-based Strategies Objects have certain characteristics that may influence the replacement decision. arrival When was the object admitted to the cache? recency When was the object requested last? frequency How often has the object been requested? size How much space does the object take? cost How expensive is it to fetch this object? expiration When is this object going to become invalid? D. Meint — Cache Replacement 6

Arrival-based Replacement First In, First Out (FIFO) Request sequence: A B A C A D enqueue dequeue D. Meint — Cache Replacement 7

Arrival-based Replacement First In, First Out (FIFO) Request sequence: A B A C A D enqueue dequeue A D. Meint — Cache Replacement 7

Arrival-based Replacement First In, First Out (FIFO) Request sequence: A B A C A D enqueue dequeue B A D. Meint — Cache Replacement 7

Arrival-based Replacement First In, First Out (FIFO) Request sequence: A B A C A D enqueue dequeue B A D. Meint — Cache Replacement 7

Arrival-based Replacement First In, First Out (FIFO) Request sequence: A B A C A D enqueue dequeue C B A D. Meint — Cache Replacement 7

Arrival-based Replacement First In, First Out (FIFO) Request sequence: A B A C A D enqueue dequeue C B A D. Meint — Cache Replacement 7

Arrival-based Replacement First In, First Out (FIFO) Request sequence: A B A C A D enqueue D C B A D. Meint — Cache Replacement 7

Recency-based Replacement Least Recently Used (LRU) Evicting the item that was not requested for the longest time. Request sequence: A B A C A D MRU LRU D. Meint — Cache Replacement 8

Recency-based Replacement Least Recently Used (LRU) Evicting the item that was not requested for the longest time. Request sequence: A B A C A D MRU A LRU D. Meint — Cache Replacement 8

Recency-based Replacement Least Recently Used (LRU) Evicting the item that was not requested for the longest time. Request sequence: A B A C A D MRU B A LRU D. Meint — Cache Replacement 8

Recency-based Replacement Least Recently Used (LRU) Evicting the item that was not requested for the longest time. Request sequence: A B A C A D MRU A B LRU D. Meint — Cache Replacement 8

Recency-based Replacement Least Recently Used (LRU) Evicting the item that was not requested for the longest time. Request sequence: A B A C A D MRU C A B LRU D. Meint — Cache Replacement 8

Recency-based Replacement Least Recently Used (LRU) Evicting the item that was not requested for the longest time. Request sequence: A B A C A D MRU A C B LRU D. Meint — Cache Replacement 8

Recency-based Replacement Least Recently Used (LRU) Evicting the item that was not requested for the longest time. Request sequence: A B A C A D MRU D A C B D. Meint — Cache Replacement 8

Recency-based Replacement Scan-vulnerability of LRU scan large sequential one-time request stream Request sequence: ... A B A C A D X Y Z A D A C C A ... MRU D A C LRU D. Meint — Cache Replacement 8

Recency-based Replacement Scan-vulnerability of LRU scan large sequential one-time request stream Request sequence: ... A B A C A D X Y Z A D A C C A ... MRU Z Y X LRU D. Meint — Cache Replacement 8

Frequency-based Replacement Least Frequently Used (LFU) LFU evicts the object with the lowest reference count. Rationale: Exploit long-term popularity patterns, some items are "hotter" than others. Web object popularity • distribution follows Zipf’s Law [1, 3]. • is often highly dynamic [5]. Distinguish Perfect LFU tracks all objects In-Cache LFU tracks cached objects only D. Meint — Cache Replacement 9

Frequency-based Replacement In-Cache LFU pollution Request sequence: X X X Y Y Y A B A B A B A B A B A B ... X (3) Y (3) D. Meint — Cache Replacement 10

Adaptive Cache Replacement ARC LRU MRU LRU B1 T1 T2 B2 L1 L2 Figure 1: Simplified ARC directory 1 L1 contains items accessed exactly once (recency component) L2 contains items accessed at least twice (frequency component) T1 and T2 form the actual cache (main cache) B1 and B2 only contain metadata (ghost cache) 1 for an in-depth description, refer to [6, 7, 8] D. Meint — Cache Replacement 11

Adaptive Cache Replacement ARC Performance ARC is scan-resistant and low-overhead. workload cache size (MB) LRU (HR %) ARC (HR %) P1 16 16.55 28.26 P2 16 18.47 27.38 P3 16 3.57 17.12 P4 16 5.24 11.24 P5 16 6.73 14.27 ConCat 16 14.38 21.67 Merge(P) 128 38.05 39.91 DS1 1024 11.65 22.52 SPC1 4096 9.19 20.00 S1 2048 23.71 33.43 S2 2048 25.91 40.68 S3 2048 25.26 40.44 Merge(S) 4096 27.62 40.44 sources: [7, 8] D. Meint — Cache Replacement 12

Predictive Cache Replacement for Web-based Multimedia Content Two separate phases need to be distinguished: 1. Prediction phase (various techniques) 2. Removal phase (P-LFU) Replacement Policy Prediction D. Meint — Cache Replacement 13

Predictive Cache Replacement for Web-based Multimedia Content Prediction Example Replacement Policy Prediction Figure 2: Optimal fit of the exponential and gaussian distributions to an example cumulative request pattern. source: [5] D. Meint — Cache Replacement 14

Predictive Cache Replacement for Web-based Multimedia Content Predictive Least Frequently Used (P-LFU) Prediction P-LFU P-LFU Remove the item with the smallest number of predicted requests. D. Meint — Cache Replacement 15

Predictive Cache Replacement for Web-based Multimedia Content On-Demand Fetching vs. Prefetching Upper bound of 50% HR under Demand Fetching model for any Replacement Policy [4, 10]. Prefetching results in • higher hit rate • higher network load Effectiveness of Prefetching is measured by Coverage fraction of total misses eliminated by prefetching Accuracy fraction of prefetched items that were eventually requested Synonyms: Anticipatory fetching, Buffering (esp. video content) D. Meint — Cache Replacement 16

Conclusion • Replacement is a central design issue in any caching scheme • There exist various proposals beyond what is covered in this talk • Strategies can be compared along different metrics including hit-rate, byte-hit-rate, and delay-savings-ratio • LRU is the most widely used policy and performs "good enough" [9] in most scenarios • ARC outperforms LRU and keeps overhead low • Multimedia caching is of special interest for future research D. Meint — Cache Replacement 17