Transient and Steady-state Regime of a Family of List-based Cache Replacement Algorithms Nicolas Gast 1 , Benny Van Houdt 2 Sigmetrics 2015, Portland, Oregon 1 Inria 2 University of Antwerp Nicolas Gast – 1 / 26
Caches are everywhere User/Application Examples: fast Processor Database cache CDN slow Single cache / hierarchy of caches data source Nicolas Gast – 2 / 26
In this talk, I focus on a single cache. The question is: which item to replace? Application requests cache data source Nicolas Gast – 3 / 26
In this talk, I focus on a single cache. The question is: which item to replace? Application hit requests cache data source Nicolas Gast – 3 / 26
In this talk, I focus on a single cache. The question is: which item to replace? Application Classical cache replacement policies: RAND, FIFO hit miss requests LRU CLIMB cache Other approaches: replace one item Time to live data source Nicolas Gast – 3 / 26
Our performance metric will be the hit probability hit probability = number of items served from cache total number of items served = 1 − miss probability Nicolas Gast – 4 / 26
Our performance metric will be the hit probability hit probability = number of items served from cache total number of items served = 1 − miss probability Theoretical studies: IRM (started with [King 1971, Gelenbe 1973]) Practical studies use trace-based simulations. Approximations: link between TTL and cache replacement policies. ◮ FIFO and LRU: [Dan and Towsley 1990, Martina at al. 14, Fofack at al. 13, Berger et al. 14] ◮ LRU: Che approximation [Che, 2002, Fricker et al. 12] Nicolas Gast – 4 / 26
0.40 0.40 0.35 0.35 0.30 0.30 probability in cache probability in cache 0.25 0.25 0.20 0.20 0.15 0.15 0.10 0.10 0.05 1 list (200) 0.05 approx 1 list (200) 4 lists (50/50/50/50) approx 4 lists (50/50/50/50) 0.00 0.00 0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000 number of requests number of requests Contributions (and Outline of the talk) We introduce a family of policies for which the cache is (virtually) divided into lists (generalization of FIFO/RANDOM) Nicolas Gast – 5 / 26
0.40 0.40 0.35 0.35 0.30 0.30 probability in cache probability in cache 0.25 0.25 0.20 0.20 0.15 0.15 0.10 0.10 0.05 1 list (200) 0.05 approx 1 list (200) 4 lists (50/50/50/50) approx 4 lists (50/50/50/50) 0.00 0.00 0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000 number of requests number of requests Contributions (and Outline of the talk) We introduce a family of policies for which the cache is (virtually) divided into lists (generalization of FIFO/RANDOM) 1 We can compute in polynomial time the steady-state distribution under the IRM model. ◮ Disprove old conjectures. Nicolas Gast – 5 / 26
Contributions (and Outline of the talk) We introduce a family of policies for which the cache is (virtually) divided into lists (generalization of FIFO/RANDOM) 1 We can compute in polynomial time the steady-state distribution under the IRM model. ◮ Disprove old conjectures. 2 We develop a mean-field approximation and show that it is accurate ◮ Fast approximation of the steady-state distribution. ◮ We can characterize the transient behavior : 0.40 0.40 0.35 0.35 0.30 0.30 probability in cache probability in cache 0.25 0.25 0.20 0.20 0.15 0.15 0.10 0.10 ODE approx. simulation 0.05 1 list (200) 0.05 approx 1 list (200) 4 lists (50/50/50/50) approx 4 lists (50/50/50/50) 0.00 0.00 0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000 number of requests number of requests Nicolas Gast – 5 / 26
Contributions (and Outline of the talk) We introduce a family of policies for which the cache is (virtually) divided into lists (generalization of FIFO/RANDOM) 1 We can compute in polynomial time the steady-state distribution under the IRM model. ◮ Disprove old conjectures. 2 We develop a mean-field approximation and show that it is accurate ◮ Fast approximation of the steady-state distribution. ◮ We can characterize the transient behavior : 0.40 0.40 0.35 0.35 0.30 0.30 probability in cache probability in cache 0.25 0.25 0.20 0.20 0.15 0.15 0.10 0.10 ODE approx. simulation 0.05 1 list (200) 0.05 approx 1 list (200) 4 lists (50/50/50/50) approx 4 lists (50/50/50/50) 0.00 0.00 0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000 number of requests number of requests 3 We provide guidelines of how to tune the parameters by using IRM and trace-based simulation Nicolas Gast – 5 / 26
Outline Cache model and IRM 1 Steady-state performance under the IRM model 2 Fast and accurate mean-field approximation 3 How to choose the size of the lists? 4 Conclusion 5 Nicolas Gast – 6 / 26
Outline Cache model and IRM 1 Steady-state performance under the IRM model 2 Fast and accurate mean-field approximation 3 How to choose the size of the lists? 4 Conclusion 5 Nicolas Gast – 7 / 26
I consider a cache (virtually) divided into lists IRM At each time step, item i is Application requested with probability p i (IRM assumption 3 ) . . . . . . list 1 list j list j +1 list h data source 3L. Breslau, P. Cao, L. Fan, G. Phillips, and S. Shenker. Web caching and Zipf-like distributions: Evidence and implications. In INFOCOM’99, volume 1, pages 126-134. IEEE, 1999. Nicolas Gast – 8 / 26
I consider a cache (virtually) divided into lists IRM At each time step, item i is Application requested with probability p i (IRM assumption 3 ) miss MISS If item i is not in the cache, it is exchanged with a item from list 1 (FIFO or RAND). . . . . . . list 1 list j list j +1 list h data source 3L. Breslau, P. Cao, L. Fan, G. Phillips, and S. Shenker. Web caching and Zipf-like distributions: Evidence and implications. In INFOCOM’99, volume 1, pages 126-134. IEEE, 1999. Nicolas Gast – 8 / 26
I consider a cache (virtually) divided into lists IRM At each time step, item i is Application requested with probability p i (IRM assumption 3 ) miss hit MISS If item i is not in the cache, it is exchanged with a item from list 1 (FIFO or RAND). . . . . . . list 1 list j list j +1 list h HIT If item i is list j , it is exchanged with a item from list j + 1 (FIFO or RAND). data source 3L. Breslau, P. Cao, L. Fan, G. Phillips, and S. Shenker. Web caching and Zipf-like distributions: Evidence and implications. In INFOCOM’99, volume 1, pages 126-134. IEEE, 1999. Nicolas Gast – 8 / 26
Items on higher lists are (supposedly) more popular. cache size = m = m 1 + · · · + m h less popular popular items hit . . . . . . list 1 list j list j +1 list h miss These algorithms are refered to as RAND( m )and FIFO( m ). Nicolas Gast – 9 / 26
Outline Cache model and IRM 1 Steady-state performance under the IRM model 2 Fast and accurate mean-field approximation 3 How to choose the size of the lists? 4 Conclusion 5 Nicolas Gast – 10 / 26
The steady-state is a product-form distribution Same for RAND and FIFO. Nicolas Gast – 11 / 26
The steady-state is a product-form distribution Same for RAND and FIFO. Example of a cache of size 4 with 3 lists and m = (1 , 2 , 1): j i k ℓ Probability of ( i , j , k , ℓ ) is proportional to p i ( p j p k ) 2 ( p ℓ ) 3 . Nicolas Gast – 11 / 26
We can compute the miss probability by using a dynamic programming approach (Generalization of [Fagin,Price] 5 ). We want to compute π ( c ) = E ( m + e 1 , n ) � � M ( m ) = p k , E ( m , n ) c ∈C n ( m ) k �∈ c i r i h � � � where E ( r , k ) = . p c ( i , j ) c ∈C k ( r ) i =1 j =1 We obtain a recursion formula on E ( r , k ): solvable in O ( n × m 1 . . . m h ). The Dan and Towsley 4 approximation is not needed for polynomial time. 4A. Dan and D. Towsley. An approximate analysis of the LRU and FIFO buffer replacement schemes. SIGMETRICS Perform. Eval. Rev., 18(1):143-152, Apr. 1990. 5R. Fagin and T. G. Price. Efficient calculation of expected miss ratios in the independent reference model. SIAM J. Comput., 7:288-296, 1978. Nicolas Gast – 12 / 26
A higher cache size and more lists (usually) leads to a lower steady-state miss probability. 0.55 h = 2, m 2 = m − 1 h = 3, m 3 = m − 2 0.5 h = 5, m 5 = m − 4 h = 10, m 10 = m − 9 0.45 Lower bounds Miss Probability 0.4 h = ∞ 0.35 0.3 0.25 n = 3000, α = 0.8 0.2 300 400 500 600 700 800 900 1000 Cache size m ( h = ∞ corresponds to LFU). Nicolas Gast – 13 / 26
Is increasing the number of lists always better 6 ? less popular popular items hit . . . m j m j +1 . . . m h m 1 ? ≥ ? Six lists: m = (1 , 1 , 1 , 1 , 1 , 1) Three lists: m = (1 , 1 , 4). 6 conjectured in O. I. Aven, E. G. Coffman, Jr., and Y. A. Kogan. Stochastic Analysis of Computer Storage. Kluwer Academic Publishers, Norwell, MA, USA, 1987. Nicolas Gast – 14 / 26
Is increasing the number of lists always better 6 ? ? ≥ ? Six lists: m = (1 , 1 , 1 , 1 , 1 , 1) Three lists: m = (1 , 1 , 4). 6 conjectured in O. I. Aven, E. G. Coffman, Jr., and Y. A. Kogan. Stochastic Analysis of Computer Storage. Kluwer Academic Publishers, Norwell, MA, USA, 1987. Nicolas Gast – 14 / 26
Outline Cache model and IRM 1 Steady-state performance under the IRM model 2 Fast and accurate mean-field approximation 3 How to choose the size of the lists? 4 Conclusion 5 Nicolas Gast – 15 / 26
Recommend
More recommend
