Worst-case Bounds and Optimized Cache on M th Request Cache - - PowerPoint PPT Presentation

▶
worst case bounds and optimized cache on m th request
SMART_READER_LITE
LIVE PREVIEW

Worst-case Bounds and Optimized Cache on M th Request Cache - - PowerPoint PPT Presentation

Apr 08, 2023 181 likes •1.08k views

Worst-case Bounds and Optimized Cache on M th Request Cache Insertion Policies under Elastic Conditions Niklas Carlsson, Linkping University Derek Eager, University of Saskatchewan Proc. IFIP Performance , Toulouse, France, Dec. 2018. Motivation

slide-1
SLIDE 1

Worst-case Bounds and Optimized Cache on Mth Request Cache Insertion Policies under Elastic Conditions

Niklas Carlsson, Linköping University Derek Eager, University of Saskatchewan

  • Proc. IFIP Performance, Toulouse, France, Dec. 2018.
slide-2
SLIDE 2

2

Motivation and problem

  • Cloud services and other shared infrastructures increasingly common
  • Typically third-party operated
  • Allow service providers to easily scale services based on current

resource demands

  • Content delivery context: Many content providers are already using

third-party operated Content Distribution Networks (CDNs) and cloud-based content delivery platforms

  • This trend towards using third-party providers on an on-demand basis

is expected to increase as new content providers enter the market Problem: Individual content provider that wants to minimize its delivery costs under the assumptions that

  • the storage and bandwidth resources it requires are elastic,
  • the content provider only pays for the resources that it consumes, and
  • costs are proportional to the resource usage.
slide-3
SLIDE 3

Motivation and problem

  • Cloud services and other shared infrastructures increasingly common
  • Typically third-party operated
  • Allow service providers to easily scale services based on current

resource demands

  • Content delivery context: Many content providers are already using

third-party operated Content Distribution Networks (CDNs) and cloud-based content delivery platforms

  • This trend towards using third-party providers on an on-demand basis

is expected to increase as new content providers enter the market Problem: Individual content provider that wants to minimize its delivery costs under the assumptions that

  • the storage and bandwidth resources it requires are elastic,
  • the content provider only pays for the resources that it consumes, and
  • costs are proportional to the resource usage.
slide-4
SLIDE 4

High-level picture

  • Analyze the optimized delivery costs of different cache on Mth request cache

insertion policies when using a Time-to-Live (TTL) based eviction policy

  • File object remains in the cache until a time T has elapsed
  • Assuming elastic resources, cache eviction is not needed to make space for a

new insertion

  • Rather to reduce cost by removing objects that are not expected to be

requested again soon

  • A TTL-based eviction policy is a good heuristic for such purposes
  • Bonus: TTL provides approximation for fixed-size LRU caching
  • Cloud service providers already provide elastic provisioning at varying

granularities for computation and storage

  • Support for fine-grained elasticity likely to increase in the future
slide-5
SLIDE 5

5

Contributions

Within this context, we

  • derive worst-case bounds for the optimal cost and competitive cost

ratios of different classes of cache on Mth request cache insertion policies,

  • derive explicit average cost expressions and bounds under arbitrary

inter-request distributions,

  • derive explicit average cost expressions and bounds for short-tailed

(deterministic, Erlang, and exponential) and heavy-tailed (Pareto) inter- request distributions, and

  • present numeric and trace-based evaluations that reveal insights into

the relative cost performance of the policies.

Our results show that a window-based cache on 2nd request policy (with parameter selected based on the best worst-case bounds) provides good average performance across the different distributions and the full parameter ranges of each considered distribution

slide-6
SLIDE 6

6

Contributions

Within this context, we

  • derive worst-case bounds for the optimal cost and competitive cost

ratios of different classes of cache on Mth request cache insertion policies,

  • derive explicit average cost expressions and bounds under arbitrary

inter-request distributions,

  • derive explicit average cost expressions and bounds for short-tailed

(deterministic, Erlang, and exponential) and heavy-tailed (Pareto) inter- request distributions, and

  • present numeric and trace-based evaluations that reveal insights into

the relative cost performance of the policies.

Our results show that a window-based cache on 2nd request policy (using a single threshold parameter optimized to minimize the best worst-case costs) provides good average performance across the different distributions and the full parameter ranges of each considered distribution

slide-7
SLIDE 7

7

System model

slide-8
SLIDE 8

8

System model

  • Assumptions:
  • storage and bandwidth resources it requires are elastic
  • content provider only pays for the resources that it consumes
  • costs are proportional to the resource usage
  • Analyze the optimized delivery costs of different cache on Mth request cache

insertion policies when using a Time-to-Live (TTL) based eviction policy

  • Policy decision: At the time a request is made for a file object not currently

in the cache, the system must, in an online fashion, decide whether the

  • bject should be cached or not

Storage close to end-user (normalized storage cost 1 per time unit) Backhaul bandwidth (remote bandwidth cost R)

slide-9
SLIDE 9

9

System model and problem

  • Assumptions:
  • storage and bandwidth resources it requires are elastic
  • content provider only pays for the resources that it consumes
  • costs are proportional to the resource usage
  • Analyze the optimized delivery costs of different cache on Mth request cache

insertion policies when using a Time-to-Live (TTL) based eviction policy

  • Policy decision: At the time a request is made for a file object not currently

in the cache, the system must, in an online fashion, decide whether the

  • bject should be cached or not

Storage close to end-user (normalized storage cost 1 per time unit) Backhaul bandwidth (remote bandwidth cost R)

slide-10
SLIDE 10

10

In Insertion policies

slide-11
SLIDE 11

In Insertion policies

slide-12
SLIDE 12

In Insertion policies

slide-13
SLIDE 13

In Insertion policies

miss

slide-14
SLIDE 14

In Insertion policies

R

slide-15
SLIDE 15

In Insertion policies

T R

slide-16
SLIDE 16

In Insertion policies

T R

slide-17
SLIDE 17

In Insertion policies

T R Always on 1st (T)

slide-18
SLIDE 18

In Insertion policies

T R Always on 1st (T) R T

slide-19
SLIDE 19

In Insertion policies

T R Always on 1st (T) R a3

slide-20
SLIDE 20

In Insertion policies

T R Always on 1st (T) R T a3

slide-21
SLIDE 21

In Insertion policies

T R Always on 1st (T) R T a3 a4

slide-22
SLIDE 22

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R

slide-23
SLIDE 23

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R

slide-24
SLIDE 24

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R R Always on 2nd (T)

slide-25
SLIDE 25

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R R (cnt=1) Always on 2nd (T)

slide-26
SLIDE 26

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R R Always on 2nd (T) R (cnt=2) T

slide-27
SLIDE 27

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R R Always on 2nd (T) R T a3

slide-28
SLIDE 28

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R R Always on 2nd (T) R T a3 a4

slide-29
SLIDE 29

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R R Always on 2nd (T) R T a3 a4 R (cnt=1)

slide-30
SLIDE 30

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R R Always on 2nd (T) R T a3 a4 R R (cnt=1) Single-window on 2nd (T)

slide-31
SLIDE 31

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R R Always on 2nd (T) R T a3 a4 R R (cnt Single-window on 2nd (T) R T

slide-32
SLIDE 32

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R R Always on 2nd (T) R T a3 a4 R R Single-window on 2nd (T) R (cnt=1)

slide-33
SLIDE 33

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R R Always on 2nd (T) R T a3 a4 R R Single-window on 2nd (T) R T R (cnt=2)

slide-34
SLIDE 34

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R R Always on 2nd (T) R T a3 a4 R R Single-window on 2nd (T) R T a4 R

slide-35
SLIDE 35

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R R Always on 2nd (T) R T a3 a4 R R Single-window on 2nd (T) R T a4 R (cnt=1) R

slide-36
SLIDE 36

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R R Always on 2nd (T) R T a3 a4 R R Single-window on 2nd (T) R T a4 R R

slide-37
SLIDE 37

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R R Always on 2nd (T) R T a3 a4 R R Single-window on 2nd (T) R T a4 R R Dual-window on 2nd (W ≤ T), here W = T/2 R T R R R R

slide-38
SLIDE 38

In Insertion policies

T R Always on 1st (T) R T a3 a4 T R R Always on 2nd (T) R T a3 a4 R R Single-window on 2nd (T) R T a4 R R Single-window on 3rd (T) R T R R R R

slide-39
SLIDE 39

39

Worst-case bounds

slide-40
SLIDE 40

Offline-optimal lower bound

R R a3 a4 R “Oracle” policy: Keep in cache until (at least) the next inter-request arrival i whenever ai < R; otherwise, do not cache.

slide-41
SLIDE 41

Offline-optimal lower bound

R R a3 a4 R “Oracle” policy: Keep in cache until (at least) the next inter-request arrival i whenever ai < R; otherwise, do not cache.

slide-42
SLIDE 42

Example: Always on 1st

st

T R R T a3 a4 T R

slide-43
SLIDE 43

Worst-case ratio: Always on 1st

st

slide-44
SLIDE 44

Worst-case ratio: Always on 1st

st

?? Given arbitrary worst- case request sequence

slide-45
SLIDE 45

Worst-case ratio: Always on 1st

st

Case: T ≤ R R R R R T T T

??

slide-46
SLIDE 46

Worst-case ratio: Always on 1st

st

Case: T ≤ R R R R R T T T

… [some steps] …

slide-47
SLIDE 47

Worst-case ratio: Always on 1st

st

Case: T ≤ R R R R R T T T

… [some steps] …

slide-48
SLIDE 48

Worst-case ratio: Always on 1st

st

Case: T ≤ R R R R R T T T

… [some steps] …

Bound monotonically decreasing in range 0 ≤ T ≤ R. Bound tight when T  R (and equal to 2); achieved with T+ spacing Similar approach for case when R ≤ T

slide-49
SLIDE 49

49

Worst-case bounds

Policy Parameters Optimal choice Tight bound Always 1st T T = R 2 Always Mth Single-window Mth Dual-window 2nd

slide-50
SLIDE 50

50

Worst-case bounds

Policy Parameters Optimal choice Tight bound Always 1st T T = R 2 Always Mth T T = R M+1 Single-window Mth T T = R M+1 Dual-window 2nd W, T W = T = R 3

slide-51
SLIDE 51

51

Worst-case bounds

Policy Parameters Optimal choice Tight bound Always 1st T T = R 2 Always Mth T T = R M+1 Single-window Mth T T = R M+1 Dual-window 2nd W, T W = T = R 3

  • Although M+1 worst-case bounds may seem discouraging, we

will see that window-based policies are good on average (across different distributions and distribution parameters)

slide-52
SLIDE 52

52

Steady-state analysis

slide-53
SLIDE 53

Offline-optimal lower bound

R R a3 a4 R “Oracle” policy: Keep in cache until (at least) the next inter-request arrival i whenever ai < R; otherwise, do not cache.

slide-54
SLIDE 54

Offline-optimal lower bound

R R a3 a4 R “Oracle” policy: Keep in cache until (at least) the next inter-request arrival i whenever ai < R; otherwise, do not cache.

Rate of new requests Cost ai

(per request)

Cost R

(per request)

slide-55
SLIDE 55

Offline-optimal lower bound

R R a3 a4 R “Oracle” policy: Keep in cache until (at least) the next inter-request arrival i whenever ai < R; otherwise, do not cache.

… [some steps] …

slide-56
SLIDE 56

56

Example dis istribution res esults

Exponential Erlang Deterministic Pareto Short-tailed Heavy-tailed

slide-57
SLIDE 57

57

Example dis istribution res esults

Exponential Erlang Deterministic Pareto Short-tailed Heavy-tailed

“Static baseline” policy: Either “always remote” or “always local”.

slide-58
SLIDE 58

58

Example dis istribution res esults

Exponential Erlang Deterministic Short-tailed

“Static baseline” policy: Either “always remote” or “always local”.

slide-59
SLIDE 59

59

Example dis istribution res esults

Exponential Erlang Deterministic Short-tailed

“Static baseline” policy: Either “always remote” or “always local”.

slide-60
SLIDE 60

60

Example dis istribution res esults

Exponential Erlang Deterministic Short-tailed

“Static baseline” policy: Either “always remote” or “always local”. … is online optimal for these cases!!

slide-61
SLIDE 61

61

Gap between onli line and offl fline optimal

slide-62
SLIDE 62

62

Gap between onli line and offl fline optimal

slide-63
SLIDE 63

63

However, not tr true for heavy-tailed …

… in fact, for Pareto the optimal static baseline can be far from optimal

slide-64
SLIDE 64

Policy analysis: Always on 1st

st

T R R T a3 a4 T R

slide-65
SLIDE 65

Policy analysis: Always on 1st

st

T R R T a3 a4 T R

slide-66
SLIDE 66

Policy analysis: Always on 1st

st

T R R T a3 a4 T R

slide-67
SLIDE 67

Policy analysis: Always on 1st

st

T R R T a3 a4 T R

slide-68
SLIDE 68

Policy analysis: Always on 1st

st

T R R T a3 a4 T R

slide-69
SLIDE 69

Policy analysis: Always on 1st

st

T R R T a3 a4 T R

slide-70
SLIDE 70

Policy analysis: Always on 1st

st

T R R T a3 a4 T R

No extension Extension case

slide-71
SLIDE 71

71

Results for example distributions

slide-72
SLIDE 72

72

Example dis istributions: : Summary ry of f costs

slide-73
SLIDE 73

73

Example dis istribution: : Exp xponential

  • Results with W = T = R
slide-74
SLIDE 74

74

Example dis istribution: : Exp xponential

  • Results with W = T = R
slide-75
SLIDE 75

Example dis istribution: : Exp xponential

  • Results with W = T = R
  • Window on 2nd performs good throughout
  • Window on 4th performs somewhat better for lower request

rates, but at an increased peak cost (at somewhat higher rates)

Always on Mth asymptotes at M/(M+1)

slide-76
SLIDE 76

Example dis istribution: : Exp xponential

  • Results with W = T = R
  • Window on 2nd performs good throughout
  • Window on 4th performs somewhat better for lower request

rates, but at an increased peak cost (at somewhat higher rates)

Always on Mth asymptotes at M/(M+1) Window on 2nd peaks at (1.052,1.588)

slide-77
SLIDE 77

Example dis istribution: : Exp xponential

  • Results with W = T = R
  • Window on 2nd performs good throughout
  • Window on 4th performs somewhat better for lower request

rates, but at an increased peak cost (at somewhat higher rates)

Always on Mth asymptotes at M/(M+1) Window on 2nd peaks at (1.052,1.588) Static peaks at (1,1.582)

slide-78
SLIDE 78

Example dis istribution: : Exp xponential

  • Results with W = T = R
  • Window on 2nd performs good throughout
  • Window on 4th performs somewhat better for lower request

rates, but at an increased peak cost (at somewhat higher rates)

Always on Mth asymptotes at M/(M+1) Window on 2nd peaks at (1.052,1.588) Static peaks at (1,1.582)

slide-79
SLIDE 79

Example dis istribution: : Exp xponential

  • Results with W = T = R
  • Window on 2nd performs good throughout
  • Window on 4th performs somewhat better for lower request

rates, but at an increased peak cost (at somewhat higher rates)

Always on Mth asymptotes at M/(M+1) Window on 2nd peaks at (1.052,1.588) Static peaks at (1,1.582)

slide-80
SLIDE 80

Example dis istribution: : Exp xponential

  • Results with W = T = R
  • Window on 2nd performs good throughout
  • Window on 4th performs somewhat better for lower request

rates, but at an increased peak cost (at somewhat higher rates)

Always on Mth asymptotes at M/(M+1) Window on 2nd peaks at (1.052,1.588) Static peaks at (1,1.582)

slide-81
SLIDE 81

81

Example dis istributions: : Low vari riabil ility dis istributions

  • Peak cost ratio for single-window on 2nd reduces as k increases

and inter-request times become increasingly deterministic (rightmost fig)

Erlang k=2 Erlang k=4 Deterministic Increasingly deterministic inter-request times

slide-82
SLIDE 82

82

Example dis istribution: : Pareto

  • As per Theorem 6.6, static baseline performs very poorly when

α  1 (and tm is small). E.g., large peak cost ratio in left-most fig

  • For larger α (e.g., α = 2), this peak reduces substantially.
  • Otherwise, the results are similar as for the other inter-request

distributions, suggesting that single-window on 2nd with T = R is a good choice

α =1.1 α =1.25 α = 2

slide-83
SLIDE 83

83

Multi-file evaluation

slide-84
SLIDE 84

84

Example dis istributions

  • Setup: 1,000,000 objects with Zipf popularity
  • Here, =1 (but results with =0.75 and =1.25 similar)
  • W = T = R
  • Significant benefits to being selective
  • Window on 2nd significantly outperforms always on Mth
  • Window on 2nd good throughout
  • Close to static optimal when Exponential and Erlang
  • Outperform static when Pareto
  • Has a peak cost-ratio of 1.4

Pareto, α =1.25 Exponential Erlang, k=4

slide-85
SLIDE 85

85

Trace-based si simulations

  • Setup: 20-month long university trace with YouTube viewings
  • 5.5 M video request to 2.4 M unique videos
  • Long tail of less popular videos
  • W = T = R
  • “Static” (highly optimistically) assumes “oracle” knowledge of which choice is

better (always local or always remote) for each individual video ...

  • Yet, window on 2nd outperform static
  • Highlights value of policy when request rates are unknown and variable
slide-86
SLIDE 86

86

Break-down of f cost contributions

  • Tail contribute to most of the costs ...

… highlighting importance of selective insertions.

Top (more than 20) Middle (4-20 views) Tail (1-3 views)

slide-87
SLIDE 87

87

Conclusions

slide-88
SLIDE 88

Conclusions

Worst-case bounds for the optimal cost and competitive cost ratios

  • E.g., Best worst-case bounds of M+1 are achieved by selecting W = T = R

Average cost expressions and bounds

  • Arbitrary inter-request distributions
  • Example inter-request distributions (both short-tailed and heavy-tailed)
  • Static is online optimal for constant and decreasing hazard rates, but can be arbitrarily

bad when heavy tailed or request rates are not known

Numeric and trace-based evaluations reveal insights into the relative cost performance of the policies

  • Substantial cost benefits of using window-based with intermediate M (e.g., 2-4) and the
  • ptimal worst-case parameter setting (i.e., W = T = R)

Window-based cache on 2nd request policy using a single threshold optimized to minimize worst-case costs provides good average performance

  • Attractive choice for a wide range of practical conditions where request rates of

individual file objects typically are not known and can change quickly ...

slide-89
SLIDE 89

Niklas Carlsson (niklas.carlsson@liu.se)

Thanks for listening!

Worst-case Bounds and Optimized Cache on Mth Request Cache Insertion Policies under Elastic Conditions

Niklas Carlsson and Derek Eager