Fluxo Improving the Responsiveness of Internet Services with - - PowerPoint PPT Presentation

▶
fluxo
SMART_READER_LITE
LIVE PREVIEW

Fluxo Improving the Responsiveness of Internet Services with - - PowerPoint PPT Presentation

May 21, 2023 352 likes •991 views

Fluxo Improving the Responsiveness of Internet Services with Automa7c Cache Placement Alexander Rasmussen UCSD (Presen7ng) Emre Kiciman MSR Redmond Benjamin Livshits MSR Redmond Madanlal Musuvathi MSR Redmond 1 Caching in

slide-1
SLIDE 1

Fluxo

Improving the Responsiveness of Internet Services with Automa7c Cache Placement

Alexander Rasmussen – UCSD (Presen7ng) Emre Kiciman – MSR Redmond Benjamin Livshits – MSR Redmond Madanlal Musuvathi – MSR Redmond

1

slide-2
SLIDE 2

Caching in Internet Services

  • Sa7sfying user request involves calling many

external components, aggrega7ng data

  • Want to cache computa7on performed by

some components to improve performance

– Disk‐intensive opera7ons, DB queries, etc.

  • What you cache and when depends on a

number of factors

– Workload, architecture, SLAs, ...

2 2

slide-3
SLIDE 3

Caching in Internet Services

  • Choice of what, where, how much to cache is

usually very ad‐hoc

– Programmer intui7on – Localized profiling

  • “Best” choice can change rapidly over 7me;

too quickly for humans to respond manually

  • Need an automa7c solu7on!

3 3

slide-4
SLIDE 4

Fluxo ‐ Automa7c Cache Op7miza7on

  • Describe Internet service as dataflow graph
  • Gather run7me request traces
  • Simulate and op7mize to converge on

reasonably good cache placement policy

4

+

... <1234567890, B, C, x> <1234567941, C, D, x’> <1234567890, A, B, x’’> <1234567892, B, E, y> <1234567921, E, C, y’> ...

Fluxo

<{A, B}, 50 MB> <{B, E, C}, 256 KB>

A B E C D

4

slide-5
SLIDE 5

Fluxo Dataflow Graphs

  • Source node produces request as tuple
  • Sink node consumes response as tuple
  • All other nodes are components which may call

external services

5 5

slide-6
SLIDE 6

Sample Service ‐ Weather Report

6

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

6

slide-7
SLIDE 7

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

7

slide-8
SLIDE 8

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

137.110.222.250 92093

7

slide-9
SLIDE 9

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

137.110.222.250 92093

7

slide-10
SLIDE 10

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

137.110.222.250 92093

7

slide-11
SLIDE 11

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

137.110.222.250 92093

7

slide-12
SLIDE 12

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

92093 La Jolla, CA

7

slide-13
SLIDE 13

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

92093 La Jolla, CA

7

slide-14
SLIDE 14

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

92093 72 F, Sunny

7

slide-15
SLIDE 15

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

92093 72 F, Sunny

7

slide-16
SLIDE 16

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

92093 72 F, Sunny

7

slide-17
SLIDE 17

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

92093 <p>72 F, Sunny</p>

7

slide-18
SLIDE 18

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

92093 <p>72 F, Sunny</p>

7

slide-19
SLIDE 19

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

92093

7

slide-20
SLIDE 20

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

72 F, Sunny

7

slide-21
SLIDE 21

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

72 F, Sunny

7

slide-22
SLIDE 22

Weather Service ‐ Sample Input

7

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

7

slide-23
SLIDE 23

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

Cache Contents:

8

slide-24
SLIDE 24

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

137.110.222.250 92093

Cache Contents:

8

slide-25
SLIDE 25

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

137.110.222.250 92093

Cache Contents:

8

slide-26
SLIDE 26

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

137.110.222.250 92093

Cache Contents:

8

slide-27
SLIDE 27

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

137.110.222.250 92093

Cache Contents:

8

slide-28
SLIDE 28

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

137.110.222.250 92093

Cache Contents:

8

slide-29
SLIDE 29

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

92093 La Jolla, CA

Cache Contents:

8

slide-30
SLIDE 30

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

92093 La Jolla, CA

Cache Contents:

8

slide-31
SLIDE 31

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

92093 72 F, Sunny

Cache Contents:

8

slide-32
SLIDE 32

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

92093 72 F, Sunny

Cache Contents:

8

slide-33
SLIDE 33

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

92093 72 F, Sunny

137.110.222.250 : “72 F, Sunny” Cache Contents:

8

slide-34
SLIDE 34

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

92093 72 F, Sunny

137.110.222.250 : “72 F, Sunny” Cache Contents:

8

slide-35
SLIDE 35

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

92093 <p>72 F, Sunny</p>

137.110.222.250 : “72 F, Sunny” Cache Contents:

8

slide-36
SLIDE 36

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

92093 <p>72 F, Sunny</p>

137.110.222.250 : “72 F, Sunny” Cache Contents:

8

slide-37
SLIDE 37

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

92093

137.110.222.250 : “72 F, Sunny” Cache Contents:

8

slide-38
SLIDE 38

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

72 F, Sunny

137.110.222.250 : “72 F, Sunny” Cache Contents:

8

slide-39
SLIDE 39

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

72 F, Sunny

137.110.222.250 : “72 F, Sunny” Cache Contents:

8

slide-40
SLIDE 40

Caching {IP to City, City to Weather}

8

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

137.110.222.250 : “72 F, Sunny” Cache Contents:

8

slide-41
SLIDE 41

Caching {IP to City, City to Weather}

9

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

137.110.222.250 : “72 F, Sunny” Cache Contents:

9

slide-42
SLIDE 42

Caching {IP to City, City to Weather}

9

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

137.110.222.250 92093

137.110.222.250 : “72 F, Sunny” Cache Contents:

9

slide-43
SLIDE 43

Caching {IP to City, City to Weather}

9

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

137.110.222.250 92093

137.110.222.250 : “72 F, Sunny” Cache Contents:

9

slide-44
SLIDE 44

Caching {IP to City, City to Weather}

9

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

137.110.222.250 92093

137.110.222.250 : “72 F, Sunny” Cache Contents:

9

slide-45
SLIDE 45

Caching {IP to City, City to Weather}

9

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

137.110.222.250 92093

137.110.222.250 : “72 F, Sunny” Cache Contents:

9

slide-46
SLIDE 46

Caching {IP to City, City to Weather}

9

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

92093 72 F, Sunny

137.110.222.250 : “72 F, Sunny” Cache Contents:

9

slide-47
SLIDE 47

Caching {IP to City, City to Weather}

9

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

92093 72 F, Sunny

137.110.222.250 : “72 F, Sunny” Cache Contents:

9

slide-48
SLIDE 48

Caching {IP to City, City to Weather}

9

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

92093 72 F, Sunny

137.110.222.250 : “72 F, Sunny” Cache Contents:

9

slide-49
SLIDE 49

Caching {IP to City, City to Weather}

9

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

92093 <p>72 F, Sunny</p>

137.110.222.250 : “72 F, Sunny” Cache Contents:

9

slide-50
SLIDE 50

Caching {IP to City, City to Weather}

9

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

92093 <p>72 F, Sunny</p>

137.110.222.250 : “72 F, Sunny” Cache Contents:

9

slide-51
SLIDE 51

Caching {IP to City, City to Weather}

9

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

92093

137.110.222.250 : “72 F, Sunny” Cache Contents:

9

slide-52
SLIDE 52

Caching {IP to City, City to Weather}

9

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

72 F, Sunny

137.110.222.250 : “72 F, Sunny” Cache Contents:

9

slide-53
SLIDE 53

Caching {IP to City, City to Weather}

9

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

72 F, Sunny

137.110.222.250 : “72 F, Sunny” Cache Contents:

9

slide-54
SLIDE 54

Caching {IP to City, City to Weather}

9

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink $

137.110.222.250 : “72 F, Sunny” Cache Contents:

9

slide-55
SLIDE 55

Simplifying Assump7ons

  • Data center, single administra7ve domain
  • Caching provided by cluster of caching servers
  • Service runs on single machine, makes calls to external

services during execu7on

  • Goal: allocate B total bytes from cache servers to a service

10

C C C

Cache (B Bytes Total)

10

slide-56
SLIDE 56

Fluxo Components

  • Fluxo Run7me

– Provides tracing and simula7on func7onality – Produces ordered stream of events as service runs

  • Fluxo Op7mizer

– Takes stream of events and service graph, produces a caching policy: {<service subgraph, cache size> pairs} – Evaluates N random cache policies, hill‐climbs from the top K policies

  • In our experiments, N=20,000 , K=200

– To evaluate a policy, simulate its performance on recorded event stream

11 11

slide-57
SLIDE 57

Evalua7on ‐ Reference Policies

12

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

Random (sample)

9% 25% 64%

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

All‐Encompassing

100%

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

Uniform (subset)

2%

12

slide-58
SLIDE 58

Results

13

31% 4% 65%

Frequency of Occurrence Input Value

  • vs. Random:
  • vs. Uniform:
  • vs. All‐Encompassing:

+5% +6% +5%

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

Median Latency Improvement:

13

slide-59
SLIDE 59

Results

14

13% 13% 52% 13%

Frequency of Occurrence Input Value

  • vs. Random:
  • vs. Uniform:
  • vs. All‐Encompassing:

+12% +15% +3%

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

Median Latency Improvement:

14

slide-60
SLIDE 60

Results

15

9% 22% 62%

Frequency of Occurrence Input Value

  • vs. Random:
  • vs. Uniform:
  • vs. All‐Encompassing:

+12% +17% +1%

Zip Code to Weather Source Split Join Build HTML IP to City City to Weather Sink

Median Latency Improvement:

15

slide-61
SLIDE 61

Future Work

  • Evalua7on on real service with real workload
  • Scaling op7mizer’s analysis

– Considering parallelized analysis, more aggressive result memoiza7on, more sophis7cated ML

  • Seems hard to beat all‐encompassing cache

– Might be an ar7fact of test service

  • Impera7ve programs?

16 16

slide-62
SLIDE 62

Conclusion

  • Fluxo:

– Dataflow model of Internet services – Run7me tracing + model = caching policy – Simula7on and search to converge on good policy

17

Thanks to John Wilkes for shepherding this work, and to MSR for travel funding

17