E2 E2E: E: Em Embr bracing ng Use ser Heterogene neity y to - - PowerPoint PPT Presentation

▶
e2 e2e e em embr bracing ng use ser heterogene neity y to
SMART_READER_LITE
LIVE PREVIEW

E2 E2E: E: Em Embr bracing ng Use ser Heterogene neity y to - - PowerPoint PPT Presentation

May 30, 2023 326 likes •710 views

E2 E2E: E: Em Embr bracing ng Use ser Heterogene neity y to Im Impr prove e Qu Quality ality of Exper erien ience e on th the e Web eb Xu Zhang 1 , Siddhartha Sen 2 , Daniar Kurniawan 1 , Haryadi Gunawi 1 , Junchen Jiang 1 1

slide-1
SLIDE 1

E2 E2E: E: Em Embr bracing ng Use ser Heterogene neity y to Im Impr prove e Qu Quality ality of Exper erien ience e on th the e Web eb

Xu Zhang1, Siddhartha Sen2, Daniar Kurniawan1, Haryadi Gunawi1, Junchen Jiang1

1University of Chicago, 2Microsoft Research

1 August 22, 2019

slide-2
SLIDE 2

Pa Page lo load ad tim time ma matters!

2

Users are happier with faster page load time!

August 22, 2019

slide-3
SLIDE 3

Con Convention

  • nal wi

wisdo dom

Cut all server-side processing delays

  • Minimize mean delay
  • Minimize P99 delay
  • Minimize rate of missing a deadline

3 August 22, 2019

slide-4
SLIDE 4

Q: Q: Sho Shoul uld we we tr trea eat all all re requests in in th the SA SAME wa way?

4 August 22, 2019

slide-5
SLIDE 5

Expe Experi rime ment: t: Ob Obser serve th the difference ce in in th the qua quality of

  • f pa

page lo load ad ev events

5 August 22, 2019

slide-6
SLIDE 6

August 22, 2019 6

Ca Can yo you se see si signi nificant im improvem emen ent? t?

slide-7
SLIDE 7

Se Set #A #A

7 August 22, 2019

slide-8
SLIDE 8

Se Set #A #A: Be Befor

  • re im

improvem emen ent

8 August 22, 2019

slide-9
SLIDE 9

Se Set #A #A: Af After im improvem emen ent

9 August 22, 2019

slide-10
SLIDE 10

August 22, 2019 10

Ca Can yo you se see si signi nificant im improvem emen ent? t?

slide-11
SLIDE 11

Se Set #B #B

11 August 22, 2019

slide-12
SLIDE 12

Se Set #2 #2: Be Befor

  • re im

improvem emen ent

12 August 22, 2019

slide-13
SLIDE 13

Se Set #2 #2: Af After im improvem emen ent

13 August 22, 2019

slide-14
SLIDE 14

August 22, 2019 14

Ca Can yo you se see si signi nificant im improvem emen ent? t?

slide-15
SLIDE 15

Do Does it it me mean Se Set #B #B ha has a bi bigger de delay reduct ction?

15 August 22, 2019

slide-16
SLIDE 16

No! No! Th The tw two se sets ts im improved ed by by th the SA SAME am amount of

  • f de

delay! y!

16 August 22, 2019

slide-17
SLIDE 17

Sa Same de delay reduct ction, but but di different im improvements

In sensitive region, people are more sensitive to additional delay.

17

Total delay QoE

Set #B

August 22, 2019

Sensitive region Set #A

slide-18
SLIDE 18

Re Requests ha have di different se sensit sitivit ivitie ies to to ad addit ition ional al de delay

18 August 22, 2019

Total delay QoE too fast to matter sensitive too slow to matter

slide-19
SLIDE 19

Re Requests ha have di different se sensit sitivit ivitie ies to to ad addit ition ional al de delay

Analysis from Microsoft online store traces and user study on MTurk

19 August 22, 2019

Total delay QoE too fast to matter sensitive too slow to matter

slide-20
SLIDE 20

Idea: Idea: Fo Focusing on

  • n mor

more se sensi sitive re request sts

20

Total delay QoE

Conventional

Total delay QoE

E2E

Treating requests equally Focusing on more sensitive requests

August 22, 2019

slide-21
SLIDE 21

Da Data ce center wi witho hout ut E2 E2E

21

Shared-resource service Frontend web server

WAN

(last-mile, ISP)

Request

(browser)

external delay server-side delay total delay Data center

August 22, 2019

slide-22
SLIDE 22

Da Data ce center wi with E2 E2E

22

Shared-resource service Frontend web server

WAN

(last-mile, ISP)

Request

(browser)

external delay server-side delay total delay Data center E2E

External delay Resource allocation decision

August 22, 2019

slide-23
SLIDE 23

Po Potential ga gain

  • We reshuffle the server-side delays between concurrent requests
  • More sensitive requests get smaller delays

23 August 22, 2019

Default E2E QoE 20% higher QoE! Default E2E Throughput 40% higher throughput!

slide-24
SLIDE 24

Ou Our op

  • pport
  • rtunity
  • Current content providers do not distinguish the requests.

24

Total delay (sec.) Server-side delay (sec.)

1 2 3 4 5 6 0.3 0.6 0.9 1.2

August 22, 2019

E2E Default

slide-25
SLIDE 25

Ca Case st study: re replica se selection

  • Assign sensitive requests to the fast replica

25

Default policy: Load balanced

Sensitive requests Insensitive requests

Load balancer Replica 1 Replica 2

August 22, 2019

slide-26
SLIDE 26

Ca Case st study: re replica se selection

  • Assign sensitive requests to the fast replica

26

Default policy: Load balanced E2E: Unbalanced load distribution

Sensitive requests Insensitive requests

Load balancer Replica 1 Replica 2 Load balancer Replica 1 Replica 2

August 22, 2019

slide-27
SLIDE 27

Ho How do do we we dec decide de a re request st‘s se sensi sitivity? y?

  • Goal: Sensitive requests will be sent to fast replicas
  • Challenge: a request’s sensitivity is not an inherent property
  • Strawman: A request’s sensitivity is the slope of this request’s external delay
  • Observation: The optimal replica selection depends on the server-

side delay distribution.

August 22, 2019 27

QoE Delay

𝑡" 𝑡#

QoE Delay

𝑡" 𝑡#

A B A B

𝑡# 𝑡"

Server-side delay distribution

slide-28
SLIDE 28

Ho How to to se select re replicas fo for he heter erogeno enous us re request sts?

  • Send requests to replicas
  • Maximize ∑% 𝑅𝑝𝐹 𝑓𝑦𝑢𝑓𝑠𝑜𝑏𝑚_𝑒𝑓𝑚𝑏𝑧% + 𝑡𝑓𝑠𝑤𝑓𝑠_𝑒𝑓𝑚𝑏𝑧%
  • Classical maximum bipartite graph matching problem

August 22, 2019 28

QoE Delay External delays of requests 𝑠𝑓𝑟# 𝑠𝑓𝑟" 𝑠𝑓𝑟6 𝑠𝑓𝑟7 𝑠𝑓𝑟8 x y Server-side delay 700 ± 100 ms 300 ± 50 ms

slide-29
SLIDE 29

Ne Need to to re reduce th the dec decision-ma making ov

  • verhead!

Reduce the time consumption of running request-replica matching algorithm Reduce the frequency of decision-making

29 August 22, 2019

slide-30
SLIDE 30

Id Idea ea #1 #1: Gr Grou

  • upin

ing re requests by by the their ex external de delays

  • spatial coarsening of E2E decision-making

30

External delay QoE

I II III IV V VI

Requests I Replicas

…

II VI

…

August 22, 2019

slide-31
SLIDE 31

Idea Idea #2 #2: Re Reducing dec decision upda update fr frequency

  • Temporal coarsening of E2E decision making
  • Cache decision
  • No need to compute the table per request

31

External delay Cached Decision <500ms Replica_x 500-1200ms Replica_y >1200ms Replica_x

August 22, 2019

slide-32
SLIDE 32

Ev Evaluation

Set up Dataset: Real-world external delays from Microsoft traces Benchmark Default: Load balanced replica selection Idealized: Server-side delay is zero Performance evaluation Overall performance E2E vs prior work E2E’s overhead

32 August 22, 2019

slide-33
SLIDE 33

Ov Overall per performanc nce of

  • f E2

E2E

Microsoft trace: reshuffle the server-side delays vs default server-side delays Distributed database: replica selection in Cassandra

33

QoE gain over default (%) Microsoft Trace Cassandra

20 15 10 5

Idealized (Zero server-side delay) E2E (Ours)

August 22, 2019

slide-34
SLIDE 34

E2 E2E vs vs Pr Prior wo work

E2E vs deadline-driven algorithm (Timecard [SOSP’13] ) Timecard: shortest-remaining time first

34

Total delay deadline set by Timecard (sec.) QoE gain (%)

15 10 5 2.0 3.4 5.9

August 22, 2019

E2E Timecard

slide-35
SLIDE 35

E2 E2E’ E’s ov

  • verhead
  • Machines in testbed: 3.0GHz Intel Xeon processor, 2GB RAM, 2GB

RAM, 146G HDD and 1Gbps Ethernet link.

35

Time consumption per request (ms) Additional memory Additional CPU QoE gain E2E (basic) ~100,000 ~100% >100% 11.8% E2E w/ grouping requests & cache decision

~0.1 ~7% ~2% 10.4%

August 22, 2019

slide-36
SLIDE 36

De Demo: mo: Ho How E2 E2E wo works

36 August 22, 2019

slide-37
SLIDE 37

Con Conclus usion

Concurrent users have different sensitivities to server-side delays Key idea: Embracing heterogenous user sensitivities leads to higher QoE E2E: A concrete design to improve web QoE by allocating resource in accordance to user sensitivity E2E improves QoE by up-to 15.4%, with negligible computing overhead

37 August 22, 2019

More details about E2E can be found in: https://people.cs.uchicago.edu/~zhangxu/e2e.html