E2 E2E: E: Em Embr bracing ng Use ser Heterogene neity y to - PowerPoint PPT Presentation

E2 E2E: E: Em Embr bracing ng Use ser Heterogene neity y to Im Impr prove e Qu Quality ality of Exper erien ience e on th the e Web eb Xu Zhang 1 , Siddhartha Sen 2 , Daniar Kurniawan 1 , Haryadi Gunawi 1 , Junchen Jiang 1 1 University of Chicago, 2 Microsoft Research August 22, 2019 1

Pa Page lo load ad tim time ma matters! Users are happier with faster page load time! August 22, 2019 2

Con Convention onal wi wisdo dom Cut all server-side processing delays - Minimize mean delay - Minimize P99 delay - Minimize rate of missing a deadline August 22, 2019 3

Q: Q: Sho Shoul uld we we tr trea eat all all re requests in in th the SA SAME wa way? August 22, 2019 4

Expe Experi rime ment: t: Ob Obser serve th the difference ce in in th the qua quality of of pa page lo load ad ev events August 22, 2019 5

Ca Can yo you se see si signi nificant im improvem emen ent? t? August 22, 2019 6

Se Set #A #A August 22, 2019 7

Se Set #A #A: Be Befor ore im improvem emen ent August 22, 2019 8

Se Set #A #A: Af After im improvem emen ent August 22, 2019 9

Ca Can yo you se see si signi nificant im improvem emen ent? t? August 22, 2019 10

Se Set #B #B August 22, 2019 11

Se Set #2 #2: Be Befor ore im improvem emen ent August 22, 2019 12

Se Set #2 #2: Af After im improvem emen ent August 22, 2019 13

Ca Can yo you se see si signi nificant im improvem emen ent? t? August 22, 2019 14

Do Does it it me mean Se Set #B #B ha has a bi bigger de delay reduct ction? August 22, 2019 15

No! No! Th The tw two se sets ts im improved ed by by th the SA SAME am amount of of de delay! y! August 22, 2019 16

Sa Same de delay reduct ction, but but di different im improvements In sensitive region, people are more sensitive to additional delay. Sensitive region Set #B QoE Set #A Total delay August 22, 2019 17

Re Requests ha have di different se sensit sitivit ivitie ies to to ad addit ition ional al de delay too fast to matter sensitive too slow to matter QoE Total delay August 22, 2019 18

Re Requests ha have di different se sensit sitivit ivitie ies to to ad addit ition ional al de delay Analysis from Microsoft online store traces and user study on MTurk too fast to matter sensitive too slow to matter QoE Total delay August 22, 2019 19

Idea: Fo Idea: Focusing on on mor more se sensi sitive re request sts Conventional E2E Treating requests Focusing on more equally sensitive requests QoE QoE Total delay Total delay August 22, 2019 20

Data ce Da center wi witho hout ut E2 E2E Data center Request (browser) WAN (last-mile, ISP) Frontend web Shared-resource server service external delay server-side delay total delay August 22, 2019 21

Da Data ce center wi with E2 E2E Data center E2E Resource allocation External delay decision Request (browser) WAN (last-mile, ISP) Frontend web Shared-resource server service external delay server-side delay total delay August 22, 2019 22

Po Potential ga gain We reshuffle the server-side delays between concurrent requests • More sensitive requests get smaller delays • Throughput 40% higher 20% higher QoE! throughput! QoE E2E Default E2E Default August 22, 2019 23

Ou Our op opport ortunity Current content providers do not distinguish the requests. • E2E Default Server-side delay (sec.) 1.2 0.9 0.6 0.3 5 6 1 2 3 4 Total delay (sec.) August 22, 2019 24

Ca Case st study: re replica se selection Assign sensitive requests to the fast replica • Sensitive requests Insensitive requests Replica 1 Load balancer Replica 2 Default policy: Load balanced August 22, 2019 25

Ca Case st study: re replica se selection Assign sensitive requests to the fast replica • Sensitive requests Insensitive requests Replica 1 Replica 1 Load balancer Load balancer Replica 2 Replica 2 E2E: Unbalanced load distribution Default policy: Load balanced August 22, 2019 26

Ho How do do we we dec decide de a re request st‘s se sensi sitivity? y? Goal : Sensitive requests will be sent to fast replicas • Challenge : a request’s sensitivity is not an inherent property • • Strawman: A request’s sensitivity is the slope of this request’s external delay Observation : The optimal replica selection depends on the server- • side delay distribution. 𝑡 " 𝑡 # Server-side delay distribution 𝑡 " 𝑡 # A A 𝑡 # QoE QoE B B 𝑡 " Delay Delay August 22, 2019 27

Ho How to to se select re replicas fo for he heter erogeno enous us re request sts? Send requests to replicas • Maximize ∑ % 𝑅𝑝𝐹 𝑓𝑦𝑢𝑓𝑠𝑜𝑏𝑚_𝑒𝑓𝑚𝑏𝑧 % + 𝑡𝑓𝑠𝑤𝑓𝑠_𝑒𝑓𝑚𝑏𝑧 % • Classical maximum bipartite graph matching problem • Server-side delay 𝑠𝑓𝑟 # External delays of requests x 300 ± 50 ms 𝑠𝑓𝑟 " QoE 𝑠𝑓𝑟 6 𝑠𝑓𝑟 7 y 700 ± 100 ms Delay 𝑠𝑓𝑟 8 August 22, 2019 28

Ne Need to to re reduce th the dec decision-ma making ov overhead! Reduce the time consumption of running request-replica matching algorithm Reduce the frequency of decision-making August 22, 2019 29

Id Idea ea #1 #1: Gr Grou oupin ing re requests by by the their ex external de delays spatial coarsening of E2E decision-making • Requests Replicas I QoE II … … I II III IV V VI External delay VI August 22, 2019 30

Idea Idea #2 #2: Re Reducing dec decision upda update fr frequency Temporal coarsening of E2E decision making • Cache decision • No need to compute the table per request • External delay Cached Decision <500ms Replica_x 500-1200ms Replica_y >1200ms Replica_x August 22, 2019 31

Ev Evaluation Set up Dataset: Real-world external delays from Microsoft traces Benchmark Default: Load balanced replica selection Idealized: Server-side delay is zero Performance evaluation Overall performance E2E vs prior work E2E’s overhead August 22, 2019 32

Overall per Ov performanc nce of of E2 E2E Microsoft trace: reshuffle the server-side delays vs default server-side delays Distributed database: replica selection in Cassandra QoE gain over default (%) E2E (Ours) 20 Idealized (Zero server-side delay) 15 10 5 Microsoft Cassandra Trace August 22, 2019 33

E2 E2E vs vs Pr Prior wo work E2E vs deadline-driven algorithm (Timecard [SOSP’13] ) Timecard: shortest-remaining time first E2E Timecard 15 QoE gain (%) 10 5 2.0 3.4 5.9 Total delay deadline set by Timecard (sec.) August 22, 2019 34

E2 E2E’ E’s ov overhead Machines in testbed: 3.0GHz Intel Xeon processor, 2GB RAM, 2GB • RAM, 146G HDD and 1Gbps Ethernet link. Time consumption Additional Additional QoE gain per request (ms) memory CPU E2E (basic) ~100,000 ~100% >100% 11.8% E2E w/ grouping requests ~0.1 ~7% ~2% 10.4% & cache decision August 22, 2019 35

De Demo: mo: Ho How E2 E2E wo works August 22, 2019 36

Con Conclus usion Concurrent users have different sensitivities to server-side delays Key idea: Embracing heterogenous user sensitivities leads to higher QoE E2E: A concrete design to improve web QoE by allocating resource in accordance to user sensitivity E2E improves QoE by up-to 15.4%, with negligible computing overhead More details about E2E can be found in: https://people.cs.uchicago.edu/~zhangxu/e2e.html August 22, 2019 37