eliminating adverse control plane interactions in
play

Eliminating Adverse Control Plane Interactions in Independent - PowerPoint PPT Presentation

Feb 14, 2024 •393 likes •1.71k views

Eliminating Adverse Control Plane Interactions in Independent Network Systems Matthew K. Mukerjee Computer Science PhD Thesis Defense May 1st, 2018 Network Control CDN server selection Routing Congestion Control VM migration Network

  1. Challenge: Difficult to schedule Workloads workloads Solution: App-specific Modification Name Node 1 1 1 2 2 Data Node Data Node Data Node Data Node 2 Data Node Data Node Data Node Data Node 3 3 Data Node Data Node Data Node Data Node 3 Data Node Data Node Data Node Data Node Rack A Rack B Rack C Rack D

  2. Challenge: Difficult to schedule Workloads workloads Solution: App-specific Modification Name Node 1 1 1 2 2 Data Node Data Node Data Node Data Node 2 Data Node Data Node Data Node Data Node 3 3 Data Node Data Node Data Node Data Node 3 Data Node Data Node Data Node Data Node Rack A Rack B Rack C Rack D

  3. workloads Name Node 1 1 1 2 2 Data Node Data Node Data Node Data Node 2 Data Node Data Node Data Node Data Node 3 3 Data Node Data Node Data Node Data Node 3 Data Node Data Node Data Node Data Node Rack A Rack B Rack C Rack D A —> B A —> C A —> D B —> C B —> D B —> A RECONFIG RECONFIG RECONFIG Schedule: C —> D C —> A C —> B DELAY DELAY DELAY D —> A D —> B D —> C Config 1 Config 2 Config 3

  4. Name Node 1 1 1 2 2 Data Node Data Node Data Node Data Node 2 Data Node Data Node Data Node Data Node 3 3 Data Node Data Node Data Node Data Node 3 Data Node Data Node Data Node Data Node Rack A Rack B Rack C Rack D A —> B A —> C A —> D B —> C B —> D B —> A RECONFIG RECONFIG RECONFIG Schedule: C —> D C —> A C —> B DELAY DELAY DELAY D —> A D —> B D —> C Config 1 Config 2 Config 3

  5. Challenge: Workloads reHDFS Solution: App-specific 1 Modification Name Node 1 1 2 2 Data Node Data Node Data Node Data Node 3 2 Data Node Data Node Data Node Data Node 3 3 Data Node Data Node Data Node Data Node Data Node Data Node Data Node Data Node Rack A Rack B Rack C Rack D A —> C B —> D Schedule: C —> A D —> B Config 1

  6. Challenge: reHDFS reduces Workloads tail latency Solution: App-specific Modification 9x decrease in write time 100 - CDF (%) 50 - HDFS reHDFS 0 - - - - - 500 0 1000 1500 HDFS write completion time (ms)

  7. Control Coordination Scenario: Scenario: Scenario: Layering Admin Scalability Etalon VDX VDN App TE + ISP TE Co fm ow Reaction Transparency BGP + BGP Internet-scale Routing Priority Hierarchical Ranking Partitioning

  8. Control Coordination Scenario: Scenario: Admin Scalability VDX VDN Etalon Co fm ow App TE + ISP TE Reaction Transparency BGP + BGP Internet-scale Routing Priority Hierarchical Scenario: Layering Ranking Partitioning

  9. Control Coordination Scenario: Scenario: Admin Scalability VDX VDN Etalon Co fm ow App TE + ISP TE Reaction Transparency BGP + BGP Internet-scale Routing Priority Hierarchical Scenario: Layering Ranking Partitioning

  10. Traditional Content Delivery Legend: Content Provider (CP) Content CDN Client

  11. Changing Content Delivery Legend: Content Provider (CP) Content CDN CDN Client Client Client

  12. Brokered Content Delivery Legend: Content Provider (CP) Content Control CDN CDN Broker Client Client Client

  13. Brokered Content Delivery Legend: Content Provider (CP) Content Control CDN CDN Broker Easier for CPs to meet performance and cost goals Client Client Client

  14. Brokered Content Delivery Legend: Content Provider (CP) Content Control Brokers select “best” CDN for clients to minimize cost and meet performance goals CDN CDN Broker Client Client Client B B B

  15. Brokered Content Delivery Legend: How do brokers and CDNs Content Provider (CP) Content impact each other? Control (this talk) CDN CDN Broker Client Client Client B B B

  16. Contributions • Identify challenges that brokers and CDNs create for each other by analyzing data from both • Examine the design space of CDN-broker interfaces • Evaluate the efficacy of different designs

  17. CDN Cost and Pricing Legend: Content Provider (CP) Content CDN Internal Costs: Bandwidth (mostly) Client

  18. CDN Cost and Pricing Legend: Content Provider (CP) Content Do bandwidth costs differ across geographic regions? CDN Internal Costs: Bandwidth (mostly) Client

  19. CDN Cost / Byte Delivered 30x difference in cost per byte between the most expensive and least expensive countries

  20. CDN Internal Cost CDN X CDN Y CDN X CDN X

  21. CDN Internal Cost CDN X $ CDN Y $ CDN X $ CDN X $$$$

  22. CDN External Price CDN Pricing CDN X Client Client $ Content Provider (CP) Client $$$ $$ Client CDN Y $ CDN X CDN X CDN Y $ Client Client Client CDN X $$$$

  23. CDN External Price CDN Pricing CDN X Client Client $ Content Provider (CP) Client $$$ $$ Client CDN Y $ CDN X CDN X CDN Y $ Client Client Client CDN Y makes money, CDN X loses money CDN X $$$$

  24. CDN External Price CDN Pricing CDN X Client Client $ Content Provider (CP) Client $$$ $$ Client CDN Y $ CDN X CDN X CDN Y $ Client Client Client Do we see traffic patterns like this at the country level? CDN X $$$$

  25. Other 0 CDN C CDN B CDN A % Used in Country 100 75 50 25 Country (Anonymized) 1 9 10 11 12 13 14 15 8 7 6 5 4 3 2 Country Level Traffic

  26. Other 0 CDN C CDN B CDN A % Used in Country 100 75 50 25 Country (Anonymized) 1 9 10 11 12 13 14 15 8 7 6 5 4 3 2 Country Level Traffic

  27. 1 6 100 75 50 25 0 Country (Anonymized) 9 10 11 12 13 14 15 8 7 5 CDN A 4 3 2 1 Other CDN C CDN B CDN A % Used in Country % Used in Country CDN B 75 Country (Anonymized) CDN C CDN B CDN A % Used in Country 100 75 50 25 0 9 10 11 12 13 14 15 CDN C 8 7 6 5 4 3 2 1 Other 100 50 2 50 2 1 Other CDN C CDN B CDN A % Used in Country 100 75 25 4 0 Country (Anonymized) 9 10 11 12 13 14 15 8 7 6 5 4 3 3 5 25 2 0 Country (Anonymized) 9 10 11 12 13 14 15 8 7 6 5 4 3 1 6 100 75 50 25 0 Country (Anonymized) 9 10 11 12 13 14 15 8 7 Other Country Level Traffic

  28. 100 6 100 75 50 25 0 Country (Anonymized) 9 10 11 12 13 14 15 8 7 5 CDN A 4 3 2 1 Other CDN C CDN B CDN A % Used in Country % Used in Country CDN B 75 Country (Anonymized) CDN C CDN B CDN A % Used in Country 100 75 50 25 0 9 10 11 12 13 14 15 CDN C 8 7 6 5 4 3 2 1 Other 1 50 Other 25 1 Other CDN C CDN B CDN A % Used in Country 100 75 50 0 3 Country (Anonymized) 9 10 11 12 13 14 15 8 7 6 5 4 3 2 2 4 25 2 0 Country (Anonymized) 9 10 11 12 13 14 15 8 7 6 5 4 3 1 5 100 75 50 25 0 Country (Anonymized) 9 10 11 12 13 14 15 8 7 6 Country 8 costly —> CDN B loses money! Country Level Traffic Country 7 cheap —> CDN A profits! Flat pricing makes CDN profits unpredictable with brokers

  29. Contributions • Identify challenges that brokers and CDNs create for each other by analyzing data from both • Examine the design space of CDN-broker interfaces • Evaluate the efficacy of different designs

  30. Brokered Delivery Today Legend: Content Provider (CP) Content Control CDN CDN Broker Client Client Client

  31. Brokered Delivery Today Content Provider (CP) CDN CDN Broker Client Client Client

  32. Brokered Delivery Today Content Provider (CP) CDN CDN Broker Client Client Client

  33. Brokered Delivery Today Content Provider (CP) CDN CDN Broker Client Client Client

  34. Brokered Delivery Today Content Provider (CP) CDN Broker CDN Client Client Client

  35. Brokered Delivery Today Content Provider (CP) CDN Broker CDN Client Client Client

  36. Brokered Delivery Today CDN Broker CDN Client Client Client

  37. Brokered Delivery Today CDN Broker CDN Latency & loss Measurements ISP, device type, location, … Client Client Client

  38. Brokered Delivery Today CDN Broker CDN Which cluster Which CDN to receive from to use Client Client Client

  39. Brokered Delivery Today CDN Broker CDN CDN Client Client Client

  40. VDX 3 2 1 CDN Broker CDN CDN Client Client Client

  41. Example CDN Pricing CDN X Client Client $ Content Provider (CP) Client $$$ $$ Client CDN Y $ CDN X CDN X CDN Y $ Client Client Client CDN X $$$$

  42. Example CDN Pricing CDN X Client Client $ Content Provider (CP) Client $$$$ $$ Client $$ CDN X CDN Y $ CDN X CDN Y CDN X $ Client Client Client CDN X $$$$

  43. Example CDN Pricing CDN X Client Client $ Content Provider (CP) Client $$$$ $$ Client $$ CDN X CDN Y $ CDN X CDN Y CDN X $ Client Client Client CDN X $$$$

  44. Example CDN Pricing CDN X Client Client $ Content Provider (CP) Client $$$$ $$ Client $$ CDN X CDN Y $ CDN X CDN Y CDN X $ Client Client Client CDN X can compete with other CDNs across regions CDN X $$$$

  45. Evaluation • Simulator using data from a broker & CDN, as well as public data from 13 other CDNs • CDN data provides cluster locations, cluster-to- client performance, delivery costs, etc. • Broker data provides client locations, request distributions, etc.

  46. Per-CDN Profits Profit Brokered Today VDX VDX 1 2 3 4 5 6 7 8 9 10 11 12 13 14 CDN

  47. Per-CDN Profits Profit Brokered Today VDX VDX 1 2 3 4 5 6 7 8 9 10 11 12 13 14 CDN

  48. Evaluation Takeaways • Today’s world (Brokered) is pretty broken (performance can be better; most CDNs lose money on brokered video delivery) • Marketplace (VDX) fixes this by exposing clusters and cost

  49. Control Coordination Scenario: Scenario: Admin Scalability VDX VDN Etalon Co fm ow App TE + ISP TE Reaction Transparency BGP + BGP Internet-scale Routing Priority Hierarchical Scenario: Layering Ranking Partitioning

  50. Control Coordination Scenario: Scenario: Admin Scalability VDX VDN Etalon Co fm ow App TE + ISP TE Reaction Transparency BGP + BGP Internet-scale Routing Priority Hierarchical Scenario: Layering Ranking Partitioning

  51. Control Coordination Scenario: Scalability VDN VDX Etalon BGP + BGP Co fm ow App TE + ISP TE Reaction Priority Transparency Internet-scale Routing Ranking Hierarchical Scenario: Scenario: Admin Layering Partitioning Some Full Information Sharing

  52. Control Coordination Scenario: Scalability VDN VDX Etalon BGP + BGP Co fm ow App TE + ISP TE Reaction Priority Transparency Internet-scale Routing Ranking Hierarchical Scenario: Scenario: Admin Layering Partitioning Some Full Information Sharing

  53. Live Video is Becoming Wildly Popular • Commercial sports streams • User-generated streams

  54. Live Video is Becoming Wildly Popular • Commercial sports streams • Single World Cup stream = 40% global Internet traffic • User-generated streams (e.g., Twitch) • Users watch 150b min of live video per month • Amazon buys Twitch for ~ $1Billion

  55. CDN Live Video Delivery Background Legend Control ▶︎ ◀ Data Video Requests: A B Sources Video 1 Video 2 HTTP RESPONSE Responses: Video 1 Reflector Video 2 Clusters C D HTTP GET Edge Clusters E F G Video Requests

  56. CDN Live Video Delivery Background Control ▶︎ ◀ Data Video A B Sources 120 100 100 3K 1K 2K Link Cost Reflector Clusters C D Link Capacity 25 15 DNS 200 20 15 300 750 2K Edge Clusters E F G 1 10 1 0 300 750 700 500 Clients H I J

  57. CDN Live Video Delivery Background Control ▶︎ ◀ Data Video A B Sources Objective: Link Cost Reflector Reasonable service quality Clusters C D Link Capacity DNS & Edge Clusters E F G Minimal delivery cost Clients H I J

  58. Problems with CDNs Today Service Quality Delivery Cost (per request) 8000 Optimal 7000 CDN Avg. Bitrate (Kbps) CDN 2.0x 6000 5000 4000 OPTIMAL 3000 1.0x 2000 1000 0 2 4 6 8 10 # of Videos (Thousands) Simulation using Conviva traces, Simulation using Conviva traces, modeling user-generated content modeling large sports events

  59. Problems with CDNs Today Service Quality Not Fine-Grained 8000 Optimal 7000 CDN Avg. Bitrate (Kbps) 6000 Videos aggregated 5000 4000 into large groups 3000 2000 1000 0 2 4 6 8 10 # of Videos (Thousands) Delivery Cost Slow DNS Updates CDN OPTIMAL Can’t push updates 2.0x 1.0x DNS entries get cached QUANTITATIVE QUALITATIVE

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Network Layer: Control Plane Goal: understand principles behind network control plane

Network Layer: Control Plane Goal: understand principles behind network control plane

Network Layer: Control Plane Goal: understand principles behind network control plane traditional (intra-domain) routing algorithms SDN controllers and their instantiation, implementation in the Internet: OSPF, RIP, OpenFlow, ODL

1.31k views • 75 slides
1 Background | Problems | Challenges | Design | Evaluation | Summary Control Plane Applications

1 Background | Problems | Challenges | Design | Evaluation | Summary Control Plane Applications

SPEED Resource-Ef+icient and High-Performance Deployment for Data Plane Programs Xiang Chen , Hongyan Liu, Qun Huang, Peiqiao Wang, Dong Zhang, Haifeng Zhou, Chunming Wu Control Plane Applications Monitor Security Routing Data Plane

826 views • 45 slides
Re-Thinking Adverse Noise Effects and Interactions with Other Exposures Dimitri Palilis, Au.D.

Re-Thinking Adverse Noise Effects and Interactions with Other Exposures Dimitri Palilis, Au.D.

2/28/2018 Re-Thinking Adverse Noise Effects and Interactions with Other Exposures Dimitri Palilis, Au.D. Director of Audiology, MedEvals of Calfornia Disclosure: I am a co-owner of Innovative Hearing Designs, a research lab which

378 views • 9 slides
Internet Control Plane Security Yongdae Kim KAIST Two Planes Data Plane: Actual data

Internet Control Plane Security Yongdae Kim KAIST Two Planes Data Plane: Actual data

Internet Control Plane Security Yongdae Kim KAIST Two Planes Data Plane: Actual data delivery Control Plane To support data delivery (efficiently, reliably, and etc.) Routing information exchange In some sense, every protocol

396 views • 39 slides
A TVWS ZigBee Prototype Cognitive Plane Control Plane Application Protocol Plane Customer

A TVWS ZigBee Prototype Cognitive Plane Control Plane Application Protocol Plane Customer

A TVWS ZigBee Prototype Cognitive Plane Control Plane Application Protocol Plane Customer API James Jody Neel Security Propagation benefits ZigBee james.neel@crtwireless.com 32- / 64- / 128-bit encryption Alliance might enable

773 views • 24 slides
Pratyaastha: An Efficient Elastic Distributed SDN Control Plane Anand Krishnamurthy, Shoban P.

Pratyaastha: An Efficient Elastic Distributed SDN Control Plane Anand Krishnamurthy, Shoban P.

Pratyaastha: An Efficient Elastic Distributed SDN Control Plane Anand Krishnamurthy, Shoban P. Chandrabose and Aaron Gember-Jackobson 1 Motivation Architecture Evaluation Summary SDN Control Plane Operator goals: 1. Better Performance

364 views • 9 slides
SCION: Control Plane Overview Adrian Perrig Network Security Group, ETH Zrich SCION Control

SCION: Control Plane Overview Adrian Perrig Network Security Group, ETH Zrich SCION Control

SCION: Control Plane Overview Adrian Perrig Network Security Group, ETH Zrich SCION Control Plane Overview Control plane: How to find and distribute end-to-end paths [Chapter 2.1, Chapter 7] Path exploration Path registration

411 views • 22 slides
Counting social interactions for discrete subsets of the plane Samantha Fairchild University of

Counting social interactions for discrete subsets of the plane Samantha Fairchild University of

Counting social interactions for discrete subsets of the plane Samantha Fairchild University of Washington skayf@uw.edu Overview The Golden L and holonomy vector population density 1 Counting closed geodesics via -orbits 2 Expected

784 views • 40 slides
Intelligent Optical Control Plane Ying ( Em ily) Hu Bell Labs-NPS, Advanced Netw ork

Intelligent Optical Control Plane Ying ( Em ily) Hu Bell Labs-NPS, Advanced Netw ork

Intelligent Optical Control Plane Ying ( Em ily) Hu Bell Labs-NPS, Advanced Netw ork Modeling&Optim ization Lucent Technologies Holm del, New Jersey, USA 1 Outline Optical Control Plane What is it Operator Drivers for Optical

415 views • 16 slides
Defined Eliminating barriers to adoption of IT Innovation Agenda t : 3 Keys to enabling

Defined Eliminating barriers to adoption of IT Innovation Agenda t : 3 Keys to enabling

Evolving towards Software Defined Eliminating barriers to adoption of IT Innovation Agenda t : 3 Keys to enabling innovation and transformation al Programmable data plane t Vertical disaggregation of network solutions

798 views • 40 slides
End Site Control Plane System (ESCPS) Network service to

End Site Control Plane System (ESCPS) Network service to

End-Site Control Plane Service (ESCaPeS) Monitoring with Perisc pe Mar8n Swany U. Delaware (Indiana University effec8ve 8.1.11) End Site

437 views • 19 slides
High-arity Interactions, Polyhedral Relaxations, and Cutting Plane Algorithm for Soft Constraint

High-arity Interactions, Polyhedral Relaxations, and Cutting Plane Algorithm for Soft Constraint

High-arity Interactions, Polyhedral Relaxations, and Cutting Plane Algorithm for Soft Constraint Optimisation (MAP-MRF) Tom a s Werner Center for Machine Perception Czech Technical University Prague, Czech Republic 1 / 18 Abstract

205 views • 18 slides
Optimal Control of Plane Poiseuille Flow Workshop on Flow Control, Poitiers 11-14th Oct 04 J.

Optimal Control of Plane Poiseuille Flow Workshop on Flow Control, Poitiers 11-14th Oct 04 J.

Optimal Control of Plane Poiseuille Flow Workshop on Flow Control, Poitiers 11-14th Oct 04 J. Mckernan , J.F .Whidborne , G.Papadakis Cranfield University, U.K. Kings College, London, U.K. Optimal Control of

572 views • 22 slides
Management of ASON- -capable capable Management of ASON Network and its Control Plane Network

Management of ASON- -capable capable Management of ASON Network and its Control Plane Network

I nternational Telecom m unication Union ITU-T Management of ASON- -capable capable Management of ASON Network and its Control Plane Network and its Control Plane H. Kam LAM Lucent Technologies I TU-T W orkshop NGN and its Transport

449 views • 28 slides
CONFIG Control Plane Architecture for 5G Systems Riccardo Trivisonno, Huawei Technologies 5G-PPP

CONFIG Control Plane Architecture for 5G Systems Riccardo Trivisonno, Huawei Technologies 5G-PPP

CONFIG Control Plane Architecture for 5G Systems Riccardo Trivisonno, Huawei Technologies 5G-PPP WS, Network Slicing and C-Plane Architecture and SDN for 5G Athens, February 6 th and 7 th , 2017 Presentation Outline Project Objectives

497 views • 11 slides
Information Model of Control-Plane and User-Plane separation BNG Michael Wang (Huawei) Jie Chen

Information Model of Control-Plane and User-Plane separation BNG Michael Wang (Huawei) Jie Chen

Information Model of Control-Plane and User-Plane separation BNG Michael Wang (Huawei) Jie Chen (Huawei) Rong Gu (China Mobile) IETF I2RS WG, March 2017, Chicago draft-wcg-i2rs-cu-separation-info-model-00 IETF98@Chicago Background BNG

377 views • 8 slides
Data Logistics in Network Computing Martin Swany Introduction and Motivation The goal of

Data Logistics in Network Computing Martin Swany Introduction and Motivation The goal of

Data Logistics in Network Computing Martin Swany Introduction and Motivation The goal of Computational Grids is to mimic the electric power grid for computing power Service-orientation to make computing power a utility Compute

783 views • 40 slides
In-Network Computing to the rescue of Faulty Links Acknowledgements: Isaac Pedisich (UPenn),

In-Network Computing to the rescue of Faulty Links Acknowledgements: Isaac Pedisich (UPenn),

In-Network Computing to the rescue of Faulty Links Acknowledgements: Isaac Pedisich (UPenn), Gordon Brebner (Xilinx), DARPA Contracts No. HR0011-17-C-0047 and HR0011-16-C-0056, and NSF grant CNS-1513679. Path Node 1 Node 2 2 Path Node

466 views • 44 slides
MONEY CHANGES EVERYTHING II: CREATING PRICE TRANSPARENCY IN NEW YORK STATE New York State Health

MONEY CHANGES EVERYTHING II: CREATING PRICE TRANSPARENCY IN NEW YORK STATE New York State Health

MONEY CHANGES EVERYTHING II: CREATING PRICE TRANSPARENCY IN NEW YORK STATE New York State Health Foundation November 12, 2013 Origins of FAIR Health Concerns led to an investigation by the New York State Attorney General Black box of

308 views • 14 slides
libcppa Now: High-Level Distributed Programming Without Sacrificing Performance Matthias

libcppa Now: High-Level Distributed Programming Without Sacrificing Performance Matthias

libcppa Now: High-Level Distributed Programming Without Sacrificing Performance Matthias Vallentin matthias@bro.org University of California, Berkeley C ++ Now May 14, 2013 Outline 1. Example Application: VAST 2. Designing Distributed

513 views • 36 slides
FTRMI: Fault-Tolerant Transparent RMI Diogo Reis and Hugo Miranda University of Lisbon LaSIGE

FTRMI: Fault-Tolerant Transparent RMI Diogo Reis and Hugo Miranda University of Lisbon LaSIGE

FTRMI: Fault-Tolerant Transparent RMI Diogo Reis and Hugo Miranda University of Lisbon LaSIGE SAC, March 28th, 2012 Remote Procedure Calls What? A simple approach for distributed computing Hides the network from the application (client and

348 views • 20 slides
Weaving Relations for Cache Performance Johannes Kern Universit at T ubingen 03.12.2010

Weaving Relations for Cache Performance Johannes Kern Universit at T ubingen 03.12.2010

Introduction Storage Models Analysis and Evaluation Summary Weaving Relations for Cache Performance Johannes Kern Universit at T ubingen 03.12.2010 Introduction Storage Models Analysis and Evaluation Summary Motivation I 2

289 views • 15 slides
An Introduction to Linked Open Data Felix.Ostrowski@googlemail.com (@literarymachine) Adrian

An Introduction to Linked Open Data Felix.Ostrowski@googlemail.com (@literarymachine) Adrian

An Introduction to Linked Open Data Felix.Ostrowski@googlemail.com (@literarymachine) Adrian Pohl@hbz-nrw.de (@acka47) SWIB 2013 Pre-Conference Workshop Monday, November 25th 2013 Hamburg Schedule Organize in teams Introduction:

1.31k views • 66 slides
Better image area annotations Robert Casties, Max Planck Institute for History of Science What

Better image area annotations Robert Casties, Max Planck Institute for History of Science What

Better image area annotations Robert Casties, Max Planck Institute for History of Science What we do Work with historical sources scanned books, manuscripts, notebooks digital photos scanned drawings and paintings digital

140 views • 10 slides

More recommend