kollaps
play

KOLLAPS Decentralized and Dynamic Topology Emulation Paulo - PowerPoint PPT Presentation

Oct 21, 2023 •812 likes •1.22k views

KOLLAPS Decentralized and Dynamic Topology Emulation Paulo Gouveia*, Joo Neves*, Carlos Segarra, Luca Liechti Shady Issa*, Valerio Schiavoni, and Miguel Matos* *: INESC-ID & IST, University of Lisbon, Portugal : University of

  1. KOLLAPS Decentralized and Dynamic Topology Emulation Paulo Gouveia*, João Neves*, Carlos Segarra†, Luca Liechti† Shady Issa*, Valerio Schiavoni†, and Miguel Matos* *: INESC-ID & IST, University of Lisbon, Portugal †: University of Neuchâtel, Switzerland 29.04.2020 Eurosys 2020 KOLLAPS

  2. 2 EuroSys 2020 KOLLAPS 29.04.2020

  3. MOTIVATION • Performance depends heavily on underlying network • Variability and Failures are the norm 3 EuroSys 2020 KOLLAPS 29.04.2020

  4. MOTIVATION • Performance depends heavily on underlying network • Variability and Failures are the norm • Need for tools for systematic evaluation of distributed applications • Ability to answer key questions: • What is the impact of halving the network latency in application throughput? • What is the effect of packet loss? • What if … 3 EuroSys 2020 KOLLAPS 29.04.2020

  5. RELATED WORK 4 EuroSys 2020 KOLLAPS 29.04.2020

  6. RELATED WORK Main limitations: - scalability/centralization - accuracy - dynamics 4 EuroSys 2020 KOLLAPS 29.04.2020

  7. KOLLAPS IN A NUTSHELL • Applications are concerned about end-to-end network properties • bandwidth, latency, jitter, packet loss • Rather than the network state leading to these properties 5 EuroSys 2020 KOLLAPS 29.04.2020

  8. KOLLAPS IN A NUTSHELL • Applications are concerned about end-to-end network properties • bandwidth, latency, jitter, packet loss • Rather than the network state leading to these properties • Emulate the emerging properties rather than the internal state of the network • Allows decentralized highly scalable emulation 5 EuroSys 2020 KOLLAPS 29.04.2020

  9. NETWORK COLLAPSING target topology 50Mb/s sv1 5ms 10Mb/s 100Mb/s c1 s1 s2 10ms 20ms 50Mb/s sv2 5ms 6 EuroSys 2020 KOLLAPS 29.04.2020

  10. NETWORK COLLAPSING collapsed topology target topology collapsed topology 50Mb/s 10Mb/s 10Mb/s 10Mb/s sv1 5ms c1 Node 35ms 35ms Router 10Mb/s 100Mb/s c1 s1 s2 10ms 20ms Throughput Latency 50Mb/s 50Mb/s sv2 sv2 sv1 10ms 5ms 6 EuroSys 2020 KOLLAPS 29.04.2020

  11. NETWORK COLLAPSING collapsed topology target topology collapsed topology 50Mb/s 10Mb/s 10Mb/s 10Mb/s sv1 5ms c1 Node 35ms 35ms Router 10Mb/s 100Mb/s c1 s1 s2 10ms 20ms Throughput Latency 50Mb/s 50Mb/s sv2 sv2 sv1 10ms 5ms 6 EuroSys 2020 KOLLAPS 29.04.2020

  12. NETWORK COLLAPSING collapsed topology target topology collapsed topology 50Mb/s 10Mb/s 10Mb/s 10Mb/s sv1 5ms c1 Node 35ms 35ms Router 10Mb/s 100Mb/s c1 s1 s2 10ms 20ms Throughput Latency 50Mb/s 50Mb/s sv2 sv2 sv1 10ms 5ms 7 EuroSys 2020 KOLLAPS 29.04.2020

  13. NETWORK COLLAPSING Minimum Minimum bandwidth on bandwidth on all links all links collapsed topology target topology collapsed topology 50Mb/s 10Mb/s 10Mb/s 10Mb/s sv1 5ms c1 Node 35ms 35ms Router 10Mb/s 100Mb/s c1 s1 s2 10ms 20ms Throughput Latency 50Mb/s 50Mb/s sv2 sv2 sv1 10ms 5ms 7 EuroSys 2020 KOLLAPS 29.04.2020

  14. NETWORK COLLAPSING Minimum Minimum bandwidth on bandwidth on all links all links collapsed topology target topology collapsed topology 50Mb/s 10Mb/s 10Mb/s 10Mb/s sv1 5ms c1 Node 35ms 35ms Router 10Mb/s 100Mb/s c1 s1 s2 10ms 20ms Throughput Latency 50Mb/s 50Mb/s sv2 sv2 sv1 10ms 5ms Sum of latencies Sum of latencies Sum of latencies on all links on all links on all links 7 EuroSys 2020 KOLLAPS 29.04.2020

  15. NETWORK COLLAPSING Minimum Minimum bandwidth on bandwidth on all links all links collapsed topology target topology collapsed topology 50Mb/s 10Mb/s 10Mb/s 10Mb/s sv1 5ms c1 Node 35ms 35ms Router 10Mb/s 100Mb/s c1 s1 s2 10ms 20ms Throughput Latency 50Mb/s 50Mb/s sv2 sv2 sv1 10ms 5ms Pre-computation of static properties Sum of latencies Sum of latencies Sum of latencies on all links on all links on all links 7 EuroSys 2020 KOLLAPS 29.04.2020

  16. ARCHITECTURE deployment monitor dashboard input design generator host-C host-B host-A tcal qdisc tcal qdisc tcal qdisc container container container tcal qdisc tcal qdisc tcal qdisc overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. em. core em. core ! em. core ! ! em. core em. core ! em. core ! Aeron ! ! ! ! ! ! ! Aeron ! ! ! Log Media Driver ! ! ! shared memory Log Media Driver emulation manager shared memory host-B host-C physical network 8 EuroSys 2020 KOLLAPS 29.04.2020

  17. ARCHITECTURE deployment monitor dashboard input design generator host-C host-B host-A tcal qdisc tcal qdisc tcal qdisc container container container tcal qdisc tcal qdisc tcal qdisc overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. em. core em. core ! em. core ! ! em. core em. core ! em. core ! Aeron ! ! ! ! ! ! ! Aeron ! ! ! Log Media Driver ! ! ! shared memory Log Media Driver emulation manager shared memory host-B host-C physical network 8 EuroSys 2020 KOLLAPS 29.04.2020

  18. ARCHITECTURE deployment monitor dashboard input design generator host-C host-B host-A tcal qdisc tcal qdisc tcal qdisc container container container tcal qdisc tcal qdisc tcal qdisc overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. em. core em. core ! em. core ! ! em. core em. core ! em. core ! Aeron ! ! ! ! ! ! ! Aeron ! ! ! Log Media Driver ! ! ! shared memory Log Media Driver emulation manager shared memory host-B host-C physical network 8 EuroSys 2020 KOLLAPS 29.04.2020

  19. ARCHITECTURE deployment monitor dashboard input design generator host-C host-B host-A tcal qdisc tcal qdisc tcal qdisc container container container tcal qdisc tcal qdisc tcal qdisc overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. em. core em. core ! em. core ! ! em. core em. core ! em. core ! Aeron ! ! ! ! ! ! ! Aeron ! ! ! Log Media Driver ! ! ! shared memory Log Media Driver emulation manager shared memory host-B host-C physical network 8 EuroSys 2020 KOLLAPS 29.04.2020

  20. ARCHITECTURE deployment monitor dashboard input design generator host-C host-B host-A tcal qdisc tcal qdisc tcal qdisc container container container tcal qdisc tcal qdisc tcal qdisc overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. em. core em. core ! em. core ! ! em. core em. core ! em. core ! Aeron ! ! ! ! ! ! ! Aeron ! ! ! Log Media Driver ! ! ! shared memory Log Media Driver emulation manager shared memory host-B host-C physical network 8 EuroSys 2020 KOLLAPS 29.04.2020

  21. ARCHITECTURE deployment monitor dashboard input design generator host-C host-B host-A tcal qdisc tcal qdisc tcal qdisc container container container tcal qdisc tcal qdisc tcal qdisc overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. em. core em. core ! em. core ! ! em. core em. core ! em. core ! Aeron ! ! ! ! ! ! ! Aeron ! ! ! Log Media Driver ! ! ! shared memory Log Media Driver emulation manager shared memory host-B host-C physical network 8 EuroSys 2020 KOLLAPS 29.04.2020

  22. ARCHITECTURE deployment monitor dashboard input design generator host-C host-B host-A tcal qdisc tcal qdisc tcal qdisc container container container tcal qdisc tcal qdisc tcal qdisc overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. em. core em. core ! em. core ! ! em. core em. core ! em. core ! Aeron ! ! ! ! ! ! ! Aeron ! ! ! Log Media Driver ! ! ! shared memory Log Media Driver emulation manager shared memory host-B host-C physical network 8 EuroSys 2020 KOLLAPS 29.04.2020

  23. ARCHITECTURE deployment monitor dashboard input design generator host-C host-B host-A tcal qdisc tcal qdisc tcal qdisc container container container tcal qdisc tcal qdisc tcal qdisc overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. overlay,.. em. core em. core ! em. core ! ! em. core em. core ! em. core ! Aeron ! ! ! ! ! ! ! Aeron ! ! ! Log Media Driver ! ! ! shared memory Log Media Driver emulation manager shared memory host-B host-C physical network 8 EuroSys 2020 KOLLAPS 29.04.2020

  24. EMULATION MANAGER (EM) •One instance per physical machine •Enforces topology properties •static properties •dynamic properties 9 EuroSys 2020 KOLLAPS 29.04.2020

  25. EM: DYNAMIC PROPERTIES target topology collapsed topology collapsed topology 50Mb/s 10Mb/s 10Mb/s 10Mb/s sv1 5ms c1 Node 35ms 35ms 10Mb/s 100Mb/s Router c1 s1 s2 10ms 20ms Throughput Latency 50Mb/s 50Mb/s sv2 sv1 sv2 5ms 10ms 10 EuroSys 2020 KOLLAPS 29.04.2020

  26. EM: DYNAMIC PROPERTIES target topology collapsed topology collapsed topology 50Mb/s 10Mb/s 10Mb/s 10Mb/s sv1 5ms c1 Node 35ms 35ms 10Mb/s 100Mb/s Router c1 s1 s2 10ms 20ms Throughput Latency 50Mb/s 50Mb/s sv2 sv1 sv2 5ms 10ms 10 EuroSys 2020 KOLLAPS 29.04.2020

  27. EM: DYNAMIC PROPERTIES target topology collapsed topology collapsed topology 50Mb/s 10Mb/s 10Mb/s 10Mb/s sv1 5ms c1 Node 35ms 35ms 10Mb/s 100Mb/s Router c1 s1 s2 10ms 20ms Throughput Latency 50Mb/s 50Mb/s sv2 sv1 sv2 5ms 10ms sv1 s / b M 0 1 C1 10 Mb/s sv2 10 EuroSys 2020 KOLLAPS 29.04.2020

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

FAI The Universal Deployment Tool Thomas Lange, University of Cologne

FAI The Universal Deployment Tool Thomas Lange, University of Cologne

FAI The Universal Deployment Tool Thomas Lange, University of Cologne lange@informatik.uni-koeln.de LCA Systems Administration Miniconf, January 2015 1 / 23 finger lange@localhost whoami Diploma in computer science, University of

391 views • 23 slides
Detecting and Surviving Intrusions Exploring New Host-Based Intrusion Detection, Recovery, and

Detecting and Surviving Intrusions Exploring New Host-Based Intrusion Detection, Recovery, and

Detecting and Surviving Intrusions Exploring New Host-Based Intrusion Detection, Recovery, and Response Approaches Ph.D. Thesis Defense December 17th, 2019 1 HP Labs (ronny.chevalier@hp.com) 2 CIDRE Team, CentraleSuplec/Inria/CNRS/IRISA

1.66k views • 140 slides
Prior Work Consensus Consensus Reliable BGP Consensus Reliable BGP Consensus Routing

Prior Work Consensus Consensus Reliable BGP Consensus Reliable BGP Consensus Routing

Prior Work Consensus Consensus Reliable BGP Consensus Reliable BGP Consensus Routing Reliable BGP Routing Reliable BGP Routing Routing Graceful Seamless Graceful Seamless Seamless Graceful Graceful Seamless Migration Migration

1.05k views • 57 slides
Network ID Subnet Host NNNN NNNN NNNN NNNN SSSS SSSS HHHH HHHH 1000 0000 0000 1010

Network ID Subnet Host NNNN NNNN NNNN NNNN SSSS SSSS HHHH HHHH 1000 0000 0000 1010

Working With Class B Address 128.10.0.0 Max # Of Subnets Min # Of Bits Subnet # Host # Max # Of Hosts 128.10.x.y 200 193 129 200 Subnets Network ID Subnet Host NNNN NNNN NNNN NNNN SSSS SSSS HHHH HHHH 1000 0000 0000 1010 128

128 views • 9 slides
Better Bug Reporting with Better Privacy M.Cast ro, M.Cost a, JP . Mart in Present ed by Horat

Better Bug Reporting with Better Privacy M.Cast ro, M.Cost a, JP . Mart in Present ed by Horat

Better Bug Reporting with Better Privacy M.Cast ro, M.Cost a, JP . Mart in Present ed by Horat iu Jula Imagine a crash Report the crash Bug reporting today Stack trace, memory dumps May be insufficient S olution: send path

822 views • 13 slides
Wireless Networks L ecture 21: Wireless and the Internet Peter Steenkiste CS and ECE, Carnegie

Wireless Networks L ecture 21: Wireless and the Internet Peter Steenkiste CS and ECE, Carnegie

Wireless Networks L ecture 21: Wireless and the Internet Peter Steenkiste CS and ECE, Carnegie Mellon University Peking University, Summer 2016 1 Peter A. Steenkiste Outline The Internet 102 Wireless and the Internet Mobility:

160 views • 15 slides
DUNE Host Lab Responsibilities Stu Fuess / Scientific Computing Division 2019 ICAC 15 October

DUNE Host Lab Responsibilities Stu Fuess / Scientific Computing Division 2019 ICAC 15 October

DUNE Host Lab Responsibilities Stu Fuess / Scientific Computing Division 2019 ICAC 15 October 2019 DISCLAIMER What is presented here is a summary of internal Fermilab discussions Starting point for discussions with the broader

455 views • 16 slides
Metrics and Best Practices for Host-based Access Control to Ensure System Integrity and

Metrics and Best Practices for Host-based Access Control to Ensure System Integrity and

Metrics and Best Practices for Host-based Access Control to Ensure System Integrity and Availability Urpo Kaila, Marco Passerini and Joni Virtanen CSC Tieteen tietotekniikan keskus Oy CSC IT Center for Science Ltd. Outline

374 views • 35 slides
Container Networking Gaetano Borgione Gaetano Borgione Sr. Staff Engineer @ VMware Sr. Staff

Container Networking Gaetano Borgione Gaetano Borgione Sr. Staff Engineer @ VMware Sr. Staff

Container Networking Gaetano Borgione Gaetano Borgione Sr. Staff Engineer @ VMware Sr. Staff Engineer Gaetano Borgione Senior Staff Engineer Cloud Native Applications VMWare SDN Technologies @ PLUMgrid Data Center Networking @ Cisco

680 views • 57 slides
Securing IPv6 neighbor discovery and SLAAC in access networks through SDN Daniel Nelle,

Securing IPv6 neighbor discovery and SLAAC in access networks through SDN Daniel Nelle,

Securing IPv6 neighbor discovery and SLAAC in access networks through SDN Daniel Nelle, dnelle@uni-potsdam.de Thomas Scheffler, thomas.scheffler@htw-berlin.de ANRW 2019, Montreal, July 22nd 2019 Outline Security issues in ICMPv6 Stateless

663 views • 21 slides
Network reconnaissance and IDS CS642: Computer Security

Network reconnaissance and IDS CS642: Computer Security

Network reconnaissance and IDS CS642: Computer Security Professor Ristenpart h9p://www.cs.wisc.edu/~rist/ rist at cs dot wisc dot edu University of

655 views • 43 slides
9/30/2013 Conflict of Interest Statement Behavioral Interventions for Hot 2 Flashes. The

9/30/2013 Conflict of Interest Statement Behavioral Interventions for Hot 2 Flashes. The

9/30/2013 Conflict of Interest Statement Behavioral Interventions for Hot 2 Flashes. The author of this presentation has no financial or commercial conflicts of interest pertinent to this 1 presentation within the last 12 months. GARY R.

409 views • 5 slides
FlexFS: A Flexible Flash File System for MLC NAND Flash Memory Sungjin Lee , Keonsoo Ha, Kangwon

FlexFS: A Flexible Flash File System for MLC NAND Flash Memory Sungjin Lee , Keonsoo Ha, Kangwon

FlexFS: A Flexible Flash File System for MLC NAND Flash Memory Sungjin Lee , Keonsoo Ha, Kangwon Zhang, Jihong Kim, and Junghwan Kim* School of Computer Science and Engineering Seoul National University * Samsung Advanced Institute of Technology

852 views • 53 slides
How to Scan 35mm Slides Make sure the scanner cable is plugged into the iMacs USB port. 1 Tie

How to Scan 35mm Slides Make sure the scanner cable is plugged into the iMacs USB port. 1 Tie

DIGITAL MEDIA LAB at the new lenox public library How to Scan 35mm Slides Make sure the scanner cable is plugged into the iMacs USB port. 1 Tie scanner cable is cream-colored. 2 Turn the scanner on by pressing the power button. Tie power

195 views • 8 slides
HCC: SYSTEMIC THERAPY FIRST LINE Akshjot Puri, MD Post Graduate Year 5 MENTOR: Madappa N.

HCC: SYSTEMIC THERAPY FIRST LINE Akshjot Puri, MD Post Graduate Year 5 MENTOR: Madappa N.

HCC: SYSTEMIC THERAPY FIRST LINE Akshjot Puri, MD Post Graduate Year 5 MENTOR: Madappa N. Kundranda, MD. PhD. CASE A 72 yo female with history of HCV cirrhosis (Child Pugh A) is found to have liver masses on screening ultrasound AFP 1072

301 views • 9 slides
Analytic models for flash-based SSD performance when subject to trimming Robin Verschoren and

Analytic models for flash-based SSD performance when subject to trimming Robin Verschoren and

Analytic models for flash-based SSD performance when subject to trimming Robin Verschoren and Benny Van Houdt Dept. Mathematics and Computer Science University of Antwerp Antwerp, Belgium MSST 2016 Robin Verschoren and Benny Van Houdt

374 views • 18 slides
AN ALTERNATIVE TO ALL- FLASH ARRAYS: PREDICTIVE STORAGE CACHING THE EASIEST WAY TO INCREASE

AN ALTERNATIVE TO ALL- FLASH ARRAYS: PREDICTIVE STORAGE CACHING THE EASIEST WAY TO INCREASE

AN ALTERNATIVE TO ALL- FLASH ARRAYS: PREDICTIVE STORAGE CACHING THE EASIEST WAY TO INCREASE PERFORMANCE AND LOWER STORAGE COSTS Bruce Kornfeld , Chief Marketing Officer, StorMagic Luke Pruen , Technical Services Director, StorMagic Peter Smith

524 views • 17 slides
Cascade Mapping: Optimizing Memory Efficiency for Flash-based Key-value Caching Kefei Wang and

Cascade Mapping: Optimizing Memory Efficiency for Flash-based Key-value Caching Kefei Wang and

Cascade Mapping: Optimizing Memory Efficiency for Flash-based Key-value Caching Kefei Wang and Feng Chen Louisiana State University SoCC '18 Carlsbad, CA Key-value Systems in Internet Services Key-value systems are widely used today

1.21k views • 84 slides
Chapter 17 17.1 Lives in the Balance Star Stuff Our goals for learning How does a

Chapter 17 17.1 Lives in the Balance Star Stuff Our goals for learning How does a

Chapter 17 17.1 Lives in the Balance Star Stuff Our goals for learning How does a stars mass affect nuclear fusion? How does a stars mass affect Stellar Mass and Fusion nuclear fusion? The mass of a main sequence star

368 views • 11 slides
Past Present, and Future Jim Handy Tutorial: Fast Storage for Big Data Hot Chips Conference

Past Present, and Future Jim Handy Tutorial: Fast Storage for Big Data Hot Chips Conference

Flashs Role in Big Data, Past Present, and Future Jim Handy Tutorial: Fast Storage for Big Data Hot Chips Conference August 25, 2013 Memorial Auditorium Stanford University O BJECTIVE A NALYSIS O BJECTIVE A NALYSIS Semiconductor Market

299 views • 26 slides
System Co-Design and Data Management for Flash Devices VLDB2011 Philippe Bonnet, Luc

System Co-Design and Data Management for Flash Devices VLDB2011 Philippe Bonnet, Luc

1 System Co-Design and Data Management for Flash Devices VLDB2011 Philippe Bonnet, Luc Bouganim, ITU, Denmark INRIA, France Ioannis Koltsidas Stratis D. Viglas IBM Research, Switzerland University of Edinburgh, United Kingdom Bonnet,

1.47k views • 127 slides
CS 135: File Systems Persistent Solid-State Storage 1 / 22 Introduction Technology Change is

CS 135: File Systems Persistent Solid-State Storage 1 / 22 Introduction Technology Change is

CS 135: File Systems Persistent Solid-State Storage 1 / 22 Introduction Technology Change is Coming Disks are cheaper than any solid-state memory Likely to be true for many years But SSDs are now cheap enough for some purposes 1000

464 views • 24 slides
Design of Flash- -Based DBMS: Based DBMS: Design of Flash Design of Flash-Based DBMS: An In-

Design of Flash- -Based DBMS: Based DBMS: Design of Flash Design of Flash-Based DBMS: An In-

SIGMOD 07 07 SIGMOD SIGMOD07 Design of Flash- -Based DBMS: Based DBMS: Design of Flash Design of Flash-Based DBMS: An In- -Page Logging Approach Page Logging Approach An In-Page Logging Approach An In '

399 views • 19 slides
NetCache: Balancing Key-Value Stores with Fast In-Network Caching Xin Jin, Xiaozhou Li , Haoyu

NetCache: Balancing Key-Value Stores with Fast In-Network Caching Xin Jin, Xiaozhou Li , Haoyu

NetCache: Balancing Key-Value Stores with Fast In-Network Caching Xin Jin, Xiaozhou Li , Haoyu Zhang, Robert Soul Jeongkeun Lee, Nate Foster, Changhoon Kim, Ion Stoica NetCache is a rack-scale key-value store that leverages in-network data

1.07k views • 31 slides

More recommend