architectures that scale deep
play

Architectures that Scale Deep: Regaining Control in Deep Systems - PowerPoint PPT Presentation

Mar 13, 2023 •274 likes •951 views

Architectures that Scale Deep: Regaining Control in Deep Systems Ben Sigelman (@el_bhs, bhs@lightstep.com) Co-founder & CEO: LightStep Co-creator: OpenTracing, OpenTelemetry, Google Dapper, Google Monarch QCon SF, November 2019 Part I

  1. Architectures that Scale Deep: Regaining Control in Deep Systems Ben Sigelman (@el_bhs, bhs@lightstep.com) Co-founder & CEO: LightStep Co-creator: OpenTracing, OpenTelemetry, Google Dapper, Google Monarch QCon SF, November 2019

  2. Part I Scaling, and Deep Systems

  3. What is scale, anyway?

  4. Scaling wide

  5. Scaling wide

  6. Scaling wide

  7. Scaling wide

  8. Scaling wide

  9. Scaling deep

  10. Scaling deep

  11. Scaling deep

  12. Scaling deep

  13. Scaling deep

  14. How does this look for software?

  15. Software: Scaling wide

  16. Software: Scaling deep

  17. How do real-world systems look?

  18. Microservices at scale aren’t just wide systems , they’re deep systems

  19. Deep Systems Deep Systems Architectures with ≥ 4 layers of Architectures with ≥ 4 layers of independently operated services independently operated services (including external/cloud dependencies) (including external/cloud dependencies)

  20. What do deep systems sound like?

  21. What do deep systems sound like? “Don’t deploy on Fridays”

  22. What do deep systems sound like? “Where’s Chris?! I’m dealing with a P0 and they’re the only one who knows how to debug this.”

  23. What do deep systems sound like? “It can’t be our fault, our dashboard says we’re healthy”

  24. What do deep systems sound like? “Kafka is on fire”

  25. What do deep systems sound like? “I need 100% availability from your team. One hundred percent .”

  26. What do deep systems sound like? “I didn’t know I depended on that region”

  27. What do deep systems sound like? “That was on a dashboard but I can’t find it”

  28. What do deep systems sound like? Lots of challenges: - People-management - Security - Multi-tenancy - “Big-customer” success - Performance - Observability

  29. Part II Control Theory: TL;DR Edition

  30. Why do we care so much about observability , anyway?

  32. Inputs Outputs A System … and its state vector,

  33. Observability Inputs Outputs A System … and its state vector, How well can you infer internal state using only the outputs ?

  34. Controllability Inputs Outputs A System … and its state vector, How well can you control internal state using only the inputs ?

  35. Controllability is the dual of Observability

  36. Controllability is the dual of Observability

  37. Part III What Deep Systems Mean for Observability

  38. Pure Monoliths developers per service Deep Systems Architectural evolution # of services

  39. Stress (n): responsibility without control Stress what you can control what you are responsible for

  41. Observability: Shrink This Gap

  42. Mental models A System

  43. Managing Deep Systems Services must have SLOs (“Service Level Objectives”: latency, errors, etc) For effective service management, only three things matter: 0. Releasing service functionality 1. Gradually improving SLOs 2. Rapidly restoring SLOs In a deep system, we must control the entire “triangle” to maintain our SLOs

  44. There’s that word again… Controllability == Observability Controllability == Observability

  45. Observability: “The Conventional Wisdom” Observing microservices is hard Google and Facebook solved this (right???) They used Metrics, Logging, and Distributed Tracing… … So we should, too.

  46. 3 Pillars, 3 Experiences Metrics Logs Traces

  48. Three Pillars? Three Pillars? Two giant pipes… Metrics Without Traces: Cognitive Load ≈ O( depth 2 ) Logs

  49. Three Pillars? Three Pillars? Two giant pipes… Metrics Logs

  51. Two giant pipes… Metrics Without Traces: Cognitive Load ≈ O( depth 2 ) Logs

  53. Traces

  54. Traces provide Context

  55. Traces provide Context And context rules out invalid hypotheses

  56. Two giant pipes and a filter Metrics Context (from traces) Logs

  57. Context reduces cognitive load Relevant Metrics Context (from traces) Relevant Logs With Traces: Cognitive Load ≈ O( depth )

  58. Observability: Shrink This Gap

  60. Let’s Review

  61. Microservices don’t just scale wide, they scale deep Recognize deep systems

  62. Stress (n): responsibility without control Stress what you can control what you are responsible for

  63. “Controllability” (of SLOs) depends on observability

  64. “The Three Pillars of Observability” is a lousy metaphor … and traces are not sprinkles

  65. Tracing can reduce cognitive load from O( depth 2 ) to O( depth )

  66. Tracing is the backbone of simple observability in deep systems

  67. Thank You Play with LightStep, Feedback always for free, anytime: welcome: (no email address required!) twitter → @el_bhs lightstep.com/play the emails → bhs@lightstep.com

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

8. Other Deep Architectures CS 519 Deep Learning, Winter 2018 Fuxin Li With materials from Zsolt

8. Other Deep Architectures CS 519 Deep Learning, Winter 2018 Fuxin Li With materials from Zsolt

8. Other Deep Architectures CS 519 Deep Learning, Winter 2018 Fuxin Li With materials from Zsolt Kira and Ian Goodfellow A brief overview of other architectures Unsupervised Architectures Deep Belief Networks Autoencoders GANs

439 views • 25 slides
Multi-Scale and Multi-Physics Simulations on Present and Future Architectures

Multi-Scale and Multi-Physics Simulations on Present and Future Architectures

Multi-Scale and Multi-Physics Simulations on Present and Future Architectures www.uintah.utah.edu Martin Berzins 1. Background and motivation 2. Uintah Software and Multicore Scalability 3. Runtime Systems for Heterogeneous Architectures 4.

759 views • 27 slides
Learning Deep Architectures Using Kernel Modules Li Deng Microsoft Research, Redmond (thanks

Learning Deep Architectures Using Kernel Modules Li Deng Microsoft Research, Redmond (thanks

MLSLP 2012 Learning Deep Architectures Using Kernel Modules Li Deng Microsoft Research, Redmond (thanks collaborations/discussions with many people) Introduction Deep neural net (modern multilayer perceptron) Hard to

330 views • 15 slides
MongoDB large scale data-centric architectures QConSF 2012 Kenny Gorman Founder, ObjectRocket

MongoDB large scale data-centric architectures QConSF 2012 Kenny Gorman Founder, ObjectRocket

MongoDB large scale data-centric architectures QConSF 2012 Kenny Gorman Founder, ObjectRocket @objectrocket @kennygorman MongoDB at scale Designing for scale Techniques to ease pain Things to avoid What is scale? Scale;

539 views • 27 slides
Stochastic Deep Networks Gwendoline De Bie, Gabriel Peyr, Marco Cuturi Deep Architectures on

Stochastic Deep Networks Gwendoline De Bie, Gabriel Peyr, Marco Cuturi Deep Architectures on

Stochastic Deep Networks Gwendoline De Bie, Gabriel Peyr, Marco Cuturi Deep Architectures on Density Inputs t=1, 1000 cells t=2, 650 cells t=3, 890 cells of varying physical attributes Representing inputs as densities (discretized in practice)

303 views • 7 slides
Key Ideas and Architectures in Deep Learning Applications that (probably) use DL Autonomous

Key Ideas and Architectures in Deep Learning Applications that (probably) use DL Autonomous

Key Ideas and Architectures in Deep Learning Applications that (probably) use DL Autonomous Driving Scene understanding /Segmentation Applications that (probably) use DL WordLens Prisma Outline of todays talk Image Recognition Fun

559 views • 43 slides
DSSTNE: Deep Learning At Scale For Large Sparse Datasets

DSSTNE: Deep Learning At Scale For Large Sparse Datasets

DSSTNE: Deep Learning At Scale For Large Sparse Datasets https://github.com/amznlabs/amazon-dsstne Scott Le Grand Senior Scientist Teza Technologies varelse2005@gmail.com Outline What's Deep Learning? Why GPUs? Deep Learning for

1.55k views • 76 slides
Heterogeneous Compute Architectures For Deep Learning In The Cloud Ken OBrien, Nicholas Fraser

Heterogeneous Compute Architectures For Deep Learning In The Cloud Ken OBrien, Nicholas Fraser

Heterogeneous Compute Architectures For Deep Learning In The Cloud Ken OBrien, Nicholas Fraser , Michaela Blott 27 th March 2019 Outline Why FPGAs? Deep Learning: Challenges & Solutions FINN FPGAs to ACAPs Mega-Trend:

475 views • 31 slides
Structure preservation in (some) deep learning architectures Brynjulf Owren Department of

Structure preservation in (some) deep learning architectures Brynjulf Owren Department of

Structure preservation in (some) deep learning architectures Brynjulf Owren Department of Mathematical Sciences, NTNU, Trondheim, Norway LMS-Bath Symposium 2020 Joint work with: Martin Benning, Elena Celledoni, Matthias Ehrhardt, Christian

805 views • 31 slides
Learning Deep Architectures Yoshua Bengio, U. Montreal CIFAR NCAP Summer School 2009 August 6th,

Learning Deep Architectures Yoshua Bengio, U. Montreal CIFAR NCAP Summer School 2009 August 6th,

Learning Deep Architectures Yoshua Bengio, U. Montreal CIFAR NCAP Summer School 2009 August 6th, 2009, Montreal Main reference: Learning Deep Architectures for AI, Y. Bengio, to appear in Foundations and Trends in Machine Learning ,

1.09k views • 87 slides
Distributed DeepLearning at Scale Soumith Chintala Facebook AI Research Overview Deep

Distributed DeepLearning at Scale Soumith Chintala Facebook AI Research Overview Deep

Distributed DeepLearning at Scale Soumith Chintala Facebook AI Research Overview Deep Learning Research at FAIR Deep Learning on GPUs Deep Learning at scale Emerging Trends Deep Learning Research at Facebook AI Research Image

1.18k views • 50 slides
Deep learning architectures for inference of AC-OPF solutions Tackling Climate Change with

Deep learning architectures for inference of AC-OPF solutions Tackling Climate Change with

Deep learning architectures for inference of AC-OPF solutions Tackling Climate Change with Machine Learning NeurIPS 2020 Thomas Falconer, UCL Energy Institute (Energy & AI Lab) Letif Mones, Invenia Labs Optimal Power Flow (OPF) Challenges

557 views • 14 slides
Toward Extreme-Scale Manycore Architectures Josep Torrellas Department of Computer Science

Toward Extreme-Scale Manycore Architectures Josep Torrellas Department of Computer Science

Toward Extreme-Scale Manycore Architectures Josep Torrellas Department of Computer Science University of Illinois at Urbana-Champaign http://iacoma.cs.uiuc.edu USC October 2016 Accelerated Progress in Transistor Integration Large

867 views • 76 slides
CNN Architectures ILSVRC: Imagenet Large Scale Visual Recognition Challenge [Russakovsky et al

CNN Architectures ILSVRC: Imagenet Large Scale Visual Recognition Challenge [Russakovsky et al

CS4501: Introduction to Computer Vision CNN Architectures ILSVRC: Imagenet Large Scale Visual Recognition Challenge [Russakovsky et al 2014] The Problem: Classification Classify an image into 1000 possible classes: e.g. Abyssinian cat,

673 views • 36 slides
Deep Epitomic Nets and Scale/Position Search for Image Classification TTIC_ECP team George

Deep Epitomic Nets and Scale/Position Search for Image Classification TTIC_ECP team George

1 TTIC_ECP: Deep Epitomic CNNs and Explicit Scale/Position Search Deep Epitomic Nets and Scale/Position Search for Image Classification TTIC_ECP team George Papandreou Iasonas Kokkinos Toyota Technological Institute Ecole Centrale Paris/INRIA

399 views • 23 slides
Advanced architectures Benoit Favre < benoit.favre@univ-mrs.fr > Aix-Marseille Universit,

Advanced architectures Benoit Favre < benoit.favre@univ-mrs.fr > Aix-Marseille Universit,

Deep learning for natural language processing Advanced architectures Benoit Favre < benoit.favre@univ-mrs.fr > Aix-Marseille Universit, LIF/CNRS 23 Feb 2017 Benoit Favre (AMU) DL4NLP: advanced architectures 23 Feb 2017 1 / 29 Deep

526 views • 29 slides
Deep Yellow Limited Indaba Presentation February 2013 Greg Cochran Managing Director ASX:

Deep Yellow Limited Indaba Presentation February 2013 Greg Cochran Managing Director ASX:

Deep Yellow Limited Indaba Presentation February 2013 Greg Cochran Managing Director ASX: DYL www.deepyellow.com.au Disclaimer This document has been prepared by Deep Yellow Limited (Deep Yellow, DYL or the Company) in connection with

499 views • 21 slides
Modeling Interestingness with Deep Neural Networks Jianfeng Gao, Patrick Pantel, Michael Gamon,

Modeling Interestingness with Deep Neural Networks Jianfeng Gao, Patrick Pantel, Michael Gamon,

Modeling Interestingness with Deep Neural Networks Jianfeng Gao, Patrick Pantel, Michael Gamon, Xiaodong He, Li Deng, Yelong Shen Presented by Scott Wen-tau Yih Microsoft Research (Redmond, USA) Computing Semantic Similarity Fundamental to

818 views • 36 slides
Bird Identification using Deep Learning Techniques Presentation by Elias Sprengel University:

Bird Identification using Deep Learning Techniques Presentation by Elias Sprengel University:

Bird Identification using Deep Learning Techniques Presentation by Elias Sprengel University: ETH Zrich Group : Data Analytics Lab http://da.inf.ethz.ch Elias Sprengel 06.09.2016 1 Outline 1 Quick overview of our approach 2 BirdCLEF

309 views • 18 slides
A Pure-Play Zinc Producer June 2018 w w w . a s c e n d a n t r e s o u r c e s . c o m T S X :

A Pure-Play Zinc Producer June 2018 w w w . a s c e n d a n t r e s o u r c e s . c o m T S X :

1 PROFITABILITY GROWTH OPPORTUNITY A Pure-Play Zinc Producer June 2018 w w w . a s c e n d a n t r e s o u r c e s . c o m T S X : A S N D OTCQX: ASDRF 2 Forward Looking Statements This presentation contains "forward-looking

520 views • 36 slides
Deep Learning Feature for Handwritten Keyword Spotting Baptiste Wicht Andreas Fischer Jean

Deep Learning Feature for Handwritten Keyword Spotting Baptiste Wicht Andreas Fischer Jean

Deep Learning Feature for Handwritten Keyword Spotting Baptiste Wicht Andreas Fischer Jean Hennebert iCoSys, University of Applied Sciences of Western Switzerland HES-SO DIVA Group, University of Fribourg, Switzerland B. Wicht, A. Fischer, J.

332 views • 20 slides
Everybody Else Has Signed It. Whats Your Problem? Why Deep South Engineers Need to

Everybody Else Has Signed It. Whats Your Problem? Why Deep South Engineers Need to

Everybody Else Has Signed It. Whats Your Problem? Why Deep South Engineers Need to Present a United Front on the Indemnity Issue Roger Guilian Volkert, Inc. Vice President General Counsel Presenter TRUE or FALSE? One short

332 views • 22 slides
Tac acom oma P a Power an and Tran ansp sport rtation on E Electrification on Cam

Tac acom oma P a Power an and Tran ansp sport rtation on E Electrification on Cam

Tac acom oma P a Power an and Tran ansp sport rtation on E Electrification on Cam LeHouillier, Manager of Energy R&D Tacom coma P Power A A win in-wi win-win in s situ tuatio ion Electricity is less expensive DRIVER EVs

848 views • 19 slides
Linearly Augmented Deep Neural Network Pegah Ghahremani, Johns Hopkins University, Jasha Droppo,

Linearly Augmented Deep Neural Network Pegah Ghahremani, Johns Hopkins University, Jasha Droppo,

Linearly Augmented Deep Neural Network Pegah Ghahremani, Johns Hopkins University, Jasha Droppo, Michael L. Seltzer, Microsoft Research Typical DNN Architecture Softmax L4 Layers are a composition of an affine and a non- linear function.

199 views • 19 slides

More recommend