Help! I accidentally distributed my system! Rachel Myers, Emily - PowerPoint PPT Presentation

Help! I accidentally distributed my system! Rachel Myers, Emily Nakashima

@eanakashima @rachelmyers, @eanakashima

@rachelmyers, @eanakashima

@rachelmyers @rachelmyers, @eanakashima

@rachelmyers, @eanakashima

@rachelmyers, @eanakashima

ModCloth Web App 
 User request (from NGINX & (Rails) loadbalancer) MySQL @rachelmyers, @eanakashima

What even is 
 a distributed system? @rachelmyers, @eanakashima

What even is a distributed system? Some 
 Node Another Node @rachelmyers, @eanakashima


 Designing distributed systems 👓 • What are the “nouns” in my system? • What are the patterns for reading data? • What are the patterns for writing data? • Do di ff erent flows have di ff erent performance, consistency, & availability requirements? 
 @rachelmyers, @eanakashima

ModCloth Web App User request (from NGINX & loadbalancer) Mobile MySQL @rachelmyers, @eanakashima

ModCloth Web App User request (from NGINX & loadbalancer) Mobile MySQL Voting Service MySQL @rachelmyers, @eanakashima

ModCloth Web App User request (from NGINX & loadbalancer) Mobile Identity MySQL Service MySQL Voting Service MySQL @rachelmyers, @eanakashima

ModCloth Web App User request (from NGINX & loadbalancer) Mobile Identity MySQL Service MySQL Asset Voting Service Service MySQL MySQL @rachelmyers, @eanakashima

ModCloth Web App Photo User request (from NGINX & Service loadbalancer) Mobile MySQL Identity MySQL Service MySQL Comment Asset Voting Service Service Service MySQL MySQL MySQL @rachelmyers, @eanakashima

ModCloth Web App Photo User request (from NGINX & Service loadbalancer) Mobile MySQL Moderation Identity MySQL Service Service MySQL MySQL Comment Asset Voting Service Service Service MySQL MySQL MySQL @rachelmyers, @eanakashima

ModCloth Web App Photo Checkout User request (from NGINX & Service Service loadbalancer) Mobile MySQL MySQL Moderation Identity MySQL Service Service MySQL MySQL Comment Asset Voting Service Service Service MySQL MySQL MySQL @rachelmyers, @eanakashima

ModCloth Web App Photo Checkout User request (from NGINX & Service Service loadbalancer) Mobile MySQL MySQL MySQL 🔦 Moderation Identity 🗒 MySQL Service Service MySQL MySQL Comment Asset Voting Service Service Service MySQL MySQL @rachelmyers, @eanakashima

ModCloth Web App Photo Checkout User request (from NGINX & Service Service loadbalancer) Mobile MySQL MySQL 😈😈😈 🤔🤔🤔 Moderation Identity MySQL Service Service MySQL MySQL Comment Asset Voting Service Service Service MySQL MySQL MySQL @rachelmyers, @eanakashima

And this was intentional 😃 @rachelmyers, @eanakashima

flexfits.com An accidental distributed system ⛈ @rachelmyers, @eanakashima

flexfits.com @rachelmyers, @eanakashima

flexfits.com Ecommerce SaaS User request @rachelmyers, @eanakashima

flexfits.com Subscription Plugin SaaS Ecommerce SaaS User request @rachelmyers, @eanakashima

flexfits.com Subscription Plugin SaaS Fulfillment Ecommerce SaaS SaaS User request @rachelmyers, @eanakashima

flexfits.com Metrics Subscription SaaS Plugin SaaS Fulfillment Ecommerce SaaS SaaS User request @rachelmyers, @eanakashima

flexfits.com @rachelmyers, @eanakashima

flexfits.com 👒 👒 @rachelmyers, @eanakashima

flexfits.com Metrics Subscription SaaS Plugin SaaS Fulfillment Ecommerce SaaS SaaS User request ? @rachelmyers, @eanakashima

flexfits.com Metrics Subscription SaaS Plugin SaaS Fulfillment Ecommerce SaaS SaaS Orders User request Service Postgres @rachelmyers, @eanakashima

flexfits.com 👓 Metrics Subscription SaaS Plugin SaaS Fulfillment Ecommerce SaaS SaaS Orders User request Service Postgres @rachelmyers, @eanakashima

New requirements for buy vs build @rachelmyers, @eanakashima

flexfits.com SaaS SaaS SaaS SaaS Home cooked User request on IaaS @rachelmyers, @eanakashima

Buy or Build • Will it improve availability? • Will it improve recovery? • Will you have visibility or points of control? • Will you have customizability? • Will you know when your changes take e ff ect? • Can you audit changes? @rachelmyers, @eanakashima

IaaS, PaaS, BaaS, & FaaS @rachelmyers, @eanakashima

Cloud Infrastructure @rachelmyers, @eanakashima

IaaS @rachelmyers, @eanakashima

IaaS • Will it improve availability? ✅ • Will it improve recovery? ✅ • Will you have visibility or points of control? ✅ • Will you have customizability? ✅ • Will you know when your changes take e ff ect? ✅ • Can you audit changes? ✅ @rachelmyers, @eanakashima

Platforms @rachelmyers, @eanakashima

PaaS @rachelmyers, @eanakashima

PaaS • Will it improve availability? ✅ • Will it improve recovery? 🤕 • Will you have visibility or points of control? ✅ • Will you have customizability? ✅ • Will you know when your changes take e ff ect? 🤕 • Can you audit changes? 🤕 @rachelmyers, @eanakashima

Backends @rachelmyers, @eanakashima

BaaS (RIP Parse) @rachelmyers, @eanakashima

BaaS • Will it improve availability? ✅🤕 • Will it improve recovery? 🤕 • Will you have visibility or points of control? 🤕 • Will you have customizability? ✅ • Will you know when your changes take e ff ect? 🤕 • Can you audit changes? 🤕 @rachelmyers, @eanakashima

Functions @rachelmyers, @eanakashima

FaaS @rachelmyers, @eanakashima

FaaS • Will it improve availability? 🤕 • Will it improve recovery? 🤕 • Will you have visibility or points of control? 🤕 • Will you have customizability? ✅ • Will you know when your changes take e ff ect? 🤕 • Can you audit changes? 🤕 @rachelmyers, @eanakashima

How to choose?

Backend Harder to change Faster to start Platform Infrastructure @rachelmyers, @eanakashima

@rachelmyers, @eanakashima

@rachelmyers, @eanakashima

Surprise Distributed Systems @rachelmyers, @eanakashima

Surprise distributed system: browsers @rachelmyers, @eanakashima

“The browser is part of your distributed system, not just a client you support” 
 -@danielespeset @rachelmyers, @eanakashima

The one where we DDOS’d ourselves @rachelmyers, @eanakashima

What happened? @rachelmyers, @eanakashima

// if the page is long enough to scroll if (document.body.clientHeight > window.innerHeight) { // add a scroll event listener document.addEventListener('scroll', function(e) { // if within 100px of the bottom of the page if (window.innerHeight + window.scrollY + 100 > document.body.clientHeight) fetchNextPage(); }); // else fetch another page of results immediately } else { fetchNextPage(); } @rachelmyers, @eanakashima

Frontend complexity is only increasing @rachelmyers, @eanakashima

Instrumentation Rule 1: Start at the edge. 
 Corollary: The edge may be further than you think. @rachelmyers, @eanakashima

Secret distributed system: vendors @rachelmyers, @eanakashima

LaunchDarkly @rachelmyers, @eanakashima

Secret distributed system: vendors @rachelmyers, @eanakashima

# instantiate the flag with a default value # falls back to default value if service is offline show_feature = ld_client.variation( "allow_traces", { :key => "user@example.com" }, false ) if show_feature # application code to show the feature else # the code to run if the feature is off @rachelmyers, @eanakashima

Secret distributed system: vendors @rachelmyers, @eanakashima

Instrumentation Rule 2: Wrap 3rd party calls @rachelmyers, @eanakashima

Secret distributed system: vendors @rachelmyers, @eanakashima

GitHub’s customer support application @rachelmyers, @eanakashima

GitHub customer support tool Customer github.com Support Tool User email @rachelmyers, @eanakashima

GitHub customer support tool Customer github.com Support Tool GMail User email @rachelmyers, @eanakashima

GitHub customer support tool Customer github.com Support Tool SendGrid GMail MailChimp User email @rachelmyers, @eanakashima

GitHub customer support tool Customer github.com Support Tool SendGrid GMail MailChimp User email @rachelmyers, @eanakashima

GitHub customer support tool Customer github.com Support Tool Outbound Ingest service Kafka mail service User email @rachelmyers, @eanakashima

GitHub customer support tool mail service 🔦 Customer github.com Support Tool 🗒 Outbound Ingest service Kafka User email @rachelmyers, @eanakashima

GitHub customer support tool Customer github.com Support Tool SendGrid GMail MailChimp User email @rachelmyers, @eanakashima

Cognitive Load @rachelmyers, @eanakashima