scaleable page counter
play

Scaleable Page Counter A gentle introduction to software - PowerPoint PPT Presentation

Nov 17, 2023 •679 likes •1.18k views

Scaleable Page Counter A gentle introduction to software engineering at Google scale Jon Tirsen, Google Monday, October 4, 2010 Weve had visitors! Monday, October 4, 2010 Monday, October 4, 2010 Monday, October 4, 2010 Monday, October

  1. Scaleable Page Counter A gentle introduction to software engineering at Google scale Jon Tirsen, Google Monday, October 4, 2010

  2. We’ve had visitors! Monday, October 4, 2010

  3. Monday, October 4, 2010

  4. Monday, October 4, 2010

  5. Monday, October 4, 2010

  6. Monday, October 4, 2010

  7. Monday, October 4, 2010

  8. Monday, October 4, 2010

  9. Counter 112 +1 Front End Front End Front End Front End Front End Front End Front End Front End Front End Monday, October 4, 2010

  10. Counter Counter Counter Front End Front End Front End Front End Front End Front End Front End Front End Front End Monday, October 4, 2010

  11. Counter Counter Counter Write Front End Shard by: random Monday, October 4, 2010

  12. Monday, October 4, 2010

  13. Monday, October 4, 2010

  14. Counter Counter Counter Front End Read fan out Monday, October 4, 2010

  15. Front end Counter Counter Counter Blocking reads Monday, October 4, 2010

  16. Front end Counter Counter Counter Async reads Monday, October 4, 2010

  17. Monday, October 4, 2010

  18. Counter Counter Counter Front End Write Monday, October 4, 2010

  19. Counter Counter Counter Front End Read Monday, October 4, 2010

  20. Availability Monday, October 4, 2010

  21. Availability 14 mins/day 7 hours/month 99% = 3 days/year 1 Number of shards 1000 Monday, October 4, 2010

  22. 99.9999...% Availability Writing 99% 1 Number of shards 1000 Monday, October 4, 2010

  23. 99% Read fan out Availability 60% 0.6% 0.004% 1 Number of shards 1000 Monday, October 4, 2010

  24. Update Counter Counter Counter Replicas Replicas Replicas Replica Replica Replica Replica Replica Replica Write Reads Front End Monday, October 4, 2010

  25. 8 nines 5 nines 99.99% 6 nines Availability Read fan out to 5 replicas 1 Number of shards 1000 Monday, October 4, 2010

  26. Read & Cache Counter Counter Counter Write & Read Front End Monday, October 4, 2010

  27. 99.9999...% Availability Read fan out with caching 99% 1 Number of shards 1000 Monday, October 4, 2010

  28. Monday, October 4, 2010

  29. Consistency Monday, October 4, 2010

  30. Read Probability Write Write Write Write Monday, October 4, 2010

  31. Read Probability Write Write Write Write Monday, October 4, 2010

  32. Read Probability Write Write Write Write Monday, October 4, 2010

  33. Monday, October 4, 2010

  34. 200 299 52 Cached: 299 Front End Monday, October 4, 2010

  35. 200 299 52 Cached: 299, 52 Front End Monday, October 4, 2010

  36. 200 299 56 Cached: 299, 52 Front End Monday, October 4, 2010

  37. 200 302 56 Cached: 299, 52 Front End Monday, October 4, 2010

  38. 201 302 56 Cached: 299, 52 +1 1 0 2 552 =299+52+ Front End Monday, October 4, 2010

  39. Monday, October 4, 2010

  40. Monday, October 4, 2010

  41. Monday, October 4, 2010

  42. Monday, October 4, 2010

  43. A different solution... Monday, October 4, 2010

  44. Read & sum Analyser Analyser Analysers Logs Logs Logs Push new count Append Front End Front End Front End Monday, October 4, 2010

  45. Front End Counter Front End Log Analyser Monday, October 4, 2010

  46. Analyser Push new count Front End Front End Front End Monday, October 4, 2010

  47. Monday, October 4, 2010

  48. Sharding Shard selection: random, by user, by document Fan out: writes, reads Replication/Caching Consistency model: eventual, shard local Sync -> Async Questions? tirsen@google.com Twitter: @tirsen Monday, October 4, 2010

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

SCRAM: Scaleable Collision-avoiding Role Assignment with Minimal-makespan for Formational

SCRAM: Scaleable Collision-avoiding Role Assignment with Minimal-makespan for Formational

SCRAM: Scaleable Collision-avoiding Role Assignment with Minimal-makespan for Formational Positioning Patrick MacAlpine, Eric Price, and Peter Stone Department of Computer Science, The University of Texas at Austin January 30, 2015 Patrick

1.31k views • 94 slides
8051 Serial Port and Timer/Counter Serial Port Timer Counter Chatchai Jantaraprim

8051 Serial Port and Timer/Counter Serial Port Timer Counter Chatchai Jantaraprim

8051 Serial Port and Timer/Counter Serial Port Timer Counter Chatchai Jantaraprim (cj@coe.psu.ac.th) 8051 Timer/Counter Two internal Timers/Counters 16-bit timer/counter Timer uses system clock as source of input pulses Counter uses external

273 views • 8 slides
OVN: Scaleable Virtual Networking for Open vSwitch Kyle Mestery (@mestery) Justin Pettit

OVN: Scaleable Virtual Networking for Open vSwitch Kyle Mestery (@mestery) Justin Pettit

OVN: Scaleable Virtual Networking for Open vSwitch Kyle Mestery (@mestery) Justin Pettit (@Justin_D_Pettit) The Case for Network Virtualization Network provisioning needs to be self-service. Virtual networking needs to be abstracted

826 views • 37 slides
Counter Braids: A novel counter architecture Balaji Prabhakar Balaji Prabhakar Stanford

Counter Braids: A novel counter architecture Balaji Prabhakar Balaji Prabhakar Stanford

Counter Braids: A novel counter architecture Balaji Prabhakar Balaji Prabhakar Stanford University Joint work with: Yi Lu, Andrea Montanari , Sarang Dharmapurikar and Abdul Kabbani Overview Counter Braids Background: current

601 views • 30 slides
Counter/Timers Overview ATmega328P has two 8-bit and one 16-bit counter/timers. Unit C

Counter/Timers Overview ATmega328P has two 8-bit and one 16-bit counter/timers. Unit C

C.1 C.2 Counter/Timers Overview ATmega328P has two 8-bit and one 16-bit counter/timers. Unit C Timers and Counters Can count at some rate up to a value, generate an interrupt and start over counting from 0. Useful for performing

402 views • 5 slides
Can We Understand Performance Counter Results? Vince Weaver ICL Lunch Talk 23 July 2010 How Do

Can We Understand Performance Counter Results? Vince Weaver ICL Lunch Talk 23 July 2010 How Do

Can We Understand Performance Counter Results? Vince Weaver ICL Lunch Talk 23 July 2010 How Do We Know if Counters are Working? Three common failures: Wrong counter (PAPI, Kernel, User) Counter works but gives wrong values Counter

376 views • 36 slides
LETS PLAY! 100 99 98 97 96 O o p s , S o Salty just got a little u r P s u s

LETS PLAY! 100 99 98 97 96 O o p s , S o Salty just got a little u r P s u s

How to play: Each player puts their counter on the space that says 'START'. Take it in turns to roll the dice. Move your counter forward the number of spaces shown on the dice. If your counter lands at the bottom of a ladder, you can move up to

390 views • 5 slides
For Loops and Arrays November 13, 2008 Counting Initialize counter Test counter against limit

For Loops and Arrays November 13, 2008 Counting Initialize counter Test counter against limit

For Loops and Arrays November 13, 2008 Counting Initialize counter Test counter against limit int fireworkCount = 0; while (fireworkCount < NUM_IN_FINALE) { new Firework (...); Repeating Task fireworkCount++; } Increment counter

527 views • 4 slides
Decidable Problems for Counter Systems Day 1 Introduction to Counter Systems St ephane Demri

Decidable Problems for Counter Systems Day 1 Introduction to Counter Systems St ephane Demri

Decidable Problems for Counter Systems Day 1 Introduction to Counter Systems St ephane Demri demri@lsv.ens-cachan.fr LSV, ENS Cachan, CNRS, INRIA ESSLLI 2010, Copenhagen, August 2010 What is in the course? Analysis of Counter Systems

778 views • 65 slides
KAMPARO page 9 page 16 page 19 page 27 page 34 2 INHOUDSOPGA VE page 4 Cables Chargers

KAMPARO page 9 page 16 page 19 page 27 page 34 2 INHOUDSOPGA VE page 4 Cables Chargers

KAMPARO page 9 page 16 page 19 page 27 page 34 2 INHOUDSOPGA VE page 4 Cables Chargers Power Protection Music Fun & Handy 3 YOUR SAFETY YOUR TESTING Compliance Social Compliance BSCI Manufactured Test labs Offical CNAS

587 views • 40 slides
Defense Department Briefing on Priore Treatment to counter Biological Warfare SLIDE LIST

Defense Department Briefing on Priore Treatment to counter Biological Warfare SLIDE LIST

The Tom Bearden Website Defense Department Briefing on Priore Treatment to counter Biological Warfare SLIDE LIST Solutions to Mass Casualties Treatment Problem Threat and Current Status 1. Title Page 2. Key Points in the New Technology 3.

1.85k views • 95 slides
PAGE 1 PAGE 2 PAGE 3 PAGE 4 Vision PAGE 5 Desire Lines of Cow Paths? PAGE 6

PAGE 1 PAGE 2 PAGE 3 PAGE 4 Vision PAGE 5 Desire Lines of Cow Paths? PAGE 6

PAGE 1 PAGE 2 PAGE 3 PAGE 4 Vision PAGE 5 Desire Lines of Cow Paths? PAGE 6 www.olifantenpaadjes.nl PAGE 7 Desire Lines: Join Them or Fight Them (but never ignore them ) expected or normative path desire line PAGE 8 Google Maps

734 views • 45 slides
Decidability and complexity issues for subclasses of counter systems Lecture 4 Counter automata

Decidability and complexity issues for subclasses of counter systems Lecture 4 Counter automata

Decidability and complexity issues for subclasses of counter systems Lecture 4 Counter automata with finite monoid property and flatness St ephane Demri demri@lsv.ens-cachan.fr LSV, ENS Cachan, CNRS, INRIA Course 2.9 MPRI

921 views • 56 slides
8051 Serial Port and Timer/Counter Serial Port Timer Counter Chatchai Jantaraprim

8051 Serial Port and Timer/Counter Serial Port Timer Counter Chatchai Jantaraprim

8051 Serial Port and Timer/Counter Serial Port Timer Counter Chatchai Jantaraprim (cj@coe.psu.ac.th) 8051 Serial Port Serial communications High speed CMOS signal levels Full duplex No modem status/control signals Registers SBUF Two

661 views • 7 slides
Contents Summary presentation Q3/02 Page 3 Nordea Page 43 Integration Page 54

Contents Summary presentation Q3/02 Page 3 Nordea Page 43 Integration Page 54

Contents Summary presentation Q3/02 Page 3 Nordea Page 43 Integration Page 54 Business areas Page 62 Balance sheet Page 103 Credit quality Page 110 Economic capital Page 127 Appendix Page 147 2 3 Stable

2.09k views • 186 slides
Does COUNTER tell the whole story? Case-by-case examples demonstrating the limitations of

Does COUNTER tell the whole story? Case-by-case examples demonstrating the limitations of

Does COUNTER tell the whole story? Case-by-case examples demonstrating the limitations of COUNTER, and suggestions for alternative evaluation metrics CARLI Spring Forum on Collections Data Analysis and Maintenance Governors State University,

467 views • 23 slides
CS137: Simplifying Structure Electronic Design Automation K-LUT can implement any K-input

CS137: Simplifying Structure Electronic Design Automation K-LUT can implement any K-input

CS137: Simplifying Structure Electronic Design Automation K-LUT can implement any K-input function Day 3: September 29, 2005 Clustering (LUT Mapping, Delay) 1 4 CALTECH CS137 Fall2005 -- DeHon CALTECH CS137 Fall2005 -- DeHon Today

409 views • 18 slides
Algorithm development, performance, and demonstration Nhan Tran CD-1 Directors Review March

Algorithm development, performance, and demonstration Nhan Tran CD-1 Directors Review March

Algorithm development, performance, and demonstration Nhan Tran CD-1 Directors Review March 19-21, 2019 1 Brief Biological Sketch Wilson Fellow (Fermilab) L3 Manager: Correlator trigger Development of Particle Flow and PUPPI in L1 trigger

949 views • 57 slides
Spiral 2-7 Capacitance, Delay and Sizing 2-7.2 Learning Outcomes I understand the sources of

Spiral 2-7 Capacitance, Delay and Sizing 2-7.2 Learning Outcomes I understand the sources of

2-7.1 Spiral 2-7 Capacitance, Delay and Sizing 2-7.2 Learning Outcomes I understand the sources of capacitance in CMOS circuits I understand how delay scales with resistance, capacitance and voltage I can determine appropriate width

756 views • 40 slides
SFE: Yaos Garbled Circuit Oblivious Transfer IDEAL World Pick one out of two, without

SFE: Yaos Garbled Circuit Oblivious Transfer IDEAL World Pick one out of two, without

SFE: Yaos Garbled Circuit Oblivious Transfer IDEAL World Pick one out of two, without revealing which Intuitive property: OT transfer partial A:up, B:down A up I need just information All 2 of one them! But cant

432 views • 20 slides
Modern OLTP Indexes (Part 2) 1 / 43 Modern OLTP Indexes (Part 2) Recap Recap 2 / 43 Modern OLTP

Modern OLTP Indexes (Part 2) 1 / 43 Modern OLTP Indexes (Part 2) Recap Recap 2 / 43 Modern OLTP

Modern OLTP Indexes (Part 2) Modern OLTP Indexes (Part 2) 1 / 43 Modern OLTP Indexes (Part 2) Recap Recap 2 / 43 Modern OLTP Indexes (Part 2) Recap Versioned Latch Coupling Optimistic coupling scheme where writers are not blocked on

610 views • 43 slides
Challenges for Polarimetry at the ILC Spin Tracking Studies Moritz Beckmann, Jenny List DESY -

Challenges for Polarimetry at the ILC Spin Tracking Studies Moritz Beckmann, Jenny List DESY -

Challenges for Polarimetry at the ILC Spin Tracking Studies Moritz Beckmann, Jenny List DESY - FLC LCWS 2012, Arlington, TX, USA October 25, 2012 Introduction: Polarimetry at the ILC Two laser Compton polarimeters per beam in the beam

840 views • 41 slides
A First Look at Modern Enterprise Traffic Ruoming Pang , Princeton University Mark Allman ( ICSI

A First Look at Modern Enterprise Traffic Ruoming Pang , Princeton University Mark Allman ( ICSI

A First Look at Modern Enterprise Traffic Ruoming Pang , Princeton University Mark Allman ( ICSI ), Mike Bennett ( LBNL ), Jason Lee ( LBNL ), Vern Paxson ( ICSI/LBNL ), and Brian Tierney ( LBNL ) The Question What does the traffic look like

533 views • 41 slides
Arithmetic Coding Mathias Winther Madsen mathias.winther@gmail.com Institute for Logic,

Arithmetic Coding Mathias Winther Madsen mathias.winther@gmail.com Institute for Logic,

Arithmetic Coding Mathias Winther Madsen mathias.winther@gmail.com Institute for Logic, Language, and Computation University of Amsterdam March 2015 Binary Intervals Definition A binary interval is an interval of the form k 2 m , k + 1

128 views • 11 slides

More recommend