caches memcache example
play

Caches & Memcache Example Client N. America Client System - PowerPoint PPT Presentation

Dec 26, 2023 •224 likes •839 views

Caches & Memcache Example Client N. America Client System Asia + Caches Client Africa Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1));

  1. Caches & Memcache

  2. Example Client N. America Client System Asia + Caches Client Africa

  3. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) Assume that clients use a sharded key-value store to coordinate their output

  4. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) Write buffering: Can we start to write done1 before we finish write to k1 ?

  5. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) Write buffering: Can we start to write done1 before we finish write to k1 ? No, if sharded and want linearizability: must serialize writes

  6. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) What if caches can hold out of date data? What might go wrong?

  7. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) Asia: done1 = true, cached (old) k1 Africa: done2 = true, cached (old) k1 and k2 Africa: done2 = true, k2 correct, cached k1 (!)

  8. Rules for caches and shards Correct execution if: 1. Operations applied in processor order, and 2. All operations to a single key are serialized (as if to a single copy ) How do we ensure #2? - Can serialize each memory location in isolation

  9. Invalidations vs. Leases Invalidations - Track where data is cached - When doing a write, invalidate all (other) locations - Data can live in multiple caches during reads Leases - Permission to serve data for some time period - Wait until lease expires before update

  10. Write-through vs. write-back Write-through - Writes go to the server - Caches only hold clean data Write-back - Writes go to cache - Dirty cache data written to server when necessary

  11. Write-through vs. write-back Mechanism Invalidations Leases Write policy AFS Write-through DNS (Andrew FS) Write-back Sprite NFS

  12. Write-through invalidations Track all caches with read copies On a write: - Send invalidations to all caches with a copy - Each cache invalidates, responds - Wait for all invalidations, do update - Return Reads can proceed: - If there is a cached copy - or if cache miss, read at server

  13. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) k1 = 0 Server k2 = 0 done1 = false done2 = false

  14. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) read miss: done1 k1 = 0 Server k2 = 0 done1 = false done2 = false

  15. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) done1: false k1 = 0 Server k2 = 0 done1 = false done1: Asia done2 = false

  16. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) done1 = false k1 = 0 Server k2 = 0 done1 = false done1: Asia done2 = false

  17. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) done1 = false read miss: done2 k1 = 0 Server k2 = 0 done1 = false done1: Asia done2 = false

  18. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) done1 = false done2: false k1 = 0 Server k2 = 0 done1 = false done1: Asia done2 = false done2: Africa

  19. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) done1 = false done2 = false k1 = 0 Server k2 = 0 done1 = false done1: Asia done2 = false done2: Africa

  20. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) k1: 42 done1 = false done2 = false k1 = 0 Server k2 = 0 done1 = false done1: Asia done2 = false done2: Africa

  21. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) done1 = false done2 = false ack k1 = 42 Server k2 = 0 done1 = false done1: Asia done2 = false done2: Africa

  22. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) done1: true done1 = false done2 = false k1 = 42 Server k2 = 0 done1 = false done1: Asia done2 = false done2: Africa

  23. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) done1 = false done2 = false invalidate: done1 k1 = 42 Server k2 = 0 done1 = false done1: Asia done2 = false done2: Africa

  24. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) done1 = false done2 = false ack k1 = 42 Server k2 = 0 done1 = true done1: Asia done2 = false done2: Africa

  25. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) done1 = false done2 = false ack k1 = 42 Server k2 = 0 done1 = true done1: done2 = false done2: Africa

  26. Client Client Client while(get(done1) == false) while(get(done2) == false) put (k1, f(data)) ; ; put (done1, true) put (k2, g(get(k1)); rslt = h(get(k1), get(k2)) put (done2, true) done1 = true done2 = false k1 = 42 Server k2 = 0 done1 = true done1: Asia done2 = false done2: Africa

  27. Questions While a write to key k is waiting on invalidations, can other clients read old values of k from their caches?

  28. Questions While a write to key k from client C is waiting on invalidations, can C perform another write to a different key m?

  29. Questions While a write to key k from client C is waiting on invalidations, can the server perform a read from a different client D to a different key m?

  30. Questions While a write to key k from client C is waiting on invalidations, can the server perform a read to k from a different client D?

  31. Questions While a write to key k from client C is waiting on invalidations, can the server perform a write from client D to the same key?

  32. Facebook’s Memcache Service

  33. Facebook’s Scaling Problem • Rapidly increasing user base – Small initial user base – 2x every 9 months – 2013: 1B users globally • Users read/update many times per day – Increasingly intensive app logic per user – 2x I/O every 4-6 months • Infrastructure has to keep pace

  34. Scaling Strategy Adapt off the shelf components where possible Fix as you go – no overarching plan Rule of thumb: Every order of magnitude requires a rethink

  35. Facebook Three Layer Architecture • Application front end – Stateless, rapidly changing program logic – If app server fails, redirect client to new app server • Memcache – Lookaside key-value cache – Keys defined by app logic (can be computed results) • Fault tolerant storage backend – Stateful – Careful engineering to provide safety and performance – Both SQL and NoSQL

  36. Workload Each user’s page is unique – draws on events posted by other users Users not in cliques – For the most part User popularity is zipf – Some user posts affect very large #’s of other pages – Most affect a much smaller number

  37. Scale By Caching: Memcache Sharded in-memory key-value cache – Key, values assigned by application code – Values can be data, result of computation – Independent of backend storage architecture (SQL, noSQL) or format – Design for high volume, low latency Lookaside architecture

  38. Lookaside Read Web Server get k (1) Cache data SQL

  39. Lookaside Read Web Server get k (1) Cache get k (2) data nope! SQL

  40. Lookaside Read Web Server put k (3) Cache get k (2) data ok! SQL

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Memcached Install, Overview & Benchmarks What is Memcached? Really, its Memcache-d (d as

Memcached Install, Overview & Benchmarks What is Memcached? Really, its Memcache-d (d as

Memcached Install, Overview & Benchmarks What is Memcached? Really, its Memcache-d (d as in daemon) TCP/IP service that allows you to store stuff in RAM, somewhat persistently Basically, a persistent hash table Super awesome

342 views • 13 slides
Multicore Workshop Caches Mark Bull David Henty EPCC, University of Edinburgh Overview

Multicore Workshop Caches Mark Bull David Henty EPCC, University of Edinburgh Overview

Multicore Workshop Caches Mark Bull David Henty EPCC, University of Edinburgh Overview Why caches are needed How caches work Cache design and performance. 20/11/2012 Caches 2 The memory speed gap Moores Law: processors

811 views • 27 slides
Trace Caches and optimizations therein CSE 240C - Rushi Chakrabarti - Winter 2009 Trace Caches

Trace Caches and optimizations therein CSE 240C - Rushi Chakrabarti - Winter 2009 Trace Caches

Trace Caches and optimizations therein CSE 240C - Rushi Chakrabarti - Winter 2009 Trace Caches [Rotenberg96] Trace Caches For those not in the know: I$ that captures dynamic instruction sequences trace n instructions (cache

432 views • 41 slides
Say Goodbye to Off-heap Caches! On-heap Caches Using Memory-Mapped I/O Iacovos G. Kolokasis 1 ,

Say Goodbye to Off-heap Caches! On-heap Caches Using Memory-Mapped I/O Iacovos G. Kolokasis 1 ,

Say Goodbye to Off-heap Caches! On-heap Caches Using Memory-Mapped I/O Iacovos G. Kolokasis 1 , Anastasios Papagiannis 1 , Foivos Zakkak 2 , Polyvios Pratikakis 1 , and Angelos Bilas 1 1 University of Crete & Foundation of Research and

457 views • 19 slides
What You Must Know about Memory, Caches, and Shared Memory Kenjiro Taura 1 / 67 Contents 1

What You Must Know about Memory, Caches, and Shared Memory Kenjiro Taura 1 / 67 Contents 1

What You Must Know about Memory, Caches, and Shared Memory Kenjiro Taura 1 / 67 Contents 1 Introduction 2 Organization of processors, caches, and memory 3 Caches 4 So how costly is it to access data? Latency Bandwidth Many algorithms are

1.16k views • 105 slides
CS 136: Advanced Architecture Review of Caches 1 / 30 Introduction Why Caches? Basic goal:

CS 136: Advanced Architecture Review of Caches 1 / 30 Introduction Why Caches? Basic goal:

CS 136: Advanced Architecture Review of Caches 1 / 30 Introduction Why Caches? Basic goal: Size of cheapest memory . . . At speed of most expensive Locality makes it work Temporal locality: If you reference x , youll probably

717 views • 33 slides
Review: Why We Use Caches Caches Review Mechanism for transparent movement of Proc 1000

Review: Why We Use Caches Caches Review Mechanism for transparent movement of Proc 1000

Review: Why We Use Caches Caches Review Mechanism for transparent movement of Proc 1000 CPU data among levels of a storage hierarchy 60%/yr. Moores Law Performance set of address/value bindings address index to

966 views • 11 slides
Memcache as a Service Tom Anderson Goals Rapid application development (velocity) - Speed

Memcache as a Service Tom Anderson Goals Rapid application development (velocity) - Speed

Memcache as a Service Tom Anderson Goals Rapid application development (velocity) - Speed of adding new features is paramount Scale Billions of users Every user on FB all the time Performance Low latency for every user

837 views • 52 slides
ECE232: Hardware Organization and Design Lecture 22: Introduction to Caches Adapted from Computer

ECE232: Hardware Organization and Design Lecture 22: Introduction to Caches Adapted from Computer

ECE232: Hardware Organization and Design Lecture 22: Introduction to Caches Adapted from Computer Organization and Design , Patterson & Hennessy, UCB Overview Caches hold a subset of data from the main memory Three types of caches

710 views • 25 slides
1 Implementation Snoop Caches Implementing Snooping Caches Write Races: Multiple processors

1 Implementation Snoop Caches Implementing Snooping Caches Write Races: Multiple processors

An Example Snoopy Protocol Invalidation protocol, write-back cache Each block of memory is in one state: Cache Coherence and Memory Clean in all caches and up-to-date in memory (Shared) Consistency OR Dirty in exactly one cache

507 views • 3 slides
CSE 351: Week 7 Tom Bergan, TA 1 Today Cache geometries Lab 4 2 Caches they make

CSE 351: Week 7 Tom Bergan, TA 1 Today Cache geometries Lab 4 2 Caches they make

CSE 351: Week 7 Tom Bergan, TA 1 Today Cache geometries Lab 4 2 Caches they make memory faster Main CPU Reg Cache Memory fast slow Tradeoff: caches are smaller than main memory 3 Why do caches work? Temporal locality: int

663 views • 19 slides
Scaling Memcache at Facebook Rajesh Nishtala, Hans Fugal, Steven Grimm, Marc Kwiatkowski, Herman

Scaling Memcache at Facebook Rajesh Nishtala, Hans Fugal, Steven Grimm, Marc Kwiatkowski, Herman

Scaling Memcache at Facebook Rajesh Nishtala, Hans Fugal, Steven Grimm, Marc Kwiatkowski, Herman Lee, Harry C. Li, Ryan McElroy, Mike Paleczny, Daniel Peek, Paul Saab, David Stafford, Tony Tung, Venkateshwaran Venkataramani Cesar Stuardo 2

824 views • 26 slides
Caches Electronic Computers M Caches 1 Cache LOCALITY PRINCIPLE (SPATIAL AND TEMPORAL)

Caches Electronic Computers M Caches 1 Cache LOCALITY PRINCIPLE (SPATIAL AND TEMPORAL)

Caches Electronic Computers M Caches 1 Cache LOCALITY PRINCIPLE (SPATIAL AND TEMPORAL) WORKING SET CPU Cache Registers Cache I lev. Cache II lev. Cache III lev. Memory Disk Tape The cache is a memory with an access time some

572 views • 55 slides
Associative caches (3 rd Ed: p.496-504, 4 th Ed: 479-487) flexible block placement schemes

Associative caches (3 rd Ed: p.496-504, 4 th Ed: 479-487) flexible block placement schemes

Associative caches (3 rd Ed: p.496-504, 4 th Ed: 479-487) flexible block placement schemes overview of set associative caches block replacement strategies associative cache implementation size and performance Direct-mapped

571 views • 22 slides
What You Must Know about Memory, Caches, and Shared Memory Kenjiro Taura 1 / 105 Contents 1

What You Must Know about Memory, Caches, and Shared Memory Kenjiro Taura 1 / 105 Contents 1

What You Must Know about Memory, Caches, and Shared Memory Kenjiro Taura 1 / 105 Contents 1 Introduction 2 Many algorithms are bounded by memory not CPU 3 Organization of processors, caches, and memory 4 So how costly is it to access data?

1.81k views • 166 slides
Caching 1 Caches break down an address into which parts? Letter Answer A Tag, delay, length

Caching 1 Caches break down an address into which parts? Letter Answer A Tag, delay, length

Caching 1 Caches break down an address into which parts? Letter Answer A Tag, delay, length B Max, min, average C High-order and low-order D Tag, index, offset E Opcode, register, immediate 2 Caches operate on units of memory

884 views • 49 slides
Human Capital Persistence and Development Rudi Rocha (UFRJ) Claudio Ferraz (PUC-Rio) Rodrigo R.

Human Capital Persistence and Development Rudi Rocha (UFRJ) Claudio Ferraz (PUC-Rio) Rodrigo R.

Human Capital Persistence and Development Rudi Rocha (UFRJ) Claudio Ferraz (PUC-Rio) Rodrigo R. Soares (EESP-FGV) WIDER Development Conference June 7, 2016 June Rocha, Ferraz e Soares (2016) Human Capital Persistence and Development - WIDER,

488 views • 20 slides
A C A CAS ASH CR H CROP OP WITH H DR. . ANDREA REA BA BASC SCHE HE Tech

A C A CAS ASH CR H CROP OP WITH H DR. . ANDREA REA BA BASC SCHE HE Tech

HOW T HOW TO T O TREA EAT T YOU OUR COVE VER R CROP OP LI LIKE KE A C A CAS ASH CR H CROP OP WITH H DR. . ANDREA REA BA BASC SCHE HE Tech Orientation Welcome & Introductions Presentation Q&A

679 views • 47 slides
Soil Fertility Management Daniel H. Smith, CCA Southwest Regional Specialist Nutrient and Pest

Soil Fertility Management Daniel H. Smith, CCA Southwest Regional Specialist Nutrient and Pest

@agronomybadger Soil Fertility Management Daniel H. Smith, CCA Southwest Regional Specialist Nutrient and Pest Management Program University of Wisconsin-Madison CALS and Division of Extension Disclaimer: Any use of trade names is for

608 views • 16 slides
Document Misplacement for IR Evaluation Nicola Ferro Information Management Systems (IMS)

Document Misplacement for IR Evaluation Nicola Ferro Information Management Systems (IMS)

Document Misplacement for IR Evaluation Nicola Ferro Information Management Systems (IMS) Research Group Department of Information Engineering (DEI) University of Padua, Italy Forum for Information Retrieval Evaluation (FIRE 2013) 4 - 6

377 views • 27 slides
Stanford CS193p Developing Applications for iOS Fall 2011 Stanford CS193p Fall 2011 Today

Stanford CS193p Developing Applications for iOS Fall 2011 Stanford CS193p Fall 2011 Today

Stanford CS193p Developing Applications for iOS Fall 2011 Stanford CS193p Fall 2011 Today Core Data and Documents This is how you store something serious in iOS Easy entre into iCloud NSNotificationCenter The little radio station

668 views • 62 slides
Change-point methods for anomaly detection in fibrous media joint work with E. Spodarev, C. Re-

Change-point methods for anomaly detection in fibrous media joint work with E. Spodarev, C. Re-

Change-point methods for anomaly detection in fibrous media joint work with E. Spodarev, C. Re- denbach, D. Dresvyanskiy, T. Karaseva Vitalii Makogin | 10.10.2019 | Institute of Stochastics, Ulm University and S. Mitrofanov Seite 2 Problem

451 views • 19 slides
Effective Anti-thrombotic Therapy without Stenting: Intravascular OCT-based Management in Plaque

Effective Anti-thrombotic Therapy without Stenting: Intravascular OCT-based Management in Plaque

Effective Anti-thrombotic Therapy without Stenting: Intravascular OCT-based Management in Plaque Erosion (the EROSION study) Ik-Kyung Jang, MD, PhD In collaboration with Bo Yu, MD, PhD Harvard Medical School Harbin Medical University

421 views • 10 slides
Characteristic number regression for fiducial facial feature extraction Presenter : Xin Fan

Characteristic number regression for fiducial facial feature extraction Presenter : Xin Fan

Characteristic number regression for fiducial facial feature extraction Presenter : Xin Fan Joint work with Prof. Zhongxuan Luo and Dr. Risheng Liu Published in IEEE TIP15 and ICME15 Outline Motivations Facial structure with

890 views • 38 slides

More recommend