23
play

23 Databases Intro to Database Systems Andy Pavlo AP AP - PowerPoint PPT Presentation

Jul 15, 2023 •340 likes •1.21k views

Distributed OLTP 23 Databases Intro to Database Systems Andy Pavlo AP AP 15-445/15-645 Computer Science Fall 2019 Carnegie Mellon University 2 ADM INISTRIVIA Homework #5 : Monday Dec 3 rd @ 11:59pm Project #4 : Monday Dec 10 th @ 11:59pm

  1. Distributed OLTP 23 Databases Intro to Database Systems Andy Pavlo AP AP 15-445/15-645 Computer Science Fall 2019 Carnegie Mellon University

  2. 2 ADM INISTRIVIA Homework #5 : Monday Dec 3 rd @ 11:59pm Project #4 : Monday Dec 10 th @ 11:59pm Extra Credit : Wednesday Dec 10 th @ 11:59pm Final Exam : Monday Dec 9 th @ 5:30pm CMU 15-445/645 (Fall 2019)

  3. 3 LAST CLASS System Architectures → Shared-Memory, Shared-Disk, Shared-Nothing Partitioning/Sharding → Hash, Range, Round Robin Transaction Coordination → Centralized vs. Decentralized CMU 15-445/645 (Fall 2019)

  4. 4 OLTP VS. OLAP On-line Transaction Processing (OLTP): → Short-lived read/write txns. → Small footprint. → Repetitive operations. On-line Analytical Processing (OLAP): → Long-running, read-only queries. → Complex joins. → Exploratory queries. CMU 15-445/645 (Fall 2019)

  5. 5 DECENTRALIZED COORDINATO R Partitions Begin Request P1 P2 Application Server P3 P4 CMU 15-445/645 (Fall 2019)

  6. 5 DECENTRALIZED COORDINATO R Partitions Query P1 P2 Query Application Server P3 P4 Query CMU 15-445/645 (Fall 2019)

  7. 5 DECENTRALIZED COORDINATO R Partitions Commit Request P1 P2 Safe to commit? Application Server P3 P4 CMU 15-445/645 (Fall 2019)

  8. 6 OBSERVATION We have not discussed how to ensure that all nodes agree to commit a txn and then to make sure it does commit if we decide that it should. → What happens if a node fails? → What happens if our messages show up late? → What happens if we don't wait for every node to agree? CMU 15-445/645 (Fall 2019)

  9. 7 IM PORTAN T ASSUM PTIO N We can assume that all nodes in a distributed DBMS are well-behaved and under the same administrative domain. → If we tell a node to commit a txn, then it will commit the txn (if there is not a failure). If you do not trust the other nodes in a distributed DBMS, then you need to use a Byzantine Fault Tolerant protocol for txns (blockchain). CMU 15-445/645 (Fall 2019)

  10. 8 TODAY'S AGENDA Atomic Commit Protocols Replication Consistency Issues (CAP) Federated Databases CMU 15-445/645 (Fall 2019)

  11. 9 ATOM IC COM M IT PROTO CO L When a multi-node txn finishes, the DBMS needs to ask all the nodes involved whether it is safe to commit. Examples: → Two-Phase Commit → Three-Phase Commit (not used) → Paxos → Raft → ZAB (Apache Zookeeper) → Viewstamped Replication CMU 15-445/645 (Fall 2019)

  12. 10 TWO- PH ASE COM M IT (SUCCESS) Commit Request Participant Application Server Node 2 Coordinator Participant Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  13. 10 TWO- PH ASE COM M IT (SUCCESS) Commit Request Participant Application Server Node 2 Phase1: Prepare Coordinator Participant Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  14. 10 TWO- PH ASE COM M IT (SUCCESS) Commit Request Participant OK Application Server Node 2 Phase1: Prepare OK Coordinator Participant Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  15. 10 TWO- PH ASE COM M IT (SUCCESS) Commit Request Participant OK Application Server Node 2 Phase1: Prepare OK Coordinator Participant Phase2: Commit Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  16. 10 TWO- PH ASE COM M IT (SUCCESS) Commit Request Participant OK OK Application Server Node 2 Phase1: Prepare OK Coordinator Participant Phase2: Commit OK Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  17. 10 TWO- PH ASE COM M IT (SUCCESS) Success! Participant Application Server Node 2 Coordinator Participant Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  18. 11 TWO- PH ASE COM M IT (ABORT) Commit Request Participant Application Server Node 2 Coordinator Participant Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  19. 11 TWO- PH ASE COM M IT (ABORT) Commit Request Participant Application Server Node 2 Phase1: Prepare Coordinator Participant Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  20. 11 TWO- PH ASE COM M IT (ABORT) Commit Request Participant Application Server Node 2 Phase1: Prepare ABORT! Coordinator Participant Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  21. 11 TWO- PH ASE COM M IT (ABORT) Aborted Participant Application Server Node 2 ABORT! Coordinator Participant Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  22. 11 TWO- PH ASE COM M IT (ABORT) Aborted Participant Application Server Node 2 ABORT! Coordinator Participant Phase2: Abort Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  23. 11 TWO- PH ASE COM M IT (ABORT) Aborted Participant OK Application Server Node 2 ABORT! Coordinator Participant Phase2: Abort OK Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  24. 12 2PC OPTIM IZATION S Early Prepare Voting → If you send a query to a remote node that you know will be the last one you execute there, then that node will also return their vote for the prepare phase with the query result. Early Acknowledgement After Prepare → If all nodes vote to commit a txn, the coordinator can send the client an acknowledgement that their txn was successful before the commit phase finishes. CMU 15-445/645 (Fall 2019)

  25. 13 EARLY ACKNOWLEDGEM EN T Commit Request Participant Application Server Node 2 Coordinator Participant Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  26. 13 EARLY ACKNOWLEDGEM EN T Commit Request Participant Application Server Node 2 Phase1: Prepare Coordinator Participant Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  27. 13 EARLY ACKNOWLEDGEM EN T Commit Request Participant OK Application Server Node 2 Phase1: Prepare OK Coordinator Participant Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  28. 13 EARLY ACKNOWLEDGEM EN T Success! Participant OK Application Server Node 2 Phase1: Prepare OK Coordinator Participant Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  29. 13 EARLY ACKNOWLEDGEM EN T Success! Participant OK Application Server Node 2 Phase1: Prepare OK Coordinator Participant Phase2: Commit Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  30. 13 EARLY ACKNOWLEDGEM EN T Success! Participant OK OK Application Server Node 2 Phase1: Prepare OK Coordinator Participant Phase2: Commit OK Node 1 Node 3 CMU 15-445/645 (Fall 2019)

  31. 14 TWO- PH ASE COM M IT Each node records the outcome of each phase in a non-volatile storage log. What happens if coordinator crashes? → Participants must decide what to do. What happens if participant crashes? → Coordinator assumes that it responded with an abort if it hasn't sent an acknowledgement yet. CMU 15-445/645 (Fall 2019)

  32. 15 PAXOS Consensus protocol where a coordinator proposes an outcome (e.g., commit or abort) and then the participants vote on whether that outcome should succeed. Does not block if a majority of participants are available and has provably minimal message delays in the best case. CMU 15-445/645 (Fall 2019)

  33. 16 PAXOS Acceptor Commit Request Node 2 Acceptor Application Server Node 3 Proposer Acceptor Node 1 Node 4 CMU 15-445/645 (Fall 2019)

  34. 16 PAXOS Acceptor Commit Request Node 2 Acceptor Application Server Propose Node 3 Proposer Acceptor Node 1 Node 4 CMU 15-445/645 (Fall 2019)

  35. 16 PAXOS Acceptor Commit Request Node 2 X Acceptor Application Server Propose Node 3 Proposer Acceptor Node 1 Node 4 CMU 15-445/645 (Fall 2019)

  36. 16 PAXOS Acceptor Agree Commit Request Node 2 X Acceptor Application Server Propose Node 3 Agree Proposer Acceptor Node 1 Node 4 CMU 15-445/645 (Fall 2019)

  37. 16 PAXOS Acceptor Agree Commit Request Node 2 X Acceptor Application Server Propose Node 3 Commit Agree Proposer Acceptor Node 1 Node 4 CMU 15-445/645 (Fall 2019)

  38. 16 PAXOS Acceptor Agree Commit Request Accept Node 2 X Acceptor Application Server Propose Node 3 Commit Agree Proposer Acceptor Accept Node 1 Node 4 CMU 15-445/645 (Fall 2019)

  39. 16 PAXOS Acceptor Success! Node 2 X Acceptor Application Server Node 3 Proposer Acceptor Node 1 Node 4 CMU 15-445/645 (Fall 2019)

  40. 17 PAXOS Proposer Acceptors Proposer TIM E CMU 15-445/645 (Fall 2019)

  41. 17 PAXOS Proposer Acceptors Proposer Propose(n) TIM E CMU 15-445/645 (Fall 2019)

  42. 17 PAXOS Proposer Acceptors Proposer Propose(n) Agree(n) TIM E CMU 15-445/645 (Fall 2019)

  43. 17 PAXOS Proposer Acceptors Proposer Propose(n) Agree(n) Propose(n+1) TIM E CMU 15-445/645 (Fall 2019)

  44. 17 PAXOS Proposer Acceptors Proposer Propose(n) Agree(n) Propose(n+1) Commit(n) TIM E CMU 15-445/645 (Fall 2019)

  45. 17 PAXOS Proposer Acceptors Proposer Propose(n) Agree(n) Propose(n+1) Commit(n) TIM E Reject(n,n+1) CMU 15-445/645 (Fall 2019)

  46. 17 PAXOS Proposer Acceptors Proposer Propose(n) Agree(n) Propose(n+1) Commit(n) TIM E Reject(n,n+1) Agree(n+1) CMU 15-445/645 (Fall 2019)

  47. 17 PAXOS Proposer Acceptors Proposer Propose(n) Agree(n) Propose(n+1) Commit(n) TIM E Reject(n,n+1) Agree(n+1) Commit(n+1) CMU 15-445/645 (Fall 2019)

  48. 17 PAXOS Proposer Acceptors Proposer Propose(n) Agree(n) Propose(n+1) Commit(n) TIM E Reject(n,n+1) Agree(n+1) Commit(n+1) Accept(n+1) CMU 15-445/645 (Fall 2019)

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

CS 764: Topics in Database Management Systems Lecture 11: Two-Phase Commit (2PC) Xiangyao Yu

CS 764: Topics in Database Management Systems Lecture 11: Two-Phase Commit (2PC) Xiangyao Yu

CS 764: Topics in Database Management Systems Lecture 11: Two-Phase Commit (2PC) Xiangyao Yu 10/12/2020 1 Announcement Submit a 1-page course project proposal by Oct. 21 A list of project ideas are uploaded to the course website

961 views • 37 slides
RPC (fjnish) / two-phase commit 1 Changelog Changes made in this version not seen in fjrst

RPC (fjnish) / two-phase commit 1 Changelog Changes made in this version not seen in fjrst

RPC (fjnish) / two-phase commit 1 Changelog Changes made in this version not seen in fjrst lecture: 19 November 2019: gRPC IDL example: update to be consistent with version of gRPC syntax used in assignment 19 November 2019: gRPC IDL example:

1.79k views • 142 slides
CS5412: TRANSACTIONS (I) Lecture XVII Ken Birman Transactions A widely used reliability

CS5412: TRANSACTIONS (I) Lecture XVII Ken Birman Transactions A widely used reliability

CS5412 Spring 2012 (Cloud Computing: Birman) 1 CS5412: TRANSACTIONS (I) Lecture XVII Ken Birman Transactions A widely used reliability technology, despite the BASE methodology we use in the first tier Goal for this week: in-depth

1.36k views • 49 slides
Statistical Methods for Plant Biology PBIO 3150/5150 Anirudh V. S. Ruhil February 24, 2016 The

Statistical Methods for Plant Biology PBIO 3150/5150 Anirudh V. S. Ruhil February 24, 2016 The

Statistical Methods for Plant Biology PBIO 3150/5150 Anirudh V. S. Ruhil February 24, 2016 The Voinovich School of Leadership and Public Affairs 1/29 Table of Contents 1 Elements of Good Research Designs 2 Three Case Studies 3 Matching

753 views • 29 slides
Distributed Transaction Management Advanced Topics in Database Management (INFSCI 2711) Some

Distributed Transaction Management Advanced Topics in Database Management (INFSCI 2711) Some

Distributed Transaction Management Advanced Topics in Database Management (INFSCI 2711) Some materials are from Database Management Systems, Ramakrishnan and Gehrke and Database System Concepts, Siberschatz, Korth and Sudarshan and Data

515 views • 24 slides
Data Streams: Random Order & Multiple Passes 2009 Barbados Workshop on Computational

Data Streams: Random Order & Multiple Passes 2009 Barbados Workshop on Computational

Data Streams: Random Order & Multiple Passes 2009 Barbados Workshop on Computational Complexity Andrew McGregor Introduction Random Order Streams: Average case analysis: data is worst-case but order is random. Lower bounds are more

1.27k views • 84 slides
Nested Transactions Nested Transactions Flat transactions The rules for committing of

Nested Transactions Nested Transactions Flat transactions The rules for committing of

Nested Transactions Nested Transactions Flat transactions The rules for committing of nested transactions Nested Transactions A transaction commits or aborts only after its child transactions have completed; Structured in

808 views • 11 slides
11/21/2013 Location of Facility Primary Beamline MI Abort Line Reconfiguration

11/21/2013 Location of Facility Primary Beamline MI Abort Line Reconfiguration

Photo by Marty Murphy Michael Geelhoed AD EBD 11/21/2013 Location of Facility Primary Beamline MI Abort Line Reconfiguration Extraction Michael Geelhoed AD EBD 11/21/2013 Location of Facility 11/21/2013 Michael Geelhoed AD

426 views • 23 slides
Transactional Memory: Architectural support for Lock-Free Data Structure Transactional Memory:

Transactional Memory: Architectural support for Lock-Free Data Structure Transactional Memory:

Transactional Memory: Architectural support for Lock-Free Data Structure Transactional Memory: Architectural Support for Lock-Free Data By Maurice Herlihy and J. Eliot B. Moss Structures 1993 By Maurice Herlihy and J Eliot B. Moss 1993

499 views • 19 slides
ADAPTIVE TWO-STAGE INTEGRATORS FOR SAMPLING ALGORITHMS BASED ON HAMILTONIAN DYNAMICS E.

ADAPTIVE TWO-STAGE INTEGRATORS FOR SAMPLING ALGORITHMS BASED ON HAMILTONIAN DYNAMICS E.

ADAPTIVE TWO-STAGE INTEGRATORS FOR SAMPLING ALGORITHMS BASED ON HAMILTONIAN DYNAMICS E. Akhmatskaya a,c , M. Fernndez-Pends a , T. Radivojevi a , J. M. Sanz-Serna b a Basque Center for Applied Mathematics (BCAM), Bilbao, Spain b Departamento

594 views • 37 slides
Distributed Systems (ICE 601) Transactions & Concurrency Control - Part1 Dongman Lee ICU

Distributed Systems (ICE 601) Transactions & Concurrency Control - Part1 Dongman Lee ICU

Distributed Systems (ICE 601) Transactions & Concurrency Control - Part1 Dongman Lee ICU Class Overview Transactions Why Concurrency Control Concurrency Control Protocols pessimistic optimistic time-based

381 views • 9 slides
What are Exceptions? Exceptions are rare events triggered by the hardware and forcing the

What are Exceptions? Exceptions are rare events triggered by the hardware and forcing the

8.1 8.2 What are Exceptions? Exceptions are rare events triggered by the hardware and forcing the processor to execute a software handler EE 457 Unit 8 Exceptions Similar to conditional branches/subroutine calls What

451 views • 6 slides
Scalability and Replication Marco Serafini COMPSCI 532 Lecture 13 Scalability 2 Scalability

Scalability and Replication Marco Serafini COMPSCI 532 Lecture 13 Scalability 2 Scalability

Scalability and Replication Marco Serafini COMPSCI 532 Lecture 13 Scalability 2 Scalability Ideal world Linear scalability Speedup Reality Ideal Bottlenecks For example: central coordinator When do we stop

938 views • 36 slides
Distributed Transactions and Concurrency CS425/ECE 428 Nikita Borisov Topics for Today

Distributed Transactions and Concurrency CS425/ECE 428 Nikita Borisov Topics for Today

Distributed Transactions and Concurrency CS425/ECE 428 Nikita Borisov Topics for Today Transaction semantics: ACID Isolation and serial equivalence Conflicting operations Two-phase locking Example transaction Switch from T3 to

289 views • 26 slides
Work Queue + Python A Framework For Scalable Scientific Ensemble Applications Peter Bui , Dinesh

Work Queue + Python A Framework For Scalable Scientific Ensemble Applications Peter Bui , Dinesh

Work Queue + Python A Framework For Scalable Scientific Ensemble Applications Peter Bui , Dinesh Rajan, Badi Abdul-Wahid, Jesus Izaguirre, Douglas Thain University of Notre Dame Distributed Computing Examples Examples Condor cluster

408 views • 21 slides
ECE 650 Systems Programming & Engineering Spring 2018 Database Transaction Processing Tyler

ECE 650 Systems Programming & Engineering Spring 2018 Database Transaction Processing Tyler

ECE 650 Systems Programming & Engineering Spring 2018 Database Transaction Processing Tyler Bletsch Duke University Slides are adapted from Brian Rogers (Duke) Transaction Processing Systems Systems with large DBs; many concurrent

695 views • 32 slides
COSC 5351 Advanced Computer Architecture Slides modified from Hennessy CS252 course slides MP

COSC 5351 Advanced Computer Architecture Slides modified from Hennessy CS252 course slides MP

COSC 5351 Advanced Computer Architecture Slides modified from Hennessy CS252 course slides MP Motivation SISD v. SIMD v. MIMD Centralized vs. Distributed Memory Challenges to Parallel Programming Consistency, Coherency, Write

1.37k views • 80 slides
Extreme Computing NoSQL www.inf.ed.ac.uk PREVIOUSLY: BATCH Query most/all data Results

Extreme Computing NoSQL www.inf.ed.ac.uk PREVIOUSLY: BATCH Query most/all data Results

Extreme Computing NoSQL www.inf.ed.ac.uk PREVIOUSLY: BATCH Query most/all data Results Eventually NOW: ON DEMAND Single Data Points Latency Matters www.inf.ed.ac.uk One problem, three ideas We want to keep track of mutable state in a

781 views • 49 slides
Serializability with Snapshot Isolation under the Hood Mihaela Bornea 1 , S. Elnikety 2 , O.

Serializability with Snapshot Isolation under the Hood Mihaela Bornea 1 , S. Elnikety 2 , O.

Outline Motivation Concurrency Control Replication Model Readset Certification Evaluation Conclusions Serializability with Snapshot Isolation under the Hood Mihaela Bornea 1 , S. Elnikety 2 , O. Hodson 2 , A Fekete 3 1 IBM Research 2

475 views • 31 slides
Assessing Medication Adherence Dr. Lauren Hanna and Dr. Delbert Robinson Northwell Health

Assessing Medication Adherence Dr. Lauren Hanna and Dr. Delbert Robinson Northwell Health

Assessing Medication Adherence Dr. Lauren Hanna and Dr. Delbert Robinson Northwell Health National Council for Behavioral Health Montefiore Medical Center Northwell Health New York State Office of Mental Health Netsmart Technologies

963 views • 41 slides
Underground Injection Control (UIC) Permitting Rob Castillo July 2020 1 Railroad Commission of

Underground Injection Control (UIC) Permitting Rob Castillo July 2020 1 Railroad Commission of

Underground Injection Control (UIC) Permitting Rob Castillo July 2020 1 Railroad Commission of Texas | June 27, 2016 (Change Date In First Master Slide) Session Overview UIC Online Manual Safe Drinking Water Act & Primacy

747 views • 71 slides
PRP Teleconference July 2018 ATC Teleconference Agenda 1. Update on the renewal of the PRP

PRP Teleconference July 2018 ATC Teleconference Agenda 1. Update on the renewal of the PRP

Petroleum Restoration Program PRP Teleconference July 2018 ATC Teleconference Agenda 1. Update on the renewal of the PRP Agency Term Contract. 2. Payment of subcontractors. 3. Site Manager Standard Operating Procedures. 4. OSHA HAZWOPER

338 views • 9 slides
Estimating beyond the trial-represented population by incorporating studies with self-selected

Estimating beyond the trial-represented population by incorporating studies with self-selected

Estimating beyond the trial-represented population by incorporating studies with self-selected treatments Eloise E. Kaizar Department of Statistics Ohio State University February 26, 2016 Joint work with: Joel Greenhouse, Kelly Kelleher,

563 views • 18 slides
NHS City and Hackney CCG Commissioning Intentions 2017/18 and 2018/19 1 Tonight is about

NHS City and Hackney CCG Commissioning Intentions 2017/18 and 2018/19 1 Tonight is about

NHS City and Hackney CCG Commissioning Intentions 2017/18 and 2018/19 1 Tonight is about hearing your thoughts 2 Planning The NHS planning timescale this year means that the contracts for the services planned for the next two years must be

611 views • 24 slides

More recommend