eris coordination free consistent transactions using in
play

Eris: Coordination-Free Consistent Transactions Using In-Network - PowerPoint PPT Presentation

Mar 10, 2023 •199 likes •1.31k views

Eris: Coordination-Free Consistent Transactions Using In-Network Concurrency Control Jialin Li, Ellis Michael, Dan R. K. Ports Web services and applications rely on distributed storage systems Web services and applications rely on distributed

  1. Independent Transaction START TRANSACTION 
 START TRANSACTION 
 START TRANSACTION 
 UPDATE tb t1 UPDATE tb t1 UPDATE tb t1 SET t1.Salary = t1.Salary + 100 SET t1.Salary = t1.Salary + 100 SET t1.Salary = t1.Salary + 100 WHERE t1.Salary < 500 WHERE t1.Salary < 500 WHERE t1.Salary < 500 COMMIT COMMIT COMMIT Name Salary Name Name Salary Salary Name Name Salary Salary Alice 600 Bob Bob 450 350 Charlie Charlie 400 500

  2. Independent Transaction START TRANSACTION 
 UPDATE tb t1 SET t1.Salary = t1.Salary + 100 WHERE 500 < (SELECT AVG(t2.Salary) FROM tb t2) COMMIT START TRANSACTION 
 START TRANSACTION 
 START TRANSACTION 
 UPDATE tb t1 UPDATE tb t1 UPDATE tb t1 SET t1.Salary = t1.Salary + 100 SET t1.Salary = t1.Salary + 100 SET t1.Salary = t1.Salary + 100 WHERE t1.Salary < 500 WHERE t1.Salary < 500 WHERE t1.Salary < 500 COMMIT COMMIT COMMIT Name Salary Name Name Salary Salary Name Name Salary Salary Alice 600 Bob Bob 350 450 Charlie Charlie 500 400

  3. Independent Transaction START TRANSACTION 
 UPDATE tb t1 SET t1.Salary = t1.Salary + 100 ! WHERE 500 < (SELECT AVG(t2.Salary) FROM tb t2) t n e COMMIT d n e START TRANSACTION 
 START TRANSACTION 
 p START TRANSACTION 
 e UPDATE tb t1 UPDATE tb t1 d UPDATE tb t1 n SET t1.Salary = t1.Salary + 100 SET t1.Salary = t1.Salary + 100 I SET t1.Salary = t1.Salary + 100 t WHERE t1.Salary < 500 WHERE t1.Salary < 500 WHERE t1.Salary < 500 o N COMMIT COMMIT COMMIT Name Salary Name Name Salary Salary Name Name Salary Salary Alice 600 Bob Bob 350 450 Charlie Charlie 500 400

  4. Independent Transaction START TRANSACTION 
 START TRANSACTION 
 START TRANSACTION 
 UPDATE tb t1 UPDATE tb t1 UPDATE tb t1 SET t1.Salary = t1.Salary + 100 SET t1.Salary = t1.Salary + 100 SET t1.Salary = t1.Salary + 100 WHERE t1.Salary < 500 WHERE t1.Salary < 500 WHERE t1.Salary < 500 COMMIT COMMIT COMMIT Name Salary Name Name Salary Salary Name Name Salary Salary Alice 600 Bob Bob 450 350 Charlie Charlie 400 500

  5. Independent Transaction Many applications consist entirely of START TRANSACTION 
 START TRANSACTION 
 START TRANSACTION 
 independent transactions (e.g. TPC-C) UPDATE tb t1 UPDATE tb t1 UPDATE tb t1 SET t1.Salary = t1.Salary + 100 SET t1.Salary = t1.Salary + 100 SET t1.Salary = t1.Salary + 100 WHERE t1.Salary < 500 WHERE t1.Salary < 500 WHERE t1.Salary < 500 COMMIT COMMIT COMMIT Name Salary Name Name Salary Salary Name Name Salary Salary Alice 600 Bob Bob 350 450 Charlie Charlie 500 400

  6. Why independent transactions? • No coordination/communication across shards • Executing them serially at each shard in a consistent order guarantees serializability • Multi-sequenced groupcast establishes such an order • How to handle message drops and sequencer/ server failures ?

  7. Outline 1. Introduction 2. In-Network Concurrency Control 3. Transaction Model 4. Eris Protocol 5. Evaluation

  8. Normal Case Client Sequencer Shard 1 Learner Replica Replica Shard 2 Learner Replica Replica Shard 3 Learner Replica Replica

  9. Normal Case Client Sequencer Shard 1 Learner Replica Replica Shard 2 Learner Replica Replica Shard 3 Learner Replica Replica

  10. Normal Case Client Sequencer Shard 1 Learner Replica Replica Shard 2 Learner Replica Replica Shard 3 Learner Replica Replica

  11. Normal Case Client Sequencer Shard 1 Learner Replica Replica Shard 2 Learner Replica Replica Shard 3 Learner Replica Replica

  12. Normal Case Client Sequencer Shard 1 Learner Replica Replica Shard 2 Learner Replica Replica Shard 3 Learner Replica Replica

  13. Normal Case 1 round trip Client Sequencer Shard 1 Learner Replica Replica Shard 2 Learner Replica Replica Shard 3 Learner Replica Replica

  14. Normal Case 1 round trip Client Sequencer Shard 1 Learner Replica Replica no coordination Shard 2 Learner Replica Replica Shard 3 Learner Replica Replica

  15. How to handle dropped messages? A A1 A3 T3 T1 B1 DROP (A) (ABC) C1 B A1 T1 B1 (ABC) C1 C A1 T1 B1 (ABC) C1

  16. How to handle dropped messages? A A1 A3 T3 T1 B1 (A) (ABC) C1 B A1 T1 B1 (ABC) C1 C A1 T1 B1 (ABC) C1

  17. How to handle dropped messages? A A1 A3 T3 T1 B1 (A) (ABC) C1 B A1 A2 T1 T2 B1 B2 (ABC) (AB) C1 C A1 T1 B1 (ABC) C1

  18. How to handle dropped messages? A A1 A3 T3 T1 B1 (A) (ABC) C1 B A1 A2 T1 T2 B1 B2 (ABC) (AB) C1 C A1 T1 B1 (ABC) C1

  19. How to handle dropped messages? A A1 A3 T3 T1 B1 (A) (ABC) C1 B A1 A2 T1 T2 B1 B2 (ABC) (AB) C1 C A1 T1 B1 (ABC) C1

  20. How to handle dropped messages? A A1 A3 T3 T1 B1 (A) (ABC) C1 B Global coordination problem A1 A2 T1 T2 B1 B2 (ABC) (AB) C1 C A1 T1 B1 (ABC) C1

  21. The Failure Coordinator A A1 A3 T3 T1 B1 DROP (A) (ABC) C1 B A1 A2 T1 T2 B1 B2 (ABC) (AB) C1 Failure Coordinator C A1 T1 B1 (ABC) C1

  22. The Failure Coordinator A A1 A3 T3 T1 B1 Received A2? DROP (A) (ABC) C1 B A1 A2 T1 T2 B1 B2 (ABC) (AB) C1 Failure Coordinator C A1 T1 B1 (ABC) C1

  23. The Failure Coordinator A A1 A3 T3 T1 B1 DROP (A) (ABC) C1 B A1 A2 Received A2? Received A2? T1 T2 B1 B2 (ABC) (AB) C1 Failure Coordinator C A1 T1 B1 (ABC) C1

  24. The Failure Coordinator A A1 A3 T3 T1 B1 DROP (A) (ABC) C1 B A1 A2 Received A2? T1 T2 B1 B2 (ABC) (AB) C1 Failure Coordinator C A1 Received A2? T1 B1 (ABC) C1

  25. The Failure Coordinator A A1 A3 T3 T1 B1 DROP (A) (ABC) C1 B A1 A2 A2 T1 T2 T2 B1 B2 B2 (ABC) (AB) (AB) C1 Failure Coordinator C A1 T1 B1 Not Found (ABC) C1

  26. The Failure Coordinator A A1 A3 T3 T1 B1 DROP (A) (ABC) C1 B A1 A2 A2 T1 T2 T2 B1 B2 B2 (ABC) (AB) (AB) C1 Failure Not Found Coordinator C A1 T1 B1 (ABC) C1

  27. The Failure Coordinator A A1 A3 T3 T1 A2 B1 DROP T2 (A) (ABC) B2 C1 (AB) B A1 A2 T1 T2 B1 B2 (ABC) (AB) C1 Failure Coordinator C A1 T1 B1 (ABC) C1

  28. The Failure Coordinator A A1 A3 A2 T3 T1 T2 B1 B2 (A) (ABC) (AB) C1 B A1 A2 T1 T2 B1 B2 (ABC) (AB) C1 Failure Coordinator C A1 T1 B1 (ABC) C1

  29. The Failure Coordinator A A1 A3 T3 T1 B1 DROP (A) (ABC) C1 B A1 T1 B1 Received A2? (ABC) C1 Failure Coordinator C A1 T1 B1 Received A2? (ABC) C1

  30. The Failure Coordinator A A1 A3 T3 T1 B1 DROP (A) (ABC) C1 B A1 T1 B1 Not Found (ABC) C1 Failure Coordinator C A1 T1 B1 Not Found (ABC) C1

  31. The Failure Coordinator A A1 A3 T3 T1 B1 DROP (A) (ABC) C1 B A1 T1 B1 Not Found (ABC) C1 Failure Not Found Coordinator C A1 T1 B1 (ABC) C1

  32. The Failure Coordinator A A1 A3 T3 T1 B1 DROP Drop A2 (A) (ABC) C1 B A1 T1 B1 Drop A2 (ABC) C1 Failure Coordinator C A1 T1 B1 Drop A2 (ABC) C1

  33. The Failure Coordinator A A1 A3 NO T3 T1 B1 Drop A2 OP (A) (ABC) C1 B A1 T1 B1 Drop A2 (ABC) C1 Failure Coordinator C A1 T1 B1 Drop A2 (ABC) C1

  34. The Failure Coordinator A A1 A3 NO T3 T1 B1 Drop A2 OP (A) (ABC) C1 Drops: A2 B A1 T1 B1 Drop A2 (ABC) C1 Failure Drops: A2 Coordinator C A1 T1 B1 Drop A2 (ABC) C1

  35. Designated Learner and Sequencer Failures Designated learner (DL) failure: • View change based protocol • Ensures new DL learns all committed transactions from previous views Sequencer failure: • Higher epoch number from the new sequencer • Epoch change ensures all replicas across all shards start the new epoch in consistent states

  36. Can we process non-independent transactions efficiently?

  37. Can we process non-independent transactions efficiently? Yes, by dividing them into multiple independent transactions (See the paper!)

  38. Outline 1. Introduction 2. In-Network Concurrency Control 3. Transaction Model 4. Eris Protocol 5. Evaluation

  39. Evaluation Setup • 3-level fat-tree topology testbed • 15 shards, 3 replicas per shard • 2.5 GHz Intel Xeon E5-2680 servers • Middlebox sequencer implementation using Cavium Octeon CN6880 • YCSB+T and TPC-C workloads

  40. Comparison Systems • Lock-Store (2PC + 2PL + Paxos) • TAPIR [SOSP ’15] • Granola [ATC ‘12] • Non-transactional, unreplicated (NT-UR)

  41. Eris performs well on independent transactions Distributed independent transactions 1,200K Throughput (txns/sec) 900K 600K 300K 0K Eris Lock-Store TAPIR Granola NT-UR

  42. Eris performs well on independent transactions Distributed independent transactions 1,200K Throughput (txns/sec) 900K Eris outperforms 
 Lock-Store, TAPIR and 600K Granola by more than 3X 300K 0K Eris Lock-Store TAPIR Granola NT-UR

  43. Eris performs well on independent transactions Distributed independent transactions 1,200K Eris achieves Throughput (txns/sec) throughput within 10% of NT-UR 900K Eris outperforms 
 Lock-Store, TAPIR and 600K Granola by more than 3X 300K 0K Eris Lock-Store TAPIR Granola NT-UR

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

THE ERIS ADAPTIVE OPTICS SYSTEM ERIS ERIS, the Enhanced Resolution Imager and Spectrograph, is

THE ERIS ADAPTIVE OPTICS SYSTEM ERIS ERIS, the Enhanced Resolution Imager and Spectrograph, is

Guido Agapito THE ERIS ADAPTIVE OPTICS SYSTEM ERIS ERIS, the Enhanced Resolution Imager and Spectrograph, is an instrument for the Cassegrain focus of UT4 at the ESO VLT. It comprises: two science instruments : NIX : IR imager

145 views • 10 slides
Coordination-Free Computations Christopher Meiklejohn LASP DISTRIBUTED, EVENTUALLY CONSISTENT

Coordination-Free Computations Christopher Meiklejohn LASP DISTRIBUTED, EVENTUALLY CONSISTENT

Coordination-Free Computations Christopher Meiklejohn LASP DISTRIBUTED, EVENTUALLY CONSISTENT COMPUTATIONS CHRISTOPHER MEIKLEJOHN (BASHO TECHNOLOGIES, INC.) PETER VAN ROY (UNIVERSIT CATHOLIQUE DE LOUVAIN) 2 LASP MOTIVATION 3

1.37k views • 134 slides
Conference on International Coordination of Secured Transactions Law Reforms Neil B. Cohen

Conference on International Coordination of Secured Transactions Law Reforms Neil B. Cohen

Conference on International Coordination of Secured Transactions Law Reforms Neil B. Cohen Jeffrey D. Forchelli Professor of Law Brooklyn Law School What I Have Learned About International Secured Transactions Law Reform (mostly the hard way)

352 views • 8 slides
Free Transactions with WITH RIO VISTA Rio Vista David E. Lowell University of Michigan David

Free Transactions with WITH RIO VISTA Rio Vista David E. Lowell University of Michigan David

FREE TRANSACTIONS Free Transactions with WITH RIO VISTA Rio Vista David E. Lowell University of Michigan David E. Lowell Peter M. Chen Rio Project Electrical Engineering and Computer Science University of Michigan FREE TRANSACTIONS

471 views • 24 slides
Meerkat: Multicore-Scalable Replicated Transactions Following the Zero Coordination Principle 1

Meerkat: Multicore-Scalable Replicated Transactions Following the Zero Coordination Principle 1

Meerkat: Multicore-Scalable Replicated Transactions Following the Zero Coordination Principle 1 Distributed storage systems are getting faster Peak throughput (million txns/s) Can we achieve this? 10 in-memory, kernel-bypass 8 6 4

458 views • 12 slides
The Europen Resarch Council Diego Chialva ERC Executive Agency, Unit A1 The ERC Research

The Europen Resarch Council Diego Chialva ERC Executive Agency, Unit A1 The ERC Research

The Europen Resarch Council Diego Chialva ERC Executive Agency, Unit A1 The ERC Research Information System: ERIS February 27 th 2019 Outline ERIS : as an Information System ERIS : the ERC, ERCEA and Unit A1 ERIS : in practice First, a bit of

614 views • 23 slides
L INGNIERIE UN MTIER VIVRE ENSEMBLE www.viveris.fr Agenda 2 Vi Viveris eris

L INGNIERIE UN MTIER VIVRE ENSEMBLE www.viveris.fr Agenda 2 Vi Viveris eris

CoMoSef project Viveris activities 14/12/2016 L INGNIERIE UN MTIER VIVRE ENSEMBLE www.viveris.fr Agenda 2 Vi Viveris eris prese esentat ntatio ion Vi Viveris eris rol ole e in in C CoM oMoSef oSef 802.11p

582 views • 14 slides
NetChain: Scale-Free Sub-RTT Coordination Xin Jin Xiaozhou Li, Haoyu Zhang, Robert Soul,

NetChain: Scale-Free Sub-RTT Coordination Xin Jin Xiaozhou Li, Haoyu Zhang, Robert Soul,

NetChain: Scale-Free Sub-RTT Coordination Xin Jin Xiaozhou Li, Haoyu Zhang, Robert Soul, Jeongkeun Lee, Nate Foster, Changhoon Kim, Ion Stoica Conventional wisdom: avoid coordination NetChain: lightning fast coordination enabled by

484 views • 37 slides
Transactions Definition a sequence of one or more operations on one or more resources

Transactions Definition a sequence of one or more operations on one or more resources

Transactions Definition a sequence of one or more operations on one or more resources that is A tomic: all or nothing Distributed Systems (ICE 601) C onsistent: takes system from one consistent state to another Transactions

855 views • 5 slides
ZFS UTH Always consistent on disk Under The Hood No journal not needed Superlite

ZFS UTH Always consistent on disk Under The Hood No journal not needed Superlite

ZFS I/O Stack Object-Based Transactions Make these 7 changes to these 3 objects ZFS All-or-nothing Transaction Group Commit Again, all-or-nothing DMU ZFS UTH Always consistent on disk Under The Hood No

519 views • 17 slides
Time and global state; Coordination and agreement; Distributed transactions Oleg Batrashev

Time and global state; Coordination and agreement; Distributed transactions Oleg Batrashev

Time and global state Coordination and agreement Distributed transactions Time and global state; Coordination and agreement; Distributed transactions Oleg Batrashev Institute of Computer Science December 11, 2015 Time and global state

454 views • 30 slides
Coordination-free query evaluation and multi-query optimization in parallel and distributed

Coordination-free query evaluation and multi-query optimization in parallel and distributed

Formal approaches to: Coordination-free query evaluation and multi-query optimization in parallel and distributed systems Bas Ketsman Outline CALM Formalization CALM Revision 1 Coordination-free evaluation Conclusion Parallel-Correctness

1.25k views • 88 slides
Science drivers for ERIS as a stepping stone for E-ELTs G. B ONO Setting the scene E-ELT in a

Science drivers for ERIS as a stepping stone for E-ELTs G. B ONO Setting the scene E-ELT in a

Science drivers for ERIS as a stepping stone for E-ELTs G. B ONO Setting the scene E-ELT in a nutshell First light instruments ERIS Paving the way for E-ELT Future Perspectives First Generation E-ELT Instruments First Light E-ELT -- CAM

554 views • 24 slides
Non-Blocking Inter-Partition Communication with Wait-Free Pair Transactions Ethan Blanton and

Non-Blocking Inter-Partition Communication with Wait-Free Pair Transactions Ethan Blanton and

Non-Blocking Inter-Partition Communication with Wait-Free Pair Transactions Ethan Blanton and Lukasz Ziarek Fiji Systems, Inc. October 10 th , 2013 Introduction WFPT Details Transactions Implementation Evaluation and Results Conclusions

547 views • 30 slides
ZooKeeper: Wait-free coordination for Internet-scale systems Xuyang Zhang Zhesheng Xie What is

ZooKeeper: Wait-free coordination for Internet-scale systems Xuyang Zhang Zhesheng Xie What is

ZooKeeper: Wait-free coordination for Internet-scale systems Xuyang Zhang Zhesheng Xie What is ZooKeeper A distributed coordination service for distributed applications that providing a simple and high performance kernel for building more

693 views • 37 slides
Advanced Tax Considerations for Negotiating, Structuring and Documenting M&amp;A Transactions

Advanced Tax Considerations for Negotiating, Structuring and Documenting M&amp;A Transactions

Presenting a live 90-minute webinar with interactive Q&amp;A Advanced Tax Considerations for Negotiating, Structuring and Documenting M&amp;A Transactions Evaluating Taxable Versus Tax-Free Deals, Stock Sales Versus Asset Sales, Tax-Free

1.09k views • 72 slides
Lab 1: Packet Sniffing and Wireshark Fengwei Zhang Wayne State University Course: Cyber

Lab 1: Packet Sniffing and Wireshark Fengwei Zhang Wayne State University Course: Cyber

Lab 1: Packet Sniffing and Wireshark Fengwei Zhang Wayne State University Course: Cyber Security Practice 1 Packet Sniffer Packet sniffer is a basic tool for observing network packet exchanges in a computer Capturing (sniffs)

593 views • 14 slides
Lessons Learned and Common Findings from QA Reviews of Research Studies CLINICAL RESEARCH

Lessons Learned and Common Findings from QA Reviews of Research Studies CLINICAL RESEARCH

Lessons Learned and Common Findings from QA Reviews of Research Studies CLINICAL RESEARCH SEMINAR Gina Daniels Fiona Rice Human Research Quality Manager Human Research Quality Manager gdaniels@bu.edu fionar@bu.edu Objectives of

701 views • 52 slides
Introduction to the Operations and Management Area in the IETF Joel jaeggli

Introduction to the Operations and Management Area in the IETF Joel jaeggli

Introduction to the Operations and Management Area in the IETF Joel jaeggli &lt;joelja@bogus.com&gt; Benot Claise &lt;bclaise@cisco.com&gt; 1 Operations and Management Area Operating a network Operational feedback, best practices

988 views • 40 slides
OPAQUE: A Strong Asymmetric PAKE Protocol Secure Against Pre-Computation Attacks Stanislaw

OPAQUE: A Strong Asymmetric PAKE Protocol Secure Against Pre-Computation Attacks Stanislaw

OPAQUE: A Strong Asymmetric PAKE Protocol Secure Against Pre-Computation Attacks Stanislaw Jarecki, Hugo Krawczyk, Jiayu Xu Motivation: Password Authentication Passwords are the prevalent tool for authentication Passwords are vulnerable

481 views • 22 slides
Outline Cryptographic protocols, pt. 1 Key distribution and PKI CSci 5271 Introduction to

Outline Cryptographic protocols, pt. 1 Key distribution and PKI CSci 5271 Introduction to

Outline Cryptographic protocols, pt. 1 Key distribution and PKI CSci 5271 Introduction to Computer Security Announcements intermission Day 17: Crypto protocols and S protocols SSH Stephen McCamant University of Minnesota, Computer

338 views • 7 slides
1 The purpose of the Quality Improvement Protocol is to engage prekindergarten programs in

1 The purpose of the Quality Improvement Protocol is to engage prekindergarten programs in

1 The purpose of the Quality Improvement Protocol is to engage prekindergarten programs in creating and maintaining a high-quality program, and to be used as an ongoing tool by the Department for technical assistance and support. 2 I.

543 views • 32 slides
Ajith Suresh CrIS Lab, IISc * Indian Institute of Science (IISc), Bangalore

Ajith Suresh CrIS Lab, IISc * Indian Institute of Science (IISc), Bangalore

Ajith Suresh CrIS Lab, IISc * Indian Institute of Science (IISc), Bangalore https://www.csa.iisc.ac.in/~cris ^ Aarhus University, Denmark Outline q Privacy Preserving Machine Learning (PPML) q Secure Multi-party Computation (MPC) q Overview of

715 views • 52 slides
Application protocols David Hovemeyer 18 November 2019 David Hovemeyer Computer Systems

Application protocols David Hovemeyer 18 November 2019 David Hovemeyer Computer Systems

Application protocols David Hovemeyer 18 November 2019 David Hovemeyer Computer Systems Fundamentals: Application Protocols 18 November 2019 Application layer 1 In the network protocol stack, the application layer is at the top Consists

782 views • 46 slides

More recommend