FAULT TOLERANCE WITH PROXYSQL, MRM AND CONSUL ABOUT ME NARCS - PowerPoint PPT Presentation

Sep 23, 2023 •360 likes •543 views

FAULT TOLERANCE WITH PROXYSQL, MRM AND CONSUL ABOUT ME NARCS PILLAO Systems Engineer letgo.com | edreams.com | palominodb | emagister.com BEHIND THE SCENES 75M Downloads | 200M listings | 3B chats in 2,5 years ! AGENDA

FAULT TOLERANCE WITH PROXYSQL, MRM AND CONSUL
ABOUT ME NARCÍS PILLAO Systems Engineer letgo.com | edreams.com | palominodb | emagister.com
BEHIND THE SCENES 75M Downloads | 200M listings | 3B chats in 2,5 years !
AGENDA • Motivations • Describe the components • Architecture • Demo • Q/A
MOTIVATIONS
MOTIVATIONS • Automatic and repeatable process to reduce downtime when the master goes down • Query digest • Filter unexpected queries as ORM nightmares • Auto scalable design thanks to Consul and Puppet • Choose open source and emerging solutions
DESCRIBING THE COMPONENTS
REPLICATION MANAGER (MRM) • Open source solution (MySQL, Percona and MariaDB) • Manual or automatic failover options • Topology: Master and many slaves (ignore server option) • Multiple MRM nodes managed by arbitrator • ProxySQL integration ( just released ) • GH: https://github.com/signal18/replication-manager
PROXYSQL • Query digest • Query filtering • Query rewrite • Query routing • Maintenance operations • https://github.com/sysown/proxysql
CONSUL • Service discovery tool • Maintenance action feature • Introduce new ProxySQL nodes via its DNS • Monitoring checks
ARCHITECTURE
Replication Manager
ARCHITECTURE
ProxySQL and Consul
ARCHITECTURE
DEMO https://github.com/narcisbcn/fault_tolerant_db
QUESTIONS?
THANK YOU ! Github: narcisbcn Twitter: @narcisbcn

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Roadmap for Section 10.1 The Notion of Fault-Tolerance Fault-Tolerance Support in NTFS Volume

Unit OS10: Fault Tolerance Windows Operating System Internals - by David A. Solomon and Mark E. Russinovich with Andreas Polze Roadmap for Section 10.1 The Notion of Fault-Tolerance Fault-Tolerance Support in NTFS Volume Management - Striped

394 views • 25 slides

Fault Tolerance in Message Passing Fault Tolerance in Message Passing and in Action and in

Fault Tolerance in Message Passing Fault Tolerance in Message Passing and in Action and in Action Jack Dongarra, Innovative Computing Laboratory University of Tennessee and Computer Science and Mathematics Division Oak Ridge National

422 views • 14 slides

General Principles of Fault- Tolerance Daniel Gottesman Perimeter Institute Whats Left For

General Principles of Fault- Tolerance Daniel Gottesman Perimeter Institute Whats Left For Fault-Tolerance? Reduce the overhead (in space and in time) needed for fault-tolerance Better fault-tolerance in 1D? Better magic state

556 views • 22 slides

Rigorous fault-tolerance thresholds Ben Reichardt UC Berkeley N gate circuit 0/1 N gate

Rigorous fault-tolerance thresholds Ben Reichardt UC Berkeley N gate circuit 0/1 N gate circuit Need error 1/N 1/0 Quantum fault-tolerance problem Classical fault-tolerance: Von Neumann (1956) 0/1 Fault-tolerant, larger C High

846 views • 63 slides

Lecture 10: Fault Tolerance Fault Tolerant Concurrent Computing The main principles of fault

Lecture 10: Fault Tolerance Fault Tolerant Concurrent Computing The main principles of fault tolerant programming are: Fault Detection - Knowing that a fault exists Fault Recovery - having atomic instructions that can be rolled back in

361 views • 18 slides

CH NG 8: FAULT TOLERANCE TS. Tr n H i Anh Content 2 1. Introduction to fault

1 Tr n H i Anh Distributed System CH NG 8: FAULT TOLERANCE TS. Tr n H i Anh Content 2 1. Introduction to fault tolerance 2. Process resilience 3. Reliable client-Server Communication 4. Reliable Group Communication 5.

966 views • 50 slides

Distributed Systems 5. Fault Tolerant Systems Fault-Tolerance - 1 Lszl Bszrmnyi

Distributed Systems 5. Fault Tolerant Systems Fault-Tolerance - 1 Lszl Bszrmnyi Distributed Systems Fault tolerance A system or a component fails due to a fault Fault tolerance means that the system continues to provide its

428 views • 40 slides

CSci 5105 Introduction to Distributed Systems Fault Tolerance Last Time Replication and

CSci 5105 Introduction to Distributed Systems Fault Tolerance Last Time Replication and Consistency Today Fault tolerance Chapter 8 TVS Fault Tolerance Basics Availability short time horizon e.g down 1 msec every hour

292 views • 27 slides

Challenging Malicious Inputs with Fault Tolerance Techniques Bruno Luiz Agenda Threats

Challenging Malicious Inputs with Fault Tolerance Techniques Bruno Luiz Agenda Threats Fault Tolerance Fault Injection for Fault Tolerance Assessment Basic and classic techniques Decision Mechanisms Implementation

962 views • 61 slides

Distributed Systems (ICE 601) Fault Tolerance Dongman Lee ICU Class Overview Introduction

Distributed Systems (ICE 601) Fault Tolerance Dongman Lee ICU Class Overview Introduction Failure Model Fault Tolerance Models state machine primary-backup Distributed Systems - Fault Tolerance Introduction

146 views • 12 slides

Fibre bundle framework for unitary quantum fault tolerance Lucy Liuxuan Zhang University of

Motivations and main idea Basics of quantum fault tolerance Geometric picture of QECCs and unitary fault tolerance Conclusion Fibre bundle framework for unitary quantum fault tolerance Lucy Liuxuan Zhang University of Toronto December 18,

795 views • 16 slides

Hypervisor-Based Fault-Tolerance Thomas C. Bressoud, Isis

Hypervisor-Based Fault-Tolerance Thomas C. Bressoud, Isis Distributed Systems Fred B. Schneider, Cornell University SOSP, 1995 Problem Fault tolerance

2.97k views • 8 slides

Adaptability and Fault Tolerance Adaptability and Fault Tolerance Rog rio rio de Lemos de

Adaptability and Fault Tolerance Adaptability and Fault Tolerance Rog rio rio de Lemos de Lemos Rog University of Kent, UK University of Kent, UK Context: self-* and dependability; Focus: adaptability and fault tolerance;

640 views • 19 slides

Towards an Efficient Fault-Tolerance Scheme for GLB Claudia Fohry, Marco Bungart and Jonas Posner

GLB Fault Tolerance Scheme Experimental Results Towards an Efficient Fault-Tolerance Scheme for GLB Claudia Fohry, Marco Bungart and Jonas Posner Programming Languages / Methodologies June 14, 2015 1 / 18 GLB Fault Tolerance Scheme

744 views • 18 slides

iQIM 5 December 2011 Quantum fault tolerance Error correction and fault tolerance will be

Protected gates for superconducting qubits V( ) John Preskill with Peter Brooks and Alexei Kitaev

511 views • 33 slides

Fault Tolerance For Sparse Linear Algebra Computations Implemented In A Grid Environment

Fault Tolerance For Sparse Linear Algebra Computations Implemented In A Grid Environment Experiments with Fault Tolerant Linear Algebra Algorithms Jack Dongarra Jeffery Chen Zhiao Shi Asim YarKhan University of Tennessee Fault Tolerance:

412 views • 22 slides

and EDMs: Energy Frontier 0 Connections M.J. Ramsey-Musolf U Mass Amherst

and EDMs: Energy Frontier 0 Connections M.J. Ramsey-Musolf U Mass Amherst http://www.physics.umass.edu/acfi/ DBD Topical Collaboration Meeting, February 2017 1 Goals For This Talk Provide some context for the heavy

1.6k views • 120 slides

Today Making Simple Decisions Making Decisions Making Sequential Decisions Planning

Today Making Simple Decisions Making Decisions Making Sequential Decisions Planning under uncertainty Reinforcement Learning CSE 592 Winter 2003 Learning to act based on punishments and Henry Kautz rewards 1 2 Summary

411 views • 9 slides

Phases of dense matter in compact stars* David.Blaschke@gmail.com University of Wroclaw, Poland

Phases of dense matter in compact stars* David.Blaschke@gmail.com University of Wroclaw, Poland & JINR Dubna & MEPhI Moscow, Russia 1. Introduction: The Wroclaw Group 2. A few topics from *) arxiv:1803.01836 (D.B. & N. Chamel)

950 views • 79 slides

CommStat 3/29/18 SubStat What is it? SubStat is a mechanism to identify individuals involved

CommStat 3/29/18 SubStat What is it? SubStat is a mechanism to identify individuals involved with Opioid addicted behavior (i.e. overdoses) with the goal of the access to treatment offering. Information from SubStat is brought back to the

567 views • 54 slides

Transport Theory for EW Baryogenesis & Leptogenesis M.J. Ramsey-Musolf Wisconsin-Madison U

Transport Theory for EW Baryogenesis & Leptogenesis M.J. Ramsey-Musolf Wisconsin-Madison U Mass-Amherst Amherst Center for Fundamental Interactions Snowmass CSS, August 2013 1 Key Points Robust tests of low-scale baryogenesis

292 views • 28 slides

A Fault-Tolerant Clock Synchronization and Geometry Determination Protocol Mahyar Malekpour NASA

A Fault-Tolerant Clock Synchronization and Geometry Determination Protocol Mahyar Malekpour NASA Langley Research Center AIAA SciTech 2018, 11 January 2018 Kissimmee, Florida Mahyar Malekpour, NASA Langley Research Center, AIAA SciTech 2018 1

482 views • 19 slides

BOND: Unifying Mobile Networks with Named Data Michael Meisel Ph.D. Dissertation Defense March

BOND: Unifying Mobile Networks with Named Data Michael Meisel Ph.D. Dissertation Defense March 16, 2011 Freeform Wireless Networks Multi-hop Unpredictable mobility Can be connected or disconnected Examples: MANETs, VANETs,

1.2k views • 82 slides

Information and Communications Technology for the Developing World Michael Meisel Senior

Information and Communications Technology for the Developing World Michael Meisel Senior Software Engineer at Large michael@inveneo.org 1 What Inveneo Does Who we are Both a non-profit and a silicon valley startup Not an aid

655 views • 24 slides