ZEPPELIN High Available KV Storage Service OVERVIEW CAP High - PowerPoint PPT Presentation

May 20, 2023 •323 likes •683 views

ZEPPELIN High Available KV Storage Service OVERVIEW CAP High Available Distributed KV storage Online Searching 600,000 QPS 24 Physical Machine 3 meta server 96 node server 40+ tables Highest Table Total query

ZEPPELIN High Available KV Storage Service
OVERVIEW • CAP High Available • Distributed KV storage
• Online Searching 600,000 QPS • 24 Physical Machine • 3 meta server 96 node server • 40+ tables • Highest Table Total query 500,000,000,000 times
OVERVIEW • Interface Supported: SET, GET, DEL, MSET, MGET, INC • TTL Supported • Hashtag Supported
OVERVIEW • Client pull meta Info • Calculate partition • Find node ip in meta info • Send request to corresponding node server
NODE SERVER • Data Distribution & Replication • Thread Model • Synchronization
NODE SERVER Data Distribution & Replication •
NODE SERVER • Data Distribution & Replication • Partition • Master • Slave
NODE SERVER Thread Model •
NODE SERVER
NODE SERVER
NODE SERVER
NODE SERVER Synchronization •
NODE SERVER • Binlog • DBSync & Binlog Sync
NODE SERVER
NODE SERVER
NODE SERVER
NODE SERVER
META SERVER • Meta Info • Thread Model • Cluster Management(Migrate)
META SERVER Meta Info •
META SERVER Meta Info • • Cluster Status • Cluster Topology
META SERVER Thread Model •
META SERVER Thread Model •
META SERVER How does meta Info apply to node server? • • 1. Client connect to leader Meta Server and modify meta info. • 2. As raft exist, follower and leader could reach a consensus about meta info • 3. Node server do ping routine. Find epoch differ from Meta Server. Pull newest Meta Info and apply this modification locally.
META SERVER Migrate •
META SERVER
META SERVER • Original Meta • Send “migreate table_test 1.1.1.1:9221 4 3.3.3.3:9221” to meta server •
META SERVER • node server apply meta info changes locally • continue ping…
META SERVER
META SERVER • node server apply meta info changes locally • continue ping…
META SERVER
META SERVER • node server apply meta info changes locally • New Partition4 Master is ACTIVE
ZEPPELIN OVERVIEW • Pros & Cons • Future……
WHAT ABOUT DISTRIBUTED PIKA • Synchronization Evolution Done • Next……
THANKS •

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

70-90 1. Hard Rock The Kinks; The Who; Led Zeppelin; Black Sabbath; Deep Purple; Kiss;

History of Rock Music 70-90 1. Hard Rock The Kinks; The Who; Led Zeppelin; Black Sabbath; Deep Purple; Kiss; Queen; AC/DC; Scorpions; Rainbow. 2. Progressive Rock Space Rock (Pink Floyd); Neo-Prog(Genesis); Symphonic Rock (Sky,

601 views • 11 slides

UHABS-5 Mission Zeppelin Team Members: Likeke Aipa, Drex Arine, Andrew Bui, Karen Calaro,

UHABS-5 Mission Zeppelin Team Members: Likeke Aipa, Drex Arine, Andrew Bui, Karen Calaro, Kanekahekilinuinanaueikalani Clark, Ka Chon Liu, Cyrus Noveloso, Reagan Paz, Yun Feng Tan, Jake Torigoe, Emanuel Valdez, Jace Yamaguchi, James Yang

1.96k views • 81 slides

UHABS-5 Mission Zeppelin Team Members: Likeke Aipa, Drex Arine, Andrew Bui, Karen Calaro,

422 views • 30 slides

Duy Hai DOAN @doanduyhai Who Am I ? Duy Hai DOAN Cassandra technical advocate talks, meetups,

Apache Zeppelin, the missing GUI for your BigData eco-system Duy Hai DOAN @doanduyhai Who Am I ? Duy Hai DOAN Cassandra technical advocate talks, meetups, confs open-source devs ( Achilles , ) OSS Cassandra point of contact

336 views • 32 slides

UDLS: Friday, Sept 23, 2011 Murray Patterson What you see... typical 70s band (formed in

UDLS: Friday, Sept 23, 2011 Murray Patterson What you see... typical 70s band (formed in 1968 actually) solid, timeless rock band one of their initial inspirations: Led Zeppelin (example: What Youre Doing) well-known songs

524 views • 15 slides

Tools for large-scale collection & analysis of source code repositories OPEN SOURCE GIT

Tools for large-scale collection & analysis of source code repositories OPEN SOURCE GIT REPOSITORY COLLECTION PIPELINE Intro Alexander Bezzubov source{d} committer & PMC @ apache zeppelin startup in Madrid engineer

436 views • 25 slides

Netflix: Integrating Spark At Petabyte Scale Ashwin Shankar Cheolsoo Park Outline 1. Netflix

Netflix: Integrating Spark At Petabyte Scale Ashwin Shankar Cheolsoo Park Outline 1. Netflix big data platform 2. Spark @ Netflix 3. Multi-tenancy problems 4. Predicate pushdown 5. S3 file listing 6. S3 insert overwrite 7. Zeppelin, Ipython

526 views • 52 slides

Egorov Yuriy Alexandrovich, Petrov Zim Egorovich Knurenko Stanislav Petrovich, Nikolay Dyachkovski

Cherenkov differential detector at the Yakutsk extensive air shower array Egorov Yuriy Alexandrovich, Petrov Zim Egorovich Knurenko Stanislav Petrovich, Nikolay Dyachkovski zeppelin@ikfia.ysn.ru Cherenkov differential detector at the Yakutsk

456 views • 14 slides

Space Activities in Bulgaria Kristian Milenov Bulgarian Information Office for EO - GMES Agency

SpaceEU conference; Session - FP7 countries Space activities Space Activities in Bulgaria Kristian Milenov Bulgarian Information Office for EO - GMES Agency for Sustainable Development and Eurointegration ECOREGIONS Member of

324 views • 16 slides

DEVELOPMENT STRATEGY DEVELOPMENT & SKILLS AUDIT STUDY PRESENTATION ON KEY FINDINGS AND DRAFT

TOURISM HUMAN RESOURCE DEVELOPMENT STRATEGY DEVELOPMENT & SKILLS AUDIT STUDY PRESENTATION ON KEY FINDINGS AND DRAFT STRATEGIC THEMES FOR HUMAN RESOURCE DEVELOPMENT IN TOURISM NOVEMBER 2016 1. INTRODUCTION 2. BACKGROUND TO STUDY 3.

1.17k views • 94 slides

Imperial Space Laboratory Launch 1 st July 2013 The Company Astrium at Work Collaboration with

Imperial Space Laboratory Launch 1 st July 2013 The Company Astrium at Work Collaboration with ICL Space Innovation & Growth The Company Imperial Space Laboratory Launch July 2013 The Company Astrium at Work Collaboration with ICL

549 views • 22 slides

The Fundamentals of the Alberta Oil and Natural Gas Sectors: How Much Growth Can Be Expected? Where

The Fundamentals of the Alberta Oil and Natural Gas Sectors: How Much Growth Can Be Expected? Where Are They Heading? The Calgary Real Estate Forum October 20, 2010 Martin P. Molyneaux Managing Director, Institutional Research Outline Oil Price

745 views • 41 slides

Scalanimate An educational programming platform on top of Scastie Xavier Pantet Scalacenter

Scalanimate An educational programming platform on top of Scastie Xavier Pantet Scalacenter EPFL January 22, 2018 1/19 Outline Motivation Introduction Already existing solutions Challenges API Implementation Limitations

1.2k views • 84 slides

What We Do Search Engine Optimization Custom Website Designs That Convert More Visitors

What We Do Search Engine Optimization Custom Website Designs That Convert More Visitors Into Customers Website Content Creation, Pages, Blogs, White Papers Website A-B Split Testing Newsletter Design & Management Logo

656 views • 17 slides

May 2016 Course rse obje jecti tives ves: 1. Be able to apply an audience-centered approach

Georgia Banking School May 2016 Course rse obje jecti tives ves: 1. Be able to apply an audience-centered approach to developing a presentation 2. Be able to develop, research, and organize a presentation 3. Gain confidence in

536 views • 18 slides

Keywords can become your best marketing tool ever! How keyword use evolved and how it can be to

Keywords can become your best marketing tool ever! How keyword use evolved and how it can be to your benefit Presentation by Marcom April 16th 2020 Introduction Overview All about keywords and SEO Mark Baines (Chartered

243 views • 13 slides

Beta Presentation Document Management Using Google Cloud Platform The Capstone Experience Team

Beta Presentation Document Management Using Google Cloud Platform The Capstone Experience Team Technology Services Group Ali Alaali Luke Kline Justin Newman Rohit Sen Joe Wan Department of Computer Science and Engineering Michigan State

278 views • 9 slides

Succeeding at Search A Job Search on Indeed Finding a job on Indeed Start your search Type the

Succeeding at Search A Job Search on Indeed Finding a job on Indeed Start your search Type the words that target what youre Place screenshot here looking for. Include job title, skill or company name. Include city, state or zip code.

296 views • 18 slides

Pr e se nte d By: T odd Paton T ra ditio na l Adve rtise me nt Billb o a rds, T V Co mme

Pr e se nte d By: T odd Paton T ra ditio na l Adve rtise me nt Billb o a rds, T V Co mme rc ia ls, Co upo ns, Ye llo w Pa g e s T he Pro b le m: NOT T ARGE T E D T o da y s Adve rtise me nt My GOAL o ve r the ne xt 45

584 views • 19 slides

Fundraising with Digital Marketing Nick Day Digital & Direct Marketing Practitioner |

Fundraising with Digital Marketing Nick Day Digital & Direct Marketing Practitioner | Consultant | Trainer Well cover 1. Digital Marketing Planning Considerations 2. Using the Digital Marketing Toolkit: Search Marketing

1.38k views • 123 slides

SEkFER S@IFE R www . safer - jobs . com Looking Ahead a t Aims . To increase fraud prevention

SEkFER S@IFE R www . safer - jobs . com Looking Ahead a t Aims . To increase fraud prevention work To increase job board activit y To increase volume of fraud intelli g enc e To improve SAFE Rjobs visibilit y S@IFE R www . safer -

564 views • 21 slides

Helping brands get discovered on Search Engines We believe in continuous improvement How does SEO

Helping brands get discovered on Search Engines We believe in continuous improvement How does SEO help Each web page must be prepared with the right words and meta fields for Googles algorithm to identify that web page as the most relevant

186 views • 7 slides

Leon Derczynski - Supervised by Dr Amanda Sharkey - 2006 This abstract relates to a document

Leon Derczynski - Supervised by Dr Amanda Sharkey - 2006 This abstract relates to a document about low-price movies This document contains the words cheap film, but is not useful - Little human feedback is gathered on what makes a

194 views • 8 slides

SKETCHNOTE SKETCH NOTES 101 SKETCHNOTES ARE RICH VISUAL NOTES CREATED FROM A MIX OF

HOW TO USE & CREATE SKETCHNOTE SKETCH NOTES 101 SKETCHNOTES ARE RICH VISUAL NOTES CREATED FROM A MIX OF HANDWRITING, DRAWINGS, HAND-DRAWN TYPOGRAPHY, SHAPES, AND VISUAL ELEMENTS LIKE ARROWS, BOXES, AND LINES - Mike Rohde SKETCH

361 views • 12 slides

ZEPPELIN High Available KV Storage Service OVERVIEW CAP High - PowerPoint PPT Presentation

ZEPPELIN High Available KV Storage Service OVERVIEW CAP High Available Distributed KV storage Online Searching 600,000 QPS 24 Physical Machine 3 meta server 96 node server 40+ tables Highest Table Total query

70-90 1. Hard Rock The Kinks; The Who; Led Zeppelin; Black Sabbath; Deep Purple; Kiss;

UHABS-5 Mission Zeppelin Team Members: Likeke Aipa, Drex Arine, Andrew Bui, Karen Calaro,

UHABS-5 Mission Zeppelin Team Members: Likeke Aipa, Drex Arine, Andrew Bui, Karen Calaro,

Duy Hai DOAN @doanduyhai Who Am I ? Duy Hai DOAN Cassandra technical advocate talks, meetups,

UDLS: Friday, Sept 23, 2011 Murray Patterson What you see... typical 70s band (formed in

Tools for large-scale collection &amp; analysis of source code repositories OPEN SOURCE GIT

Netflix: Integrating Spark At Petabyte Scale Ashwin Shankar Cheolsoo Park Outline 1. Netflix

Egorov Yuriy Alexandrovich, Petrov Zim Egorovich Knurenko Stanislav Petrovich, Nikolay Dyachkovski

Space Activities in Bulgaria Kristian Milenov Bulgarian Information Office for EO - GMES Agency

DEVELOPMENT STRATEGY DEVELOPMENT &amp; SKILLS AUDIT STUDY PRESENTATION ON KEY FINDINGS AND DRAFT

Imperial Space Laboratory Launch 1 st July 2013 The Company Astrium at Work Collaboration with

The Fundamentals of the Alberta Oil and Natural Gas Sectors: How Much Growth Can Be Expected? Where

Scalanimate An educational programming platform on top of Scastie Xavier Pantet Scalacenter

What We Do Search Engine Optimization Custom Website Designs That Convert More Visitors

May 2016 Course rse obje jecti tives ves: 1. Be able to apply an audience-centered approach

Keywords can become your best marketing tool ever! How keyword use evolved and how it can be to

Beta Presentation Document Management Using Google Cloud Platform The Capstone Experience Team

Succeeding at Search A Job Search on Indeed Finding a job on Indeed Start your search Type the

Pr e se nte d By: T odd Paton T ra ditio na l Adve rtise me nt Billb o a rds, T V Co mme

Fundraising with Digital Marketing Nick Day Digital &amp; Direct Marketing Practitioner |

SEkFER S@IFE R www . safer - jobs . com Looking Ahead a t Aims . To increase fraud prevention

Helping brands get discovered on Search Engines We believe in continuous improvement How does SEO

Leon Derczynski - Supervised by Dr Amanda Sharkey - 2006 This abstract relates to a document

SKETCHNOTE SKETCH NOTES 101 SKETCHNOTES ARE RICH VISUAL NOTES CREATED FROM A MIX OF

Tools for large-scale collection & analysis of source code repositories OPEN SOURCE GIT

DEVELOPMENT STRATEGY DEVELOPMENT & SKILLS AUDIT STUDY PRESENTATION ON KEY FINDINGS AND DRAFT

Fundraising with Digital Marketing Nick Day Digital & Direct Marketing Practitioner |