ZEPPELIN High Available KV Storage Service OVERVIEW CAP High - - PowerPoint PPT Presentation

▶
zeppelin
SMART_READER_LITE
LIVE PREVIEW

ZEPPELIN High Available KV Storage Service OVERVIEW CAP High - - PowerPoint PPT Presentation

May 20, 2023 323 likes •684 views

ZEPPELIN High Available KV Storage Service OVERVIEW CAP High Available Distributed KV storage Online Searching 600,000 QPS 24 Physical Machine 3 meta server 96 node server 40+ tables Highest Table Total query

slide-1
SLIDE 1

ZEPPELIN

High Available KV Storage Service

slide-2
SLIDE 2

OVERVIEW

  • CAP High Available
  • Distributed KV storage
slide-3
SLIDE 3
  • Online Searching 600,000 QPS
  • 24 Physical Machine
  • 3 meta server 96 node server
  • 40+ tables
  • Highest Table Total query 500,000,000,000 times
slide-4
SLIDE 4
  • Interface Supported: SET, GET, DEL, MSET, MGET,

INC

  • TTL Supported
  • Hashtag Supported

OVERVIEW

slide-5
SLIDE 5

OVERVIEW

  • Client pull meta Info
  • Calculate partition
  • Find node ip in meta

info

  • Send request to

corresponding node server

slide-6
SLIDE 6

NODE SERVER

  • Data Distribution & Replication
  • Thread Model
  • Synchronization
slide-7
SLIDE 7

NODE SERVER

  • Data Distribution & Replication
slide-8
SLIDE 8

NODE SERVER

  • Data Distribution & Replication
  • Partition
  • Master
  • Slave
slide-9
SLIDE 9

NODE SERVER

  • Thread Model
slide-10
SLIDE 10

NODE SERVER

slide-11
SLIDE 11

NODE SERVER

slide-12
SLIDE 12

NODE SERVER

slide-13
SLIDE 13

NODE SERVER

  • Synchronization
slide-14
SLIDE 14

NODE SERVER

  • DBSync & Binlog Sync
  • Binlog
slide-15
SLIDE 15

NODE SERVER

slide-16
SLIDE 16

NODE SERVER

slide-17
SLIDE 17

NODE SERVER

slide-18
SLIDE 18

NODE SERVER

slide-19
SLIDE 19

META SERVER

  • Meta Info
  • Thread Model
  • Cluster Management(Migrate)
slide-20
SLIDE 20

META SERVER

  • Meta Info
slide-21
SLIDE 21

META SERVER

  • Cluster Status
  • Cluster

Topology

  • Meta Info
slide-22
SLIDE 22

META SERVER

  • Thread Model
slide-23
SLIDE 23

META SERVER

  • Thread Model
slide-24
SLIDE 24

META SERVER

  • How does meta Info apply to node server?
  • 1. Client connect to leader Meta Server and modify meta info.
  • 2. As raft exist, follower and leader could reach a consensus

about meta info

  • 3. Node server do ping routine. Find epoch differ from Meta
  • Server. Pull newest Meta Info and apply this modification

locally.

slide-25
SLIDE 25

META SERVER

  • Migrate
slide-26
SLIDE 26

META SERVER

slide-27
SLIDE 27

META SERVER

  • Original Meta
  • Send “migreate table_test 1.1.1.1:9221 4 3.3.3.3:9221” to

meta server

slide-28
SLIDE 28

META SERVER

  • node server apply meta info

changes locally

  • continue ping…
slide-29
SLIDE 29

META SERVER

slide-30
SLIDE 30

META SERVER

  • node server apply meta info

changes locally

  • continue ping…
slide-31
SLIDE 31

META SERVER

slide-32
SLIDE 32

META SERVER

  • node server apply meta info

changes locally

  • New Partition4 Master is ACTIVE
slide-33
SLIDE 33

ZEPPELIN OVERVIEW

  • Pros & Cons
  • Future……
slide-34
SLIDE 34
  • Synchronization Evolution Done
  • Next……

WHAT ABOUT DISTRIBUTED PIKA

slide-35
SLIDE 35
  • THANKS