Vitess@QCon Sugu Sougoumarane, Co-creator CTO, PlanetScale - - PowerPoint PPT Presentation

▶

Nov 21, 2023 469 likes •757 views

Vitess@QCon Sugu Sougoumarane, Co-creator CTO, PlanetScale @ssougou ssougou.blogspot.com Database trends Transactional data explosion Move to cloud Transactional data options Single instance RDBMS NoSQL App Sharded

SLIDE 1

Sugu Sougoumarane, Co-creator CTO, PlanetScale @ssougou ssougou.blogspot.com

Vitess@QCon

SLIDE 2

Database trends

Transactional data explosion
Move to cloud

SLIDE 3

Transactional data options

Single instance RDBMS
NoSQL
App Sharded RDBMS
NewSQL

SLIDE 4

Cloud

Instances go down all the time

○ Good master promotion story

Durability through replication

○ Semi-sync

Small instances
Discovery mechanism

SLIDE 5

Vitess

Leverage MySQL
Massive Scale
Battle-hardened
Runs “anywhere”

SLIDE 6

pxhere CC0

Vitess stats

Started 2010

6,000+ stars 15,000+ commits

Marquee adopters

100+ contributors 500+ Slack members

SLIDE 7

Key adopters

20% Migrated to Vitess 10,000+ Tablets on Kubernetes Cash App fully runs on Vitess

SLIDE 8

The Community

In production Evaluating

Quiz of Kings

SLIDE 9

MySQL Community Award

“Application of the year 2018”

SLIDE 10

Growing with Vitess

Vitess app server app server app server MainDB Connection pooling Deadlines Hot row protection Row count limit Blacklisting

SLIDE 11

Growing with Vitess

Vitess app server app server app server Master Replica Replica routing Load balancing Master promotion with Orchestrator

SLIDE 12

Growing with Vitess

Vitess app server app server app server DB2 Replica DB1 Replica Unified View

SLIDE 13

Growing with Vitess

Vitess app server app server app server DB2 Replica DB1:S1 Replica DB1:S2 Replica DB1:S3 Replica Sharding agnostic

SLIDE 14

Growing with Vitess

Vitess app server app server app server DB2 Replica DB1:S1 Replica DB1:S2 Replica DB1:S3 Replica Replica Replica Replica Replica Replica Replica Replica Replica Zone 1 Zone 2 Zone 3 Multi-zone Isolation

SLIDE 15

ALL YOUR DATABASE

SLIDE 16

Architecture

shard n shard 1

vtgate app server app server app server mysqld vttablet master vtgate big data Vitess lockserver mysqld vttablet mysqld vttablet replicas mysqld vttablet big data replicas mysqld vttablet vtctld App

SLIDE 17

Sharding concepts

VTGate

VSchema
Primary Vindex
Secondary Vindex

○ Unique ○ Non-unique

Shared Vindex
Sequences

Database

Schema
Primary Key
Secondary Key

○ Unique ○ Non-unique

Foreign Key
Auto-increment

SLIDE 18

Resharding

No downtime
Vertical and Horizontal
Splits and Merges
Checks and balances

SLIDE 19

Demo

user id name

rder

id uid pid mname merchant name lookup unsharded product id user sharded merchant sharded name_user_idx name uid

SLIDE 20

Questions?

Sign into slack channel at vitess.io
Try bringing up a cluster on your laptop or kubernetes
GH: http://github.com/vitessio/vitess

SLIDE 21

Appendix

SLIDE 22

Demo

user id name

rder

id uid pid mname merchant name lookup unsharded product id user sharded merchant sharded name_user_idx name uid product id

rder

id uid pid mname product id

select * from product select name, id as uid from user select * from merchant where in_keyrange( mname, ‘unicode_loose_md5’, ‘-80’)

SLIDE 23

DB Protection

Connection pools
Hot rows
Damage control

○ Row-count limit ○ Time limit ○ Transaction time limits

Blacklisting

SLIDE 24

Observability

Stats end points
Per-query stats
Querylog

SLIDE 25

Life of a Query: VTGate

Parse PlanBuilder vschema

AST

Engine Part 1

Plan

Gateway topology vttablets

SQL SQL

shardmap Vindexes

vindex Keyspace ids

Engine Part 2

shards

ScatterConn health

SLIDE 26

Life of a Query: VTTablet

Parse PlanBuilder schema

AST

ACL

Plan SQL PKs

Consolidator Pool MySQL Deadliner Transaction Watcher BinlogPlayer Filtered Replication

SLIDE 27

Roadmap

Materialized views
Real-time rollups
VSchema-based resharding
Full cross-shard SQL support