TiDB: Overview, New Features, Multi-Cloud Shen Li, Kevin Xu - PowerPoint PPT Presentation

TiDB: Overview, New Features, Multi-Cloud Shen Li, Kevin Xu PingCAP

What is Ti(tanium) DB? • NewSQL (RDBMS at Scale) • MySQL Compatible (5.7) • Hybrid OLTP + OLAP Workloads • Multi-Cloud � 2

Product: the TiDB Platform • TiDB Platform: • TiDB (stateless SQL layer) • TiKV (persistent key-value store w/ RocksDB) • TiSpark (Apache Spark plug-in to TiKV) • Open Sourced from Day 1 • GA 2.0: April 2018 � 3

Community • Stars: • TiDB: 15,600+ • TiKV: 3,800+ • Contributors: • TiDB: 210+ • TiKV: 105+ � 4

Recent News � 5

TiDB Overview

Platform Architecture Monitoring & Observability: Prometheus + Grafana � 7

TiKV: the Storage Foundation � 8

TiDB: the (My)SQL Layer � 9

300+ Companies in Production � 10

MySQL Compatible Tools and Migration

Tools • TiDB-DM (Data Migration) • TiDB-Lightning • TiDB-Binlog � 12

TiDB-DM • Why? To support migrating data from MySQL/MariaDB to TiDB • Full data migration + binlog replication • Features: • Data migration job management • Monitoring system • Sharded table merge migration • Data transformation � 13

DM Architecture � 14

Sharded Table Merge � 15

Data Transformation � 16

TiDB-Lightning • Why? Enables fast , full import of SQL dump -> TiDB cluster • Fast: • Can import > 100 GB / hour on production grade hardware • Full Import: • Import the entire database once from scratch; not incremental import • Suitable for migrating an existing database to a new TiDB cluster • TiDB Cluster: • Lightning interacts with TiDB’s internal structures • Only works with TiDB cluster � 17

TiDB-Lightning Architecture � 18

TiDB-Binlog • Why? Provides real-time data synchronization and backup • Synchronizing downstream data platform in real-time to: • Hot backup • MySQL slave • OLAP data warehouses • Writing incremental backup files • Event subscription & trigger � 19

TiDB-Binlog Architecture � 20

Migration: in and out of TiDB � 21

Sneak Peek of TiDB 2.1

TiDB 2.1 - SQL Layer Smarter SQL optimizer Faster online DDL • More efficient plan • Run DDL concurrently • Dynamic stats update based on • Faster “Add Index” operation query feedback Improved debugging & diagnosis Faster aggregation operator • More readable EXPLAIN result • New aggregation framework • “Explain Analyze” statement • Multi-thread hash aggregation • Expose cluster information through operator HTTP API 50% faster OLTP point query � 23

TiDB 2.1 - Storage Engine (aka TiKV) Raft learner • Improve cross-DC data availability Hotspot scheduling • Automatically distribute hotspots to more idle instances Batch region-splits • Avoid large region formation under heavy write workloads 2X read performance improvement � 24

1.0 vs 2.0 on TPC-H Scale 50 � 25

2.0 vs 2.1 on TPC-H Scale 50 � 26

2.1 vs 2.0 on Sysbench Point Select � 27

TiDB Cloud

TiDB As A Service • Fully Managed by PingCAP • Upgrades, Patching, Online DDL with No Downtime • Backup and Restore • Multi-Cloud Deployment https://www.pingcap.com/tidb-cloud/ � 29

TiDB Operator � 30

GCP Marketplace � 31

See it to believe it • PingCAP Booth #6 • Demo by Greg Weber TODAY at 1:20PM � 32

Oh…one more thing, TiDB Academy https://www.pingcap.com/tidb-academy/ � 33

Thank you! @PingCAP, @shenli3514, @kevinsxu