SLIDE 1
PELOTON
THE SELF-DRIVING DBMS
PELOTON THE SELF-DRIVING DBMS 2008 5,000 txn/sec H-Store: A - - PowerPoint PPT Presentation
PELOTON THE SELF-DRIVING DBMS 2008 5,000 txn/sec H-Store: A - - PowerPoint PPT Presentation
PELOTON THE SELF-DRIVING DBMS 2008 5,000 txn/sec H-Store: A High-Performance, Distributed Main Memory Transaction Processing System VLDB 2008 2008 5,000 txn/sec 2010 11,000 txn/sec On Predictive Modeling for Optimizing Transaction
SLIDE 2
SLIDE 3
2008
5,000 txn/sec
H-Store: A High-Performance, Distributed Main Memory Transaction Processing System VLDB 2008 SLIDE 4
2008
5,000 txn/sec
2010
11,000 txn/sec
On Predictive Modeling for Optimizing Transaction Execution in Parallel OLTP Systems VLDB 2011 SLIDE 5
2008
5,000 txn/sec
2010
11,000 txn/sec
2012
50,000 txn/sec
Skew-Aware Automatic Database Partitioning in Shared-Nothing, Parallel OLTP Systems SIGMOD 2012 SLIDE 6
2008
5,000 txn/sec
2010
11,000 txn/sec
2012
50,000 txn/sec
2015
4,000,000 txn/sec
TicToc: Time Traveling Optimistic Concurrency Control SIGMOD 2016 SLIDE 7
SLIDE 8
ONLY MAXIMIZING OLTP Throughput Leads to an Unfulfilling life.
SLIDE 9
SLIDE 10
SLIDE 11
AVERAGE SALARY FOR Database ADMINS IN 2015 was $8 $81, 1,710.
Source: Bureau of Labor Statistics
SLIDE 12
Self-Driving
A DBMS THAT can configure, tune, and optimize itself without any human intervention.
SLIDE 13
YES
Database Design Data Placement Query Optimization Knob Configuration Back-up & Recovery Provisioning
NO
Security & ACLS Data Integration UNPLANNED HALTS Version Control
SLIDE 14
What’s New?
Previous EFFORTS are reactive & human-driven. A self-driving Dbms has to be predictive.
SLIDE 15
Why Now?
Recent advancements in hardware and deep neural networks make autonomous
- peration now possible.
SLIDE 16
SLIDE 17
In-MEMORY OLTP+OLAP Autonomous LLVM EXEC
SLIDE 18
The Brain
Integrated Deep Learning FRAMEWORK to model, predict, and optimize HTAP Database workloads.
Self-Driving Database Management Systems CIDR 2017 SLIDE 19
Workload Categorization
...
2 4 H r s 7 d a y s 4 d a y s
Unsupervised SLIDE 20
Workload Categorization Workload Forecasting
...
Long short-term Memory2 4 H r s 7 d a y s 4 d a y s
Unsupervised SLIDE 21
Workload Categorization Workload Forecasting Optimization Planning
...
Unsupervised SLIDE 22
Workload Categorization Workload Forecasting Optimization Planning
...
Catalog Benefit Unsupervised SLIDE 23
Evaluation
Synthetic workload based
- n Reddit Traffic Data.
Forecast with Tensorflow. Adaptive storage.
SLIDE 24
1min intervals Error Rate: 14.7% CPU Training: 25min Probe: 2MS Update: 5ms Size: 2MB 1hr intervals Error Rate: 17.9% CPU Training: 18min Probe: 2ms Update: 5MS Size: 2MB
SLIDE 25
Adaptive Storage
Change the layout of data
- ver time based on how it
is accessed.
Bridging the Archipelago Between Row-Stores and Column-Stores for Hybrid Workloads SIGMOD 2016 SLIDE 26 A B C D
SELECT AVG(B) FROM myTable WHERE C < “yyy” UPDATE myTable SET A = 123, B = 456, C = 789 WHERE D = “xxx”
Cold Hot
SLIDE 27 A B C D A B C D
SELECT AVG(B) FROM myTable WHERE C < “yyy” UPDATE myTable SET A = 123, B = 456, C = 789 WHERE D = “xxx”
A B C D
Cold Hot
SLIDE 28
400 800 1200 1600
Row Layout Column Layout Adaptive Layout
Sep-15
Scan Insert Scan Insert Scan Insert Scan Insert Scan Insert Scan InsertExecution Time (ms)
Sep-16 Sep-17 Sep-18 Sep-19 Sep-20
SLIDE 29
Current Status
Single-node Only. Looking for real-world Deployments to test. Apache 2.0 License
SLIDE 30
http:/ /pelotondb.org
SLIDE 31
SLIDE 32