ERMIA: Fast Memory-Optimized Database System for Heterogeneous - - PowerPoint PPT Presentation

ERMIA: Fast Memory-Optimized Database System for Heterogeneous Workloads

Kangnyeon Kim Tianzheng Wang Ryan Johnson Ippokratis Pandis

Heterogeneous is the new OLTP

Convenient/traditional Emerging workloads

W R R R R W R W R R R W RR RRR RRRR R RR

• Short write-intensive
• Short read-only
• Short write-intensive
• Longer read-mostly

Worst case for current systems, esp. CC

W W

2

Transaction 1

Modern OCC 101

• Decentralized optimistic

concurrency control (OCC)

• Silo [Tu et al. ’13]
• FOEDUS [Kimura ’15]
• No locking for reads
• Locally-cached writes
• Commit phase
• 1. Lock writes
• 2. Verify reads
• 3. Apply writes

Read set

A B

Write set

D

Database

A B’

Transaction 2

Write set

B’ D

Read set

A Clobbered abort

3

1 2 3 4 5 6 ERMIA Silo-OCC Ideal ERMIA Silo-OCC Aggregate commit ratio NewOrder Payment OrderStatus Delivery StockLevel Read-mostly

OCC: Not a panacea

Starved

4

TPC-C TPC-C + long read-mostly tx

Fair

0% 20% 40% 60% 80% 100% Silo-OCC ERMIA-SSN Silo-OCC ERMIA-SSN Commit NewOrder Commit Payment Commit Delivery Commit OrderStatus Commit StockLevel Commit Read-mostly Abort NewOrder Abort Payment Abort Delivery Abort OrderStatus Abort StockLevel Abort Read-mostly

OCC + Read-mostly = Wasted cycles

5

Read-mostly tx: fair and robust CC needed

TPC-C TPC-C with long read-mostly

ERMIA comes to the rescue

Lock- free Index

Snapshot Isolation

Epoch-based Resource Management

Serial Safety Net (SSN)

Serializable Read-friendly, fair, robust, timely abort

Logical layer: robust && fair CC Scalable physical layer

In-memory data

Database == Scalable centralized log

6

Scalable Centralized Redo Logging

7

Log Private log buffer

• Async. filling

Upon commit, LSN = XADD(current LSN, log size) Current LSN Current LSN

The only global communication

XADD  Scalability + Global Ordering

Durable LSN

Latch-free Indirection Array

• Object IDs rather than pointers in leaf nodes
• No update propagation to secondary indexes

8 Lock-free Index

OID Address 1 2 3 … … … OIDs in leaf nodes

Versions in durable log

V1 V2 V3

CAS to install new version Fast recovery: fetch header only

The Serial Safety Net

9 T4 T1 π(T4) T2 T3

commit time dependency order

T4’s earliest successor c(T3): commit time Forbid $P  T : π(T) ≤ c(P) ≤ c(T)

“exclusion window” of T

The Serial Safety Net: Efficient Concurrency Control on Modern Hardware

• T. Wang, R. Johnson, A. Fekete, I. Pandis, DaMoN ’15

T1, T2, T3: predecessors of T4

T3 T4 T1 T2 π(T2)

Visualizing SSN

10

Exclusion window satisfied

T4 T1 π(T4) T2 T3

commit time dependency order

π(T2) T5 T1 T4 T3 T2 (T1) T2 T1 ?? π(T2)

Exclusion window violation

Evaluation

• 24-core, quad-socket Xeon E7-4807
• Everything in main memory
• OCC vs. ERMIA-SI vs. ERMIA-SSN
• “Convenient”/traditional OLTP
• Original TPC-C
• Original TPC-E
• Heterogeneous OLTP
• TPC-C with read-mostly transaction (TPC-C-hybrid)
• TPC-E with read-mostly transaction (TPC-E-hybrid)

11

“Convenient”

12

TPC-C TPC-E

Higher is better

Comparable performance to OCC

Robust heterogeneous performance

13

TPC-C-hybrid TPC-E-hybrid

Higher is better

(Much) better overall throughput

Robust heterogeneous performance

14

Higher is better

(Much) lower abort rate for read-mostly tx

Lower is better

Conclusions

• Heterogeneous OLTP: fair && robust CC needed
• ERMIA = Snapshot Isolation + SSN

+ Lock/latch-free physical layer

• Also performs well for traditional OLTP

15

Find out more in our paper and code repo! https://github.com/ermia-db Read-friendly Fair, robust, serializable Scalable Thank you!