H-Store: A Specialized Architecture for High-throughput OLTP - - PowerPoint PPT Presentation

▶

Nov 21, 2023 307 likes •509 views

H-Store: A Specialized Architecture for High-throughput OLTP Applications Evan Jones (MIT) Andrew Pavlo (Brown) 13 th Intl. Workshop on High Performance Transaction Systems October 26, 2009 October 26, 2009 Intel Xeon E5540 (Nehalem/Core i7)

SLIDE 1

H-Store:

A Specialized Architecture for High-throughput OLTP Applications

Evan Jones (MIT) Andrew Pavlo (Brown) 13th Intl. Workshop on High Performance Transaction Systems October 26, 2009

SLIDE 2

Intel Xeon E5540 (Nehalem/Core i7)

October 26, 2009

Source: Intel 64 and IA-32 Architectures Optimization Reference Manual

SLIDE 3

Distributed Clusters

October 26, 2009

SLIDE 4

Scaling OLTP on Multi-Core?

Use a distributed shared-nothing design

October 26, 2009

SLIDE 5

How to Make a Faster OLTP DBMS

Main memory storage
Replication for durability
Explicitly partition the data
Specialized concurrency control
Stored procedures only
Single partition: execute one transaction at time
Multiple partitions: supported but slow

October 26, 2009

SLIDE 6

Source: S. Harizopoulos, D. J. Abadi, S. Madden, M. Stonebraker, “OLTP Under the Looking Glass”, SIGMOD 2008.

OLTP: Where does the time go?

October 26, 2009

SLIDE 7

Users Rely on Partitioning

October 26, 2009

Source: R. Shoup, D. Pritchett, “The eBay Architecture,” SD Forum, Nov. 2006.

SLIDE 8

What about multi-core?

Traditional approach:
One database process
Thread per connection
Shared-memory, locks and latches
H-Store approach:
Thread per partition
Distributed transactions

October 26, 2009

SLIDE 9

Example Microbenchmark

One table per client

Table(id INTEGER, counter INTEGER)

Each client executes the following query:

UPDATE Table SET counter = counter + 1 WHERE id = 0;

Add clients to find maximum throughput
Data on RAM disk

October 26, 2009

SLIDE 10

Experimental Configuration

October 26, 2009

Threads Partitions

SLIDE 11

Partitions versus Threads

2 4 6 8 10 12 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Relative Speed Up CPUs

October 26, 2009

Ideal Partitions Threads

SLIDE 12

Scalability Analysis

Partitions scale better than threads.

Threads: contention for shared resources [1]
Partitions: memory bottleneck causes sublinear

scaling H-Store: Not just for distributed shared-nothing clusters

October 26, 2009

[1] R. Johnson et al., "Shore-MT: A Scalable Storage Manager for the Multicore Era," EDBT 2009.

SLIDE 13

Multi-core Design Problem

How to automatically create a data placement

scheme to improve multi-core throughput?

Data Partitioning:
Maximize the number of single-partition transactions.
Data Placement:
Maximize the number of single-node transactions.

October 26, 2009

SLIDE 14

Database Partitioning

Select partitioning keys and construct schema tree.

October 26, 2009

DISTRICT CUSTOMER ORDER_ITEM ITEM STOCK WAREHOUSE ORDERS DISTRICT CUSTOMER ORDER_ITEM STOCK ORDERS ITEM

Replicated

WAREHOUSE

TPC-C Schema Schema Tree

SLIDE 15

ITEM ITEMj ITEM ITEM ITEM

Database Partitioning

Combine table fragments into partitions.

October 26, 2009

P2 P4 DISTRICT CUSTOMER ORDER_ITEM STOCK ORDERS

Replicated

WAREHOUSE P1 P1 P1 P1 P1 P1 P2 P2 P2 P2 P2 P2 P3 P3 P3 P3 P3 P3 P4 P4 P4 P4 P4 P4 P5 P5 P5 P5 P5 P5 P5 P3 P1

ITEM ITEM ITEM ITEM ITEM

Partitions

ITEM

Schema Tree

SLIDE 16

Partition Affinity

Core1 Core2 HT2 HT1 Core1 Core2 HT2 HT1

Data Placement

Assign partitions to cores on each node.

October 26, 2009

ITEM

Partitions Cluster Node

ITEM

Node 1 Node n

P2 P3 P4 P5

SLIDE 17

H-Store’s Future

New Name. New Company.
Six full-time developers.
Open-source project (GPL)
Beta by end of 2009
Multiple deployments in financial service areas.

October 26, 2009

SLIDE 18

More Information

H-Store Info + Papers:
http://db.cs.yale.edu/hstore/
VoltDB Project Information:
http://www.voltdb.org/

October 26, 2009