SLIDE 1
Peak Performance
Remote Memory Revisited
Hannes Mühleisen, Romulo Goncalves and Martin Kersten
Database Scalability
Scale-Up (Big Iron) Scale-Out (Many Boxes) “Cheating” Full Virtualization Storage Clusters Remote Memory
2
Peak Performance Remote Memory Revisited Hannes Mhleisen, Romulo - - PowerPoint PPT Presentation
Peak Performance Remote Memory Revisited Hannes Mhleisen, Romulo - - PowerPoint PPT Presentation
Peak Performance Remote Memory Revisited Hannes Mhleisen, Romulo Goncalves and Martin Kersten Database Scalability Scale-Up Scale-Out (Big Iron) (Many Boxes) Cheating Full Virtualization Storage Clusters Remote Memory 2 Why
SLIDE 2
SLIDE 3
Why more memory?
- Memory is a critical resource, especially in
OLAP use cases
- Hash tables, intermediate results, ...
- OS overcommits, leads to thrashing
3
SLIDE 4
- RAX XT
RAX−SX4 RAX−XS3 RAX−XS4 T1650 T3600 T5600
30 40 50 60 70 500 1000 1500 2000
Main Memory (GB) US$/GB
4
SLIDE 5
Remote memory then
- Hack Kernel to page out to remote
machines? [Tell et al. 2013]
- Store swapfile to remote file system?
- But DBs like to avoid swap anyway, so...
- Store DB temporary files on remote
system!
5
SLIDE 6
New Toys
6
SLIDE 7
Network Adapter Memory CPU Network Adapter Memory CPU Network Adapter
Memory
CPU Network Adapter
Memory
CPU
The way it was RDMA Many-Copy Zero-Copy
7
SLIDE 8
8
SLIDE 9
Experimental Setup
- 14 Linux COTS Boxes
- 16 GB RAM
- InfiniBand QDR
- 182 GB Memory total (and usable!)
9
SLIDE 10
Throughput
1 2 3 HDD SSD iSCSI/ETH iSCSI/EoIB iSCSI/RDMA NFS/ETH NFS/EoIB NFS/RDMA NFS/RDMA/RAID0 NFS/RDMA/RAID5
Throughput (GB/s)
Write Read
10
SLIDE 11
Latency
200 400 600 SSD RAM iSCSI/ETH iSCSI/EoIB iSCSI/RDMA NFS/ETH NFS/EoIB NFS/RDMA NFS/RDMA/RAID0 NFS/RDMA/RAID5
Latency (µs)
11
SLIDE 12
OLAP DB (TPC-H)
- TPC-H: benchmark for relational databases
focused on analytics (OLAP)
- Queries tend to have large intermediate
results (SF=100):
Query Read (GB) Write (GB)
1 14 50 18 5 28 21 7 9 3 6 6 13 2 7
12
SLIDE 13
Example: Query 18
13
0.0 0.5 1.0 1.5 2.0 30 60 90
Time (s) Traffic (GB/s)
Direction Read Write
SLIDE 14
TPC-H Experiment
- Single node runs MonetDB with TPC-H
database (SF=100)
- Gets remote memory from the 14
memory providers
- DB temporary partition resides either on
disk or in remote memory
- Hot runs, 5 repetitions per query and setup
14
SLIDE 15
TPC-H 100 Results
10 1000 q01 q02 q03 q04 q05 q06 q07 q08 q09 q10 q11 q12 q13 q14 q15 q16 q17 q18 q19 q20 q21 q22
Query Average Execution Time (s)
Experiment HDD RRAM
15
SLIDE 16
Summary
- Remote Memory is interesting (...)
- Lightweight technique
- RDMA allows for remote memory to make
sense from a technical perspective
- OLAP database scenarios can benefit from
this
- Open issue: Hardware pricing/TCO
16
SLIDE 17
Thank You!
Questions?
http://is.gd/remotemem