Peak Performance Remote Memory Revisited Hannes Mhleisen, Romulo - PowerPoint PPT Presentation

Peak Performance Remote Memory Revisited Hannes Mühleisen, Romulo Goncalves and Martin Kersten

Database Scalability Scale-Up Scale-Out (Big Iron) (Many Boxes) “Cheating” Full Virtualization Storage Clusters Remote Memory 2

Why more memory? • Memory is a critical resource, especially in OLAP use cases • Hash tables, intermediate results, ... • OS overcommits, leads to thrashing 3

70 RAX XT ● 60 US$/GB 50 T1650 ● 40 T5600 ● ● T3600 ● RAX − SX4 ● ● 30 RAX − XS3 ● ● RAX − XS4 ● 0 500 1000 1500 2000 Main Memory (GB) 4

Remote memory then • Hack Kernel to page out to remote machines? [Tell et al. 2013] • Store swapfile to remote file system? • But DBs like to avoid swap anyway, so... • Store DB temporary files on remote system! 5

New Toys 6

The way it was RDMA CPU CPU Memory Memory Network Adapter Network Adapter Network Adapter Network Adapter Memory CPU Memory CPU Many-Copy Zero-Copy 7

8

Experimental Setup • 14 Linux COTS Boxes • 16 GB RAM • InfiniBand QDR • 182 GB Memory total (and usable!) 9

Throughput (GB/s) 0 1 2 3 HDD SSD Throughput iSCSI/ETH iSCSI/EoIB iSCSI/RDMA 10 NFS/ETH NFS/EoIB NFS/RDMA NFS/RDMA/RAID0 NFS/RDMA/RAID5 Read Write

Latency (µs) 200 400 600 0 SSD RAM iSCSI/ETH Latency iSCSI/EoIB iSCSI/RDMA 11 NFS/ETH NFS/EoIB NFS/RDMA NFS/RDMA/RAID0 NFS/RDMA/RAID5

OLAP DB (TPC-H) • TPC-H: benchmark for relational databases focused on analytics (OLAP) • Queries tend to have large intermediate results (SF=100): Query Read (GB) Write (GB) 1 14 50 18 5 28 21 7 9 3 6 6 13 2 7 12

Example: Query 18 2.0 Traffic (GB/s) 1.5 Direction Read 1.0 Write 0.5 0.0 0 30 60 90 Time (s) 13

TPC-H Experiment • Single node runs MonetDB with TPC-H database (SF=100) • Gets remote memory from the 14 memory providers • DB temporary partition resides either on disk or in remote memory • Hot runs, 5 repetitions per query and setup 14

Average Execution Time (s) 1000 10 TPC-H 100 Results q01 q02 q03 q04 q05 q06 q07 q08 q09 q10 Query q11 15 q12 q13 q14 q15 q16 q17 q18 q19 q20 q21 q22 Experiment RRAM HDD

Summary • Remote Memory is interesting (...) • Lightweight technique • RDMA allows for remote memory to make sense from a technical perspective • OLAP database scenarios can benefit from this • Open issue: Hardware pricing/TCO 16

Thank You! Questions? http://is.gd/remotemem