L a r g e r - t h a n - Me mo r y D a t a Ma n - - PowerPoint PPT Presentation

l a r g e r t h a n me mo r y d a t a ma n a g e me n t o
SMART_READER_LITE
LIVE PREVIEW

L a r g e r - t h a n - Me mo r y D a t a Ma n - - PowerPoint PPT Presentation

L a r g e r - t h a n - Me mo r y D a t a Ma n a g e me n t o n Mo d e r n S t o r a g e H a r d w a r e f o r I n - Me mo r y O L T P D a t a b a s e S y s t e ms L i n


slide-1
SLIDE 1

L a r g e r

  • t

h a n

  • Me

mo r y D a t a Ma n a g e me n t

  • n

Mo d e r n S t

  • r

a g e H a r d w a r e f

  • r

I n

  • Me

mo r y O L T P D a t a b a s e S y s t e ms

L i n Ma , J

  • y

A r u l r a j , S a m Z h a

  • ,

A n d r e w P a v l

  • ,

S u b r a m a n y a R . D u l l

  • r

, Mi c h a e l J . G i a r d i n

  • ,

J e f f P a r k h u r s t , J a s

  • n

L . G a r d n e r , K s h i t i j D

  • s

h i , C

  • l

. S t a n l e y Z d

  • n

i k

slide-2
SLIDE 2

2

MO T I V A T I O N

  • A

l l

  • w

a n i n

  • m

e m

  • r

y D B MS t

  • s

t

  • r

e / a c c e s s d a t a

  • n

d i s k w i t h

  • u

t b r i n g i n g b a c k a l l t h e s l

  • w

p a r t s

  • f

a d i s k

  • r

i e n t e d D B MS .

  • D

i f f e r e n t p r

  • p

e r t i e s

  • f

s t

  • r

a g e d e v i c e s m a y a f f e c t i m p

  • r

t a n t d e s i g n d e c i s i

  • n

s .

slide-3
SLIDE 3

3

S T O R A G E T E C H N O L O G I E S

  • 1

m T u p l e s – 1 K B e a c h

  • S

y n c h r

  • n

i z a t i

  • n

E n a b l e d

HDD SMR SSD 3D XPoint NVRAM DRAM 0.01 1 100 10000 1000000 1KB Read 1KB Write 64KB Read 64KB Write Latency (nanosec)

r a n d

  • m

s e q u e n t i a l

slide-4
SLIDE 4

4

D E S I G N D E C I S I O N S

  • H

a r d w a r e i n d e p e n d e n t p

  • l

i c i e s –C

  • l

d T u p l e I d e n t i f i c a t i

  • n

–E v i c t e d T u p l e Me t a

  • d

a t a

  • H

a r d w a r e d e p e n d e n t p

  • l

i c i e s –C

  • l

d T u p l e R e t r i e v a l –Me r g i n g T h r e s h

  • l

d –A c c e s s Me t h

  • d

s

slide-5
SLIDE 5

5

H A R D WA R E I N D E P E N D E N T P O L I C I E S

slide-6
SLIDE 6

6

I N D E P E N D E N T P O L I C I E S

  • C
  • l

d T u p l e I d e n t i f i c a t i

  • n

–O p t i

  • n

# 1 : O n

  • l

i n e i d e n t i f i c a t i

  • n

–O p t i

  • n

# 2 : O f f

  • l

i n e i d e n t i f i c a t i

  • n
  • E

v i c t e d T u p l e Me t a

  • d

a t a

–O p t i

  • n

# 1 : Ma r k e r t

  • r

e p r e s e n t t h e

  • n
  • d

i s k p

  • s

i t i

  • n

–O p t i

  • n

# 2 : B l

  • m

f i l t e r –O p t i

  • n

# 3 : R e l y

  • n

v i r t u a l p a g i n g

slide-7
SLIDE 7

7

E V I C T E D T U P L E ME T A

  • D

A T A

In-Memory Table Heap

Tuple #01 Tuple #03 Tuple #04 Tuple #00 Tuple #02

Cold-Data Storage

header

Tuple #01 Tuple #03 Tuple #04

In-Memory Index

Access Frequency

Tuple #00 Tuple #01 Tuple #02 Tuple #03 Tuple #04 Tuple #05

<Block,Offset> <Block,Offset> <Block,Offset>

Index Bloom Filter

slide-8
SLIDE 8

8

H A R D WA R E D E P E N D E N T P O L I C I E S

slide-9
SLIDE 9

9 Transaction

C O L D T U P L E R E T R I E V A L

In-Memory Table Heap

Tuple #00 Tuple #02

Cold-Data Storage

header

Tuple #03 Tuple #04

  • O

p t i

  • n

# 1 : A b

  • r

t

  • a

n d

  • R

e s t a r t

Tuple #01 Tuple #01 Tuple #00 Read: ? Read: Tuple #01 Read: Tuple #02 Read:

Abort

slide-10
SLIDE 10

1 Transaction

C O L D T U P L E R E T R I E V A L

In-Memory Table Heap

Tuple #00 Tuple #02

Cold-Data Storage

header

Tuple #03 Tuple #04

  • O

p t i

  • n

# 2 : S y n c h r

  • n
  • u

s R e t r i e v a l

Tuple #01 Tuple #01 Tuple #00 Read: ? Read: Tuple #01 Read: Tuple #02 Read:

Stall

slide-11
SLIDE 11

1 1

  • O

p t i

  • n

# 1 : A l w a y s Me r g e

  • O

p t i

  • n

# 2 : Me r g e O n l y

  • n

U p d a t e

  • O

p t i

  • n

# 3 : S e l e c t i v e Me r g e Transaction

ME R G I N G T H R E S H O L D

In-Memory Table Heap

Tuple #00 Tuple #02

Cold-Data Storage

header

Tuple #03 Tuple #04 Tuple #01 Tuple #00 Read: Tuple #01 Read: Tuple #02 Read:

Sketch

slide-12
SLIDE 12

1 2

A C C E S S ME T H O D S

  • O

p t i

  • n

# 1 : B l

  • c

k

  • a

d d r e s s a b l e –B l

  • c

k

  • l

e v e l a c c e s s t h r

  • u

g h f i l e s y s t e m

  • O

p t i

  • n

# 2 : B y t e

  • a

d d r e s s a b l e ( N V R A M) –U s e m m a p t h r

  • u

g h a f i l e s y s t e m d e s i g n e d f

  • r

b y t e

  • a

d d r e s s a b l e N V R A M ( P MF S ) –D i r e c t l y

  • p

e r a t e

  • n

N V R A M- r e s i d e n t d a t a a s i f i t e x i s t e d i n D R A M

slide-13
SLIDE 13

1 3

E V A L U A T I O N

  • C
  • m

p a r e d e s i g n d e c i s i

  • n

s i n H

  • S

t

  • r

e w i t h a n t i

  • c

a c h i n g .

  • S

t

  • r

a g e D e v i c e s : –H a r d

  • D

i s k D r i v e ( H D D ) –S h i n g l e d Ma g n e t i c R e c

  • r

d i n g D r i v e ( S MR ) –S

  • l

i d

  • S

t a t e D r i v e ( S S D ) –3 D X P

  • i

n t ( 3 D X ) –N

  • n
  • v
  • l

a t i l e Me m

  • r

y ( N V R A M)

slide-14
SLIDE 14

1 4

C O L D T U P L E R E T R I E V A L

  • Y

C S B Wo r k l

  • a

d – 9 % R e a d s / 1 % Wr i t e s

  • 1

G B D a t a b a s e u s i n g 1 . 2 5 G B Me m

  • r

y

HDD SMR SDD 3DX NVM 50000 100000 150000 200000 Abort and Restart Synchronous Retrieval Throughput (txn/sec)

l a r g e b l

  • c

k s s m a l l b l

  • c

k s

slide-15
SLIDE 15

1 5

ME R G I N G T H R E S H O L D

  • Y

C S B Wo r k l

  • a

d – 9 % R e a d s / 1 % Wr i t e s

  • 1

G B D a t a b a s e u s i n g 1 . 2 5 G B Me m

  • r

y

HDD (AR) HDD (SR) 50000 100000 150000 200000

Merge (Update-Only) Merge (Top-5%) Merge (Top-20%) Merge (All)

Throughput (txn/sec)

I m p r

  • v

e m e n t :

  • S

MR : 3 %

  • S

S D : 1 7 %

  • 3

D X : 2 1 %

  • N

V R A M: 1 %

slide-16
SLIDE 16

1 7

C O N F I G U R A T I O N C O MP A R I S O N

  • G

e n e r i c C

  • n

f i g u r a t i

  • n

( 2 1 3 A n t i

  • c

a c h i n g ) –A b

  • r

t

  • a

n d

  • R

e s t a r t R e t r i e v a l –Me r g e ( A l l ) T h r e s h

  • l

d –1 2 4 K B B l

  • c

k S i z e

  • O

p t i mi z e d C

  • n

f i g u r a t i

  • n

–S y n c h r

  • n
  • u

s R e t r i e v a l –T

  • p
  • 5

% Me r g e T h r e s h

  • l

d –B l

  • c

k S i z e s ( H D D / S MR

  • 1

2 4 K B ) ( S S D / 3 D X

  • 1

6 K B ) –B y t e

  • a

d d r e s s a b l e a c c e s s f

  • r

N V R A M

slide-17
SLIDE 17

1 8

G E N E R I C V S O P T I MI Z E D

HDD SMR SSD 3D XPoint NVRAM

100000 200000

Generic Optimized VOTER Txn / sec HDD SMR SSD 3D XPoint NVRAM

200000 400000

TATP Txn / sec DRAM

slide-18
SLIDE 18

1 9

C O N C L U S I O N

  • L
  • w
  • l

a t e n c y s t

  • r

a g e d e v i c e s : S m a l l e r b l

  • c

k s i z e s a n d s y n c h r

  • n
  • u

s r e t r i e v a l

  • C
  • n

s t r a i n t s

  • n

m e r g e f r e q u e n c y i m p r

  • v

e p e r f

  • r

m a n c e

  • T

h e p e r f

  • r

m a n c e

  • f

N V R A M i s a s g

  • d

a s p u r e D R A M i f t r e a t e d c

  • r

r e c t l y

slide-19
SLIDE 19

2

l i n . m a @ c s . c m u . e d u 1

  • 8

4 4

  • 8

8

  • C

MU D B

E N D

slide-20
SLIDE 20

2 1

R E A L

  • WO

R L D I MP L E ME N T A T I O N S

  • H
  • S

t

  • r

e – A n t i

  • C

a c h i n g

  • Mi

c r

  • s
  • f

t H e k a t

  • n

– P r

  • j

e c t S i b e r i a

  • E

P F L ’ s V

  • l

t D B P r

  • t
  • t

y p e

  • A

p a c h e G e

  • d

e – O v e r f l

  • w

T a b l e s

  • Me

m S Q L – C

  • l

u m n a r T a b l e s

  • S
  • l

i d D B

  • P

* T I ME

slide-21
SLIDE 21

2 2

ME R G I N G T H R E S H O L D

  • Y

C S B Wo r k l

  • a

d – 9 % R e a d s / 1 % Wr i t e s

  • 1

G B D a t a b a s e u s i n g 1 . 2 5 G B Me m

  • r

y