Scalable Low-Latency Indexes for a Key-Value Store Ankita Kejriwal - - PowerPoint PPT Presentation

▶
scalable low latency indexes for a key value store
SMART_READER_LITE
LIVE PREVIEW

Scalable Low-Latency Indexes for a Key-Value Store Ankita Kejriwal - - PowerPoint PPT Presentation

Jan 08, 2024 458 likes •1.23k views

Scalable Low-Latency Indexes for a Key-Value Store Ankita Kejriwal With Arjun Gopalan, Ashish Gupta, Zhihao Jia, Stephen Yang and John Ousterhout Conjecture Can a key value store support strongly consistent secondary indexes while operating

slide-1
SLIDE 1

Scalable Low-Latency Indexes for a Key-Value Store

Ankita Kejriwal

With Arjun Gopalan, Ashish Gupta, Zhihao Jia, Stephen Yang and John Ousterhout

slide-2
SLIDE 2

Can a key value store support strongly consistent secondary indexes while operating at low latency and large scale?

SLIK Slide 2

Conjecture

slide-3
SLIDE 3
  • Scalable Low-latency Indexes for a Key-value Store: SLIK

§ Enables multiple secondary keys for each object § Allows lookups and range queries on these keys

  • Key design features:

§ Scalability using independent partitioning § Strong consistency using an ordered write approach

  • Implemented in RAMCloud

§ Low-latency, DRAM-based, distributed key-value store

  • Performance:

§ Scalability: Linear throughput increase with increasing number of partitions § Low-latency:11-13 µs indexed reads, 29-37 µs durable writes/overwrites § Latency approximately 2x non-indexed reads and writes

Summary of Results

SLIK Slide 3

slide-4
SLIDE 4

SLIK Slide 4

Talk Outline

  • Motivation
  • Design
  • Performance
  • Related Work
  • Summary
slide-5
SLIDE 5

SLIK Slide 5

Motivation

Traditional RDBMs

MySQL

slide-6
SLIDE 6

SLIK Slide 6

Motivation

Traditional RDBMs NoSQL Systems

+ scalability

  • data models
  • consistency

MySQL

slide-7
SLIDE 7

SLIK Slide 7

Motivation

Traditional RDBMs NoSQL Systems

+ scalability

  • data models
  • consistency

MySQL

+ consistency + data models + low latency

H-Base Espresso PNUTS Tao Spanner Megastore HyperDex MongoDB H-Store RAMCloud Mem- cached Yesquel

slide-8
SLIDE 8

SLIK Slide 8

Motivation

Traditional RDBMs NoSQL Systems

+ scalability

  • data models
  • consistency

MySQL

+ consistency + data models + low latency

H-Base Espresso PNUTS Tao Spanner Megastore HyperDex MongoDB H-Store RAMCloud Mem- cached Yesquel

slide-9
SLIDE 9

SLIK Slide 9

Talk Outline

  • Motivation
  • Design
  • Performance
  • Related Work
  • Summary
slide-10
SLIDE 10
  • Data model
  • Scalability
  • Strong consistency
  • Storage
  • Durability
  • Availability

SLIK Slide 10

Design

slide-11
SLIDE 11
  • Data model
  • Scalability
  • Strong consistency
  • Storage
  • Durability
  • Availability

SLIK Slide 11

Design

slide-12
SLIDE 12
  • Data model
  • Scalability
  • Strong consistency
  • Storage
  • Durability
  • Availability

SLIK Slide 12

Design

slide-13
SLIDE 13
  • Nearly constant low latency irrespective of the

server span

  • Linear increase in throughput with the server span

SLIK Slide 13

Scalability

slide-14
SLIDE 14

Index Partitioning: Colocation

Slide 14

  • Colocate index entries and objects
  • One of the keys used to partition the objects and indexes

Indexlet Tablet 1 q rose 2 a tulip 3 n violet Tablet 4 e clover 5 v daily 6 g iris

v è 5 e è 4 g è 6

Indexlet Tablet 7 m lily 8 b dahlia

b è 8 m è 7

Indexlet Server 1 Server 2 Server 3

n è 3 q è 1 a è 2

SLIK

primary key index key value index key primary key

slide-15
SLIDE 15

Index Partitioning: Colocation

Slide 15

  • Colocate index entries and objects
  • One of the keys used to partition the objects and indexes
  • No association between index partitions and index key ranges

Indexlet Tablet 1 q rose 2 a tulip 3 n violet Tablet 4 e clover 5 v daily 6 g iris

v è 5 e è 4 g è 6

Indexlet Tablet 7 m lily 8 b dahlia

b è 8 m è 7

Indexlet Server 1 Server 2 Server 3

n è 3 q è 1 a è 2

primary key index key value index key primary key

Metadata: tablet & indexlet w/ pk 1 to 3: S 1 tablet & indexlet w/ pk 4 to 6: S 2 tablet & indexlet w/ pk >= 7: S 3

slide-16
SLIDE 16

Slide 16

Index Partitioning: Colocation

Indexlet Tablet 1 q rose 2 a tulip 3 n violet

a è 2 n è 3 q è 1

Tablet 4 e clover 5 v daily 6 g iris

v è 5 e è 4 g è 6

Indexlet Tablet 7 m lily 8 b dahlia

b è 8 m è 7

Indexlet Server 1 Server 2 Server 3

SLIK

Client query: objects with index key between m - q

slide-17
SLIDE 17

Slide 17

Index Partitioning: Colocation

Client query: objects with index key between m - q

Indexlet Tablet 1 q rose 2 a tulip 3 n violet

a è 2 n è 3 q è 1

Tablet 4 e clover 5 v daily 6 g iris

v è 5 e è 4 g è 6

Indexlet Tablet 7 m lily 8 b dahlia

b è 8 m è 7

Indexlet Server 1 Server 2 Server 3

SLIK

slide-18
SLIDE 18

Slide 18

Index Partitioning: Colocation

Indexlet Tablet 1 q rose 2 a tulip 3 n violet

a è 2 n è 3 q è 1

Tablet 4 e clover 5 v daily 6 g iris

v è 5 e è 4 g è 6

Indexlet Tablet 7 m lily 8 b dahlia

b è 8 m è 7

Indexlet Server 1 Server 2 Server 3

Client query: objects with index key between m - q

SLIK

slide-19
SLIDE 19

Slide 19

Index Partitioning: Colocation

Not Scalable!

Indexlet Tablet 1 q rose 2 a tulip 3 n violet

a è 2 n è 3 q è 1

Tablet 4 e clover 5 v daily 6 g iris

v è 5 e è 4 g è 6

Indexlet Tablet 7 m lily 8 b dahlia

b è 8 m è 7

Indexlet Server 1 Server 2 Server 3

Client query: objects with index key between m - q

slide-20
SLIDE 20

Index Partitioning: Independent

Slide 20

  • Partition each index and table independently
  • Partition each index according to sort order for that index

SLIK

Tablet 1 q rose 2 a tulip 3 n violet Tablet 4 e clover 5 v daily 6 g iris Tablet 7 m lily 8 b dahlia Server 1 Server 2 Server 3 Indexlet

a è 2 b è 8 e è 4 g è 6

Server 4 Indexlet

n è 3 m è 7 v è 5 q è 1

Server 5 primary key index key value index key primary key

slide-21
SLIDE 21

Index Partitioning: Independent

Slide 21

  • Partition each index and table independently
  • Partition each index according to sort order for that index

SLIK

Tablet 1 q rose 2 a tulip 3 n violet Tablet 4 e clover 5 v daily 6 g iris Tablet 7 m lily 8 b dahlia Server 1 Server 2 Server 3 Indexlet

a è 2 b è 8 e è 4 g è 6

Server 4 Indexlet

n è 3 m è 7 v è 5 q è 1

Server 5 primary key index key value index key primary key

Metadata: tablet w/ pk 1 to 3: S 1 tablet w/ pk 4 to 6: S 2 tablet w/ pk >= 7: S 3 indexlet w/ sk a to g: S 4 indexlet w/ sk >= h: S 5

slide-22
SLIDE 22

Index Partitioning: Independent

Slide 22 SLIK

Client query: objects with index key between m - q

Tablet 1 q rose 2 a tulip 3 n violet Tablet 4 e clover 5 v daily 6 g iris Tablet 7 m lily 8 b dahlia Server 1 Server 2 Server 3 Indexlet

a è 2 b è 8 e è 4 g è 6

Server 4 Indexlet

n è 3 m è 7 v è 5 q è 1

Server 5

slide-23
SLIDE 23

Index Partitioning: Independent

Slide 23

Client query: objects with index key between m - q

Indexlet

a è 2 b è 8 e è 4 g è 6

Server 4 Indexlet

n è 3 m è 7 v è 5 q è 1

Server 5 Tablet 1 q rose 2 a tulip 3 n violet Tablet 4 e clover 5 v daily 6 g iris Tablet 7 m lily 8 b dahlia Server 1 Server 2 Server 3

SLIK

slide-24
SLIDE 24

Index Partitioning: Independent

Slide 24

Client query: objects with index key between m - q

Tablet 1 q rose 2 a tulip 3 n violet Tablet 4 e clover 5 v daily 6 g iris Tablet 7 m lily 8 b dahlia Server 1 Server 2 Server 3 Indexlet

a è 2 b è 8 e è 4 g è 6

Server 4 Indexlet

n è 3 m è 7 v è 5 q è 1

Server 5

SLIK

slide-25
SLIDE 25

Index Partitioning: Independent

Slide 25

Client query: objects with index key between m - q

Tablet 1 q rose 2 a tulip 3 n violet Tablet 4 e clover 5 v daily 6 g iris Tablet 7 m lily 8 b dahlia Server 1 Server 2 Server 3 Indexlet

a è 2 b è 8 e è 4 g è 6

Server 4 Indexlet

n è 3 m è 7 v è 5 q è 1

Server 5

Scalable!

slide-26
SLIDE 26
  • Nearly constant low latency irrespective of the

server span

  • Linear increase in throughput with the server span

SLIK Slide 26

Scalability

slide-27
SLIDE 27
  • Nearly constant low latency irrespective of the

server span

  • Linear increase in throughput with the server span
  • Solution: Use independent partitioning
  • But: indexed object writes: distributed operations
  • Potential consistency issues between indexes and
  • bjects

SLIK Slide 27

Scalability

slide-28
SLIDE 28
  • Data model
  • Scalability
  • Strong consistency
  • Storage
  • Durability
  • Availability

SLIK Slide 28

Design

slide-29
SLIDE 29

SLIK Slide 29

Design

  • Data model
  • Scalability
  • Strong consistency
  • Storage
  • Durability
  • Availability
slide-30
SLIDE 30

SLIK Slide 30

Consistency Properties

  • If an object contains a given secondary key, then an

index lookup with that key will return the object

  • If an object is returned by index lookup, then this
  • bject contains a secondary key for that index within

the specified range

slide-31
SLIDE 31

SLIK Slide 31

Consistency Properties

  • If an object contains a given secondary key, then an

index lookup with that key will return the object

  • If an object is returned by index lookup, then this
  • bject contains a secondary key for that index within

the specified range

Frank Bob Alice Trent Carol Peggy

slide-32
SLIDE 32

SLIK Slide 32

Consistency Properties

  • If an object contains a given secondary key, then an

index lookup with that key will return the object

  • If an object is returned by index lookup, then this
  • bject contains a secondary key for that index within

the specified range

students with name between a – d?

Alice Carol Frank Bob Alice Trent Carol Peggy

?

slide-33
SLIDE 33

SLIK Slide 33

Consistency Properties

  • If an object contains a given secondary key, then an

index lookup with that key will return the object

  • If an object is returned by index lookup, then this
  • bject contains a secondary key for that index within

the specified range

slide-34
SLIDE 34

SLIK Slide 34

Consistency Properties

  • If an object contains a given secondary key, then an

index lookup with that key will return the object

  • If an object is returned by index lookup, then this
  • bject contains a secondary key for that index within

the specified range

Frank Bob Alice Trent Carol Peggy

slide-35
SLIDE 35

SLIK Slide 35

Consistency Properties

  • If an object contains a given secondary key, then an

index lookup with that key will return the object

  • If an object is returned by index lookup, then this
  • bject contains a secondary key for that index within

the specified range

students with name between a – d?

Alice Bob Carol Peggy Frank Bob Alice Trent Carol Peggy

?

slide-36
SLIDE 36

SLIK Slide 36

Consistency Properties

  • If an object contains a given secondary key, then an

index lookup with that key will return the object

  • If an object is returned by index lookup, then this
  • bject contains a secondary key for that index within

the specified range

students with name between a – d?

Alice Bob Carol Frank Bob Alice Trent Carol Peggy

slide-37
SLIDE 37

Consistency

Slide 37

  • Consistency properties:

§ If an object contains a given secondary key, then an index lookup with that key will return the object § If an object is returned by index lookup, then this object contains a secondary key for that index within the specified range

  • Solution:

§ Longer index lifespan (via ordered writes) § Object data is ground truth and index entries serve as hints

slide-38
SLIDE 38

Consistency

Slide 38

  • Consistency properties:

§ If an object contains a given secondary key, then an index lookup with that key will return the object § If an object is returned by index lookup, then this object contains a secondary key for that index within the specified range

  • Solution:

§ Longer index lifespan (via ordered writes) § Object data is ground truth and index entries serve as hints

Foo Bob ... Object Index Entry Bob è Foo

slide-39
SLIDE 39

Consistency

Slide 39

  • Consistency properties:

§ If an object contains a given secondary key, then an index lookup with that key will return the object § If an object is returned by index lookup, then this object contains a secondary key for that index within the specified range

  • Solution:

§ Longer index lifespan (via ordered writes) § Object data is ground truth and index entries serve as hints

Foo Bob ... Object Index Entry Bob è Foo Sam è Foo

1. Add new index entry

slide-40
SLIDE 40

Consistency

Slide 40

  • Consistency properties:

§ If an object contains a given secondary key, then an index lookup with that key will return the object § If an object is returned by index lookup, then this object contains a secondary key for that index within the specified range

  • Solution:

§ Longer index lifespan (via ordered writes) § Object data is ground truth and index entries serve as hints

Foo Sam ... Object Index Entry Bob è Foo Sam è Foo

1. Add new index entry 2. Modify object

slide-41
SLIDE 41

Consistency

Slide 41

  • Consistency properties:

§ If an object contains a given secondary key, then an index lookup with that key will return the object § If an object is returned by index lookup, then this object contains a secondary key for that index within the specified range

  • Solution:

§ Longer index lifespan (via ordered writes) § Object data is ground truth and index entries serve as hints

Foo Sam ... Object Index Entry Sam è Foo

1. Add new index entry 2. Modify object 3. Remove old index entry

slide-42
SLIDE 42

Consistency

Slide 42

  • Consistency properties:

§ If an object contains a given secondary key, then an index lookup with that key will return the object § If an object is returned by index lookup, then this object contains a secondary key for that index within the specified range

  • Solution:

§ Longer index lifespan (via ordered writes) § Object data is ground truth and index entries serve as hints

time Bob è Foo Sam è Foo Foo Bob ... Foo Sam ... Object Index Entry modify object remove object write object

slide-43
SLIDE 43

Consistency

Slide 43

  • Consistency properties:

§ If an object contains a given secondary key, then an index lookup with that key will return the object § If an object is returned by index lookup, then this object contains a secondary key for that index within the specified range

  • Solution:

§ Longer index lifespan (via ordered writes) § Object data is ground truth and index entries serve as hints

time Bob è Foo Sam è Foo Foo Bob ... Foo Sam ... Object Index Entry modify object remove object write object

slide-44
SLIDE 44

Consistency

Slide 44

  • Consistency properties:

§ If an object contains a given secondary key, then an index lookup with that key will return the object § If an object is returned by index lookup, then this object contains a secondary key for that index within the specified range

  • Solution:

§ Longer index lifespan (via ordered writes) § Object data is ground truth and index entries serve as hints

time Bob è Foo Sam è Foo commit point commit point commit point Foo Bob ... Foo Sam ... Object Index Entry modify object remove object write object x

slide-45
SLIDE 45

Consistency

Slide 45

  • Consistency properties:

§ If an object contains a given secondary key, then an index lookup with that key will return the object § If an object is returned by index lookup, then this object contains a secondary key for that index within the specified range

  • Solution:

§ Longer index lifespan (via ordered writes) § Object data is ground truth and index entries serve as hints

time Bob è Foo Sam è Foo commit point commit point commit point Foo Bob ... Foo Sam ... Object Index Entry modify object remove object write object

slide-46
SLIDE 46

SLIK Slide 46

Talk Outline

  • Motivation
  • Design
  • Performance
  • Related Work
  • Summary
slide-47
SLIDE 47
  • Does SLIK provide low latency?
  • Does SLIK provide scalability?
  • How does the performance of indexing with SLIK

compare to other state-of-the-art systems?

SLIK Slide 47

Performance: Questions

slide-48
SLIDE 48
  • H-Store:

§ Main memory database § Data (and indexes) partitioned based on specified attribute § Many parameters for tuning

  • Got assistance from developers to tune for each test
  • Examples: txn_incoming_delay, partitioning column
  • HyperDex:

§ Spaces containing objects § Data (and indexes) partitioned using hyperspace hashing § Each index contains all object data § Designed to use disk for storage

Slide 48

Performance: Systems for Comparison

SLIK

slide-49
SLIDE 49

SLIK Slide 49

Hardware

slide-50
SLIDE 50

Slide 50

Latency

Experiments:

1.

Lookups: table with single secondary index

2.

Overwrites: table with single secondary index

3.

Overwrites: varying number of secondary indexes Configuration:

  • Single client
  • Single partition for table and (each) index
  • Object: 30 B pk, 30 B sk, 100 B value
  • SLIK: Three-way replication to durable backups
  • H-Store: No replication, durability disabled, single server
slide-51
SLIDE 51

Slide 51

Lookup Latency

10 10 10 10 10 10 10 Size of Index (# objects)

50 100 150 200 250 100 101 102 103 104 105 106 (a) Lookup Latency (µs)

H-Store

142.10 138.18 150.11 152.37 134.97 147.42 132.00

SLIK TCP

45.2 44.9 42.7 48.5 49.7 45.6 54.5

SLIK

11.0 10.2 11.7 11.6 12.7 12.8 13.1

slide-52
SLIDE 52

Slide 52

Lookup Latency

10 10 10 10 10 10 10 Size of Index (# objects)

50 100 150 200 250 100 101 102 103 104 105 106 (a) Lookup Latency (µs)

H-Store

142.10 138.18 150.11 152.37 134.97 147.42 132.00

SLIK TCP

45.2 44.9 42.7 48.5 49.7 45.6 54.5

SLIK

11.0 10.2 11.7 11.6 12.7 12.8 13.1

slide-53
SLIDE 53

Overwrite Latency

Slide 53

50 100 150 200 250 100 101 102 103 104 105 106 (b) Overwrite Latency (µs) Size of Index (# objects)

H-Store

143.51 143.00 151.39 153.28 137.74 148.88 133.54

SLIK TCP

124.4 135.8 126.5 125.9 124.3 123.8 129.7

SLIK

31.4 32.7 34.2 34.3 35.2 35.2 37.0

slide-54
SLIDE 54

Multiple Secondary Indexes

Slide 54

10 100 1000 1 2 3 4 5 6 7 8 9 10 Overwrite Latency (µs) Number of Indexes

H-Store via PK

152.97 265.91 273.41 285.99 287.72

H-Store via SK

181.98 1474.39 1682.16 1704.69 1756.99

SLIK TCP

138.5 139.1 156.2 165.2 164.3 175.3 175.7 179.1 182.0 184.6

SLIK

33.0 35.3 39.8 39.0 42.7 42.4 46.4 47.3 49.5 51.2

slide-55
SLIDE 55

Multiple Secondary Indexes

Slide 55

10 100 1000 1 2 3 4 5 6 7 8 9 10 Overwrite Latency (µs) Number of Indexes

H-Store via PK

152.97 265.91 273.41 285.99 287.72

H-Store via SK

181.98 1474.39 1682.16 1704.69 1756.99

SLIK TCP

138.5 139.1 156.2 165.2 164.3 175.3 175.7 179.1 182.0 184.6

SLIK

33.0 35.3 39.8 39.0 42.7 42.4 46.4 47.3 49.5 51.2

slide-56
SLIDE 56

Multiple Secondary Indexes

Slide 56

10 100 1000 1 2 3 4 5 6 7 8 9 10 Overwrite Latency (µs) Number of Indexes

H-Store via PK

152.97 265.91 273.41 285.99 287.72

H-Store via SK

181.98 1474.39 1682.16 1704.69 1756.99

SLIK TCP

138.5 139.1 156.2 165.2 164.3 175.3 175.7 179.1 182.0 184.6

SLIK

33.0 35.3 39.8 39.0 42.7 42.4 46.4 47.3 49.5 51.2

slide-57
SLIDE 57

Slide 57

Scalability

Compare: (a) partitioning approaches (b) systems Experiments:

1.

Lookup throughput with increasing number of partitions

2.

Lookup latency with increasing number of partitions Configuration:

  • Single table with one secondary index
  • Table and index partitioned across servers
  • Object: 30 B pk, 30 B sk, 100 B value
  • Throughput experiments: Loaded system
  • Latency experiments: Unloaded system
slide-58
SLIDE 58

Scalability: Throughput

500 1000 1500 2000 2500 3000 3500 4000 2 4 6 8 10 Throughput (103 lookups/sec) Number of Indexlets

Independent Partitioning Colocation

634 940 1249 1558 1859 2184 2478 2782 3092 335 461 423 463 447 457 447 357 441 418 435

Slide 58

slide-59
SLIDE 59

Slide 59

Scalability: Throughput

1000 2000 3000 4000 5000 2 4 6 8 10 12 14 16 18 20 Throughput (103 lookups/sec) Number of Servers

H-Store SLIK TCP SLIK

96.90 312.35 347.32 396.27 430 653 794 1001 1199 1352 1445 1663 1807 220 580 1127 1619 2197 2655 3199 3629 4248 4629 5069

slide-60
SLIDE 60

Scalability: Latency

20 40 60 80 100 10 20 30 40 50 60 70 80 Lookup Latency (µs) Number of Servers

Colocation size 1 Colocation size 10 Independent size 1 Independent size 10 8.3 26.7 87.3 16.2 89.7 15.2 16.7 12.7 22.3 28.8

Slide 60

slide-61
SLIDE 61

Slide 61

Scalability: Latency

50 100 150 200 250 300 1 2 3 4 5 6 7 8 9 10

Average Latency per Lookup (µs)

Number of Indexlets

H-Store

119.4 152.6 179.6 199.2 215.6 243.4 240.3 267.4 269.8 267.2

SLIK TCP

49.9 55.6 58.2 69.1 81.6 90.6 91.0 112.7 114.4 113.3

SLIK

13.1 13.3 13.9 13.7 14.4 14.7 14.5 14.4 14.6 14.7

slide-62
SLIDE 62

SLIK Slide 62

Talk Outline

  • Motivation
  • Design
  • Performance
  • Related Work
  • Summary
slide-63
SLIDE 63

SLIK Slide 63

Related Work

Data storage system

§ Data model (spectrum from key-value to relational) § Consistency (spectrum from eventual to strong) § Performance: latency and/or throughput

slide-64
SLIDE 64

Current Web Scale Datastores

Slide 64 SLIK

100s 10s 1s 100ms 10ms 1ms 100µs Eventual Strong 10µs Causal, SI, “Define your own”

Read / write latency (approx) Consistency Level

Better Better

slide-65
SLIDE 65

Current Web Scale Datastores

Slide 65 SLIK

100s 10s 1s 100ms 10ms 1ms 100µs Eventual Strong 10µs Causal, SI, “Define your own”

Read / write latency (approx) Consistency Level

Better Better Tao CouchDB Espresso PNUTS

slide-66
SLIDE 66

Current Web Scale Datastores

Slide 66 SLIK

100s 10s 1s 100ms 10ms 1ms 100µs Eventual Strong 10µs Causal, SI, “Define your own”

Read / write latency (approx) Consistency Level

Better Better Tao CouchDB H-Store HyperDex MongoDB Spanner Espresso H-Base PNUTS

slide-67
SLIDE 67

Current Web Scale Datastores

Slide 67 SLIK

100s 10s 1s 100ms 10ms 1ms 100µs Eventual Strong 10µs Causal, SI, “Define your own”

Read / write latency (approx) Consistency Level

Better Better Tao Cassandra CouchDB H-Store HyperDex MongoDB Spanner Espresso H-Base PNUTS

slide-68
SLIDE 68

Current Web Scale Datastores

Slide 68 SLIK

100s 10s 1s 100ms 10ms 1ms 100µs Eventual Strong 10µs Causal, SI, “Define your own”

Read / write latency (approx) Consistency Level

Better Better SLIK Tao Cassandra CouchDB H-Store HyperDex MongoDB Spanner Espresso H-Base PNUTS

slide-69
SLIDE 69

SLIK Slide 69

Talk Outline

  • Motivation
  • Design
  • Performance
  • Related Work
  • Summary
slide-70
SLIDE 70

Can a key value store support strongly consistent secondary indexes while operating at low latency and large scale?

SLIK Slide 70

Conjecture

slide-71
SLIDE 71

A key value store can support strongly consistent secondary indexes while operating at low latency and large scale.

SLIK Slide 71

Summary

Lookups and range queries on secondary keys

slide-72
SLIDE 72

A key value store can support strongly consistent secondary indexes while operating at low latency and large scale.

SLIK Slide 72

Summary

Lookups and range queries on secondary keys By using ordered writes and treating indexes as hints

slide-73
SLIDE 73

A key value store can support strongly consistent secondary indexes while operating at low latency and large scale.

SLIK Slide 73

Summary

Lookups and range queries on secondary keys By using ordered writes and treating indexes as hints By using independent partitioning we get: linear throughput increase and minimal impact on latency as the scale increases

slide-74
SLIDE 74

A key value store can support strongly consistent secondary indexes while operating at low latency and large scale.

SLIK Slide 74

Summary

Lookups and range queries on secondary keys By using approaches that have minimal overheads we get: 11-13 µs lookups and 30-37 µs (over)writes By using ordered writes and treating indexes as hints By using independent partitioning we get: linear throughput increase and minimal impact on latency as the scale increases

slide-75
SLIDE 75

A key value store can support strongly consistent secondary indexes while operating at low latency and large scale.

SLIK Slide 75

Summary

Lookups and range queries on secondary keys By using approaches that have minimal overheads we get: 11-13 µs lookups and 30-37 µs (over)writes By using ordered writes and treating indexes as hints By using independent partitioning we get: linear throughput increase and minimal impact on latency as the scale increases

slide-76
SLIDE 76

Thank you!

Code available free and open source: github.com/PlatformLab/RAMCloud My papers and other information at: stanford.edu/~ankitak I can be reached at: ankitak@cs.stanford.edu