Autonomic Configuration of HyperDex via Analytical Modelling SAC - - PowerPoint PPT Presentation

▶
autonomic configuration of hyperdex via analytical
SMART_READER_LITE
LIVE PREVIEW

Autonomic Configuration of HyperDex via Analytical Modelling SAC - - PowerPoint PPT Presentation

Apr 11, 2023 162 likes •842 views

Autonomic Configuration of HyperDex via Analytical Modelling SAC 2014 (DADS) Nuno Diegues , Muhammet Orazov, Joo Paiva, Lus Rodrigues, Paolo Romano Autonomic Configuration HyperDex Analytical Modelling Autonomic

slide-1
SLIDE 1

Autonomic Configuration of HyperDex via Analytical Modelling

Nuno Diegues, Muhammet Orazov, João Paiva, Luís Rodrigues, Paolo Romano

SAC 2014 (DADS)

slide-2
SLIDE 2
  • Autonomic
  • Configuration
  • HyperDex
  • Analytical Modelling
slide-3
SLIDE 3
  • Autonomic
  • Configuration
  • HyperDex
  • Analytical Modelling

Distributed Multi-dimensional Key-Value Store

slide-4
SLIDE 4
  • Autonomic
  • Configuration
  • HyperDex
  • Analytical Modelling

Distributed Multi-dimensional Key-Value Store Non-trivial to tune for performance

slide-5
SLIDE 5
  • Autonomic
  • Configuration
  • HyperDex
  • Analytical Modelling

Distributed Multi-dimensional Key-Value Store Automatic Adaptation/Management Non-trivial to tune for performance

slide-6
SLIDE 6
  • Autonomic
  • Configuration
  • HyperDex
  • Analytical Modelling

Distributed Multi-dimensional Key-Value Store Automatic Adaptation/Management Non-trivial to tune for performance White-box Approach

slide-7
SLIDE 7

Why Key-Value Stores?

Key-value stores: high performance elastic scalability

slide-8
SLIDE 8

Why Key-Value Stores?

Key-value stores: high performance elastic scalability restrictive API:

  • operations only given the primary key
slide-9
SLIDE 9

Motivation: Key-Value Stores

ID:$$$$$$$$hiltonparis1 Name:$$$$$$Hotel$Hilton Price:$$$$$250€ City:$$$$$$Paris Stars:$$$$$5 Phone:$$$$$87162801020 Animals:$$$No Postal:$$$$ParisA1291 Category:$$Hotel (…) ID:$$$$$$$$ibisparis1 Name:$$$$$$Hotel$Ibis Price:$$$$$50€ City:$$$$$$Paris Stars:$$$$$3 Phone:$$$$$87188190021 Animals:$$$No Postal:$$$$ParisA1194 Category:$$Hotel (…) ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisA7723 Category:$$Motel (…) ID:$$$$$$$$alifparis1 Name:$$$$$$Hotel$Alif Price:$$$$$30€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$35192900912 Animals:$$$No Postal:$$$$LisbonA1233 Category:$$Hotel (…) ID:$$$$$$$$tivolilx1 Nome:$$$$$$Hotel$Tivoli Preço:$$$$$90€ Cidade:$$$$Lisbon Estrelas:$$4 TLF:$$$$$$$351923821911 Animais:$$$No Postal:$$$$LisbonA1000 Categoria:$Hotel (…)

Large, Partitionable Data Space

slide-10
SLIDE 10

Motivation: Key-Value Stores

ID:$$$$$$$$hiltonparis1 Name:$$$$$$Hotel$Hilton Price:$$$$$250€ City:$$$$$$Paris Stars:$$$$$5 Phone:$$$$$87162801020 Animals:$$$No Postal:$$$$ParisA1291 Category:$$Hotel (…) ID:$$$$$$$$ibisparis1 Name:$$$$$$Hotel$Ibis Price:$$$$$50€ City:$$$$$$Paris Stars:$$$$$3 Phone:$$$$$87188190021 Animals:$$$No Postal:$$$$ParisA1194 Category:$$Hotel (…) ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisA7723 Category:$$Motel (…) ID:$$$$$$$$alifparis1 Name:$$$$$$Hotel$Alif Price:$$$$$30€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$35192900912 Animals:$$$No Postal:$$$$LisbonA1233 Category:$$Hotel (…) ID:$$$$$$$$tivolilx1 Nome:$$$$$$Hotel$Tivoli Preço:$$$$$90€ Cidade:$$$$Lisbon Estrelas:$$4 TLF:$$$$$$$351923821911 Animais:$$$No Postal:$$$$LisbonA1000 Categoria:$Hotel (…)

Large, Partitionable Data Space E.g.: by primary key

slide-11
SLIDE 11

Motivation: Key-Value Stores

ID:$$$$$$$$hiltonparis1 Name:$$$$$$Hotel$Hilton Price:$$$$$250€ City:$$$$$$Paris Stars:$$$$$5 Phone:$$$$$87162801020 Animals:$$$No Postal:$$$$ParisA1291 Category:$$Hotel (…) ID:$$$$$$$$ibisparis1 Name:$$$$$$Hotel$Ibis Price:$$$$$50€ City:$$$$$$Paris Stars:$$$$$3 Phone:$$$$$87188190021 Animals:$$$No Postal:$$$$ParisA1194 Category:$$Hotel (…) ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisA7723 Category:$$Motel (…) ID:$$$$$$$$alifparis1 Name:$$$$$$Hotel$Alif Price:$$$$$30€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$35192900912 Animals:$$$No Postal:$$$$LisbonA1233 Category:$$Hotel (…) ID:$$$$$$$$tivolilx1 Nome:$$$$$$Hotel$Tivoli Preço:$$$$$90€ Cidade:$$$$Lisbon Estrelas:$$4 TLF:$$$$$$$351923821911 Animais:$$$No Postal:$$$$LisbonA1000 Categoria:$Hotel (…)

A Z

Mappable to a space E.g.: by hashing the key

slide-12
SLIDE 12

Motivation: Key-Value Stores

ID:$$$$$$$$hiltonparis1 Name:$$$$$$Hotel$Hilton Price:$$$$$250€ City:$$$$$$Paris Stars:$$$$$5 Phone:$$$$$87162801020 Animals:$$$No Postal:$$$$ParisA1291 Category:$$Hotel (…) ID:$$$$$$$$ibisparis1 Name:$$$$$$Hotel$Ibis Price:$$$$$50€ City:$$$$$$Paris Stars:$$$$$3 Phone:$$$$$87188190021 Animals:$$$No Postal:$$$$ParisA1194 Category:$$Hotel (…) ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisA7723 Category:$$Motel (…) ID:$$$$$$$$alifparis1 Name:$$$$$$Hotel$Alif Price:$$$$$30€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$35192900912 Animals:$$$No Postal:$$$$LisbonA1233 Category:$$Hotel (…) ID:$$$$$$$$tivolilx1 Nome:$$$$$$Hotel$Tivoli Preço:$$$$$90€ Cidade:$$$$Lisbon Estrelas:$$4 TLF:$$$$$$$351923821911 Animais:$$$No Postal:$$$$LisbonA1000 Categoria:$Hotel (…)

A Z

Space 1 Space 2 Space … Space N

slide-13
SLIDE 13

Motivation: Key-Value Stores

ID:$$$$$$$$hiltonparis1 Name:$$$$$$Hotel$Hilton Price:$$$$$250€ City:$$$$$$Paris Stars:$$$$$5 Phone:$$$$$87162801020 Animals:$$$No Postal:$$$$ParisA1291 Category:$$Hotel (…) ID:$$$$$$$$ibisparis1 Name:$$$$$$Hotel$Ibis Price:$$$$$50€ City:$$$$$$Paris Stars:$$$$$3 Phone:$$$$$87188190021 Animals:$$$No Postal:$$$$ParisA1194 Category:$$Hotel (…) ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisA7723 Category:$$Motel (…) ID:$$$$$$$$alifparis1 Name:$$$$$$Hotel$Alif Price:$$$$$30€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$35192900912 Animals:$$$No Postal:$$$$LisbonA1233 Category:$$Hotel (…) ID:$$$$$$$$tivolilx1 Nome:$$$$$$Hotel$Tivoli Preço:$$$$$90€ Cidade:$$$$Lisbon Estrelas:$$4 TLF:$$$$$$$351923821911 Animais:$$$No Postal:$$$$LisbonA1000 Categoria:$Hotel (…)

A Z

Space 1 Space 2 Space … Space N

slide-14
SLIDE 14

Motivation: Key-Value Stores

ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisC7723 Category:$$Motel (…)

A Z

Operation on a single item?

slide-15
SLIDE 15

Motivation: Key-Value Stores

ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisC7723 Category:$$Motel (…)

A Z

Operation on a single item? use the primary key

slide-16
SLIDE 16

Motivation: Key-Value Stores

ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisC7723 Category:$$Motel (…)

A Z

Operation on a single item? use the primary key

slide-17
SLIDE 17

Motivation: Key-Value Stores

ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisC7723 Category:$$Motel (…)

A Z

Efficient operation

slide-18
SLIDE 18

Motivation: Key-Value Stores

ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisC7723 Category:$$Motel (…)

A Z

Search by price and city

slide-19
SLIDE 19

Motivation: Key-Value Stores

ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisC7723 Category:$$Motel (…)

A Z

?

Search by price and city

slide-20
SLIDE 20

Motivation: Key-Value Stores

ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisC7723 Category:$$Motel (…)

A Z

Search by price and city Inefficient

slide-21
SLIDE 21

Motivation: Multi-Dimensional KVS

Extended functionality and APIs

  • Support for operations with secondary keys

Maintain high performance Preserve scalability

e.g.: HyperDex [SIGCOMM12]

slide-22
SLIDE 22

Motivation: Multi-Dimensional KVS

Extended functionality and APIs

  • Support for operations with secondary keys

Maintain high performance Preserve scalability Complex to configure properly

e.g.: HyperDex [SIGCOMM12]

slide-23
SLIDE 23

Motivation: Multi-Dimensional Systems

ID:$$$$$$$$hiltonparis1 Name:$$$$$$Hotel$Hilton Price:$$$$$250€ City:$$$$$$Paris Stars:$$$$$5 Phone:$$$$$87162801020 Animals:$$$No Postal:$$$$ParisA1291 Category:$$Hotel (…) ID:$$$$$$$$ibisparis1 Name:$$$$$$Hotel$Ibis Price:$$$$$50€ City:$$$$$$Paris Stars:$$$$$3 Phone:$$$$$87188190021 Animals:$$$No Postal:$$$$ParisA1194 Category:$$Hotel (…) ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisA7723 Category:$Motel (…) ID:$$$$$$$$alifparis1 Name:$$$$$$Hotel$Alif Price:$$$$$30€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$35192900912 Animals:$$$No Postal:$$$$LisbonA1233 Category:$$Hotel (…) ID:$$$$$$$$tivolilx1 Name:$$$$$$Hotel$Tivoli Price:$$$$$90€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$351923821911 Animals:$$$No Postal:$$$$LisbonA1000 Category:$$Hotel (…)

How do they work?

slide-24
SLIDE 24

Motivation: Multi-Dimensional Systems

ID:$$$$$$$$hiltonparis1 Name:$$$$$$Hotel$Hilton Price:$$$$$250€ City:$$$$$$Paris Stars:$$$$$5 Phone:$$$$$87162801020 Animals:$$$No Postal:$$$$ParisA1291 Category:$$Hotel (…) ID:$$$$$$$$ibisparis1 Name:$$$$$$Hotel$Ibis Price:$$$$$50€ City:$$$$$$Paris Stars:$$$$$3 Phone:$$$$$87188190021 Animals:$$$No Postal:$$$$ParisA1194 Category:$$Hotel (…) ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisA7723 Category:$Motel (…) ID:$$$$$$$$alifparis1 Name:$$$$$$Hotel$Alif Price:$$$$$30€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$35192900912 Animals:$$$No Postal:$$$$LisbonA1233 Category:$$Hotel (…) ID:$$$$$$$$tivolilx1 Name:$$$$$$Hotel$Tivoli Price:$$$$$90€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$351923821911 Animals:$$$No Postal:$$$$LisbonA1000 Category:$$Hotel (…)

A Z City 400€ 0€ Price

E.g.: with 2 dimensions

slide-25
SLIDE 25

Motivation: Multi-Dimensional Systems

ID:$$$$$$$$hiltonparis1 Name:$$$$$$Hotel$Hilton Price:$$$$$250€ City:$$$$$$Paris Stars:$$$$$5 Phone:$$$$$87162801020 Animals:$$$No Postal:$$$$ParisA1291 Category:$$Hotel (…) ID:$$$$$$$$ibisparis1 Name:$$$$$$Hotel$Ibis Price:$$$$$50€ City:$$$$$$Paris Stars:$$$$$3 Phone:$$$$$87188190021 Animals:$$$No Postal:$$$$ParisA1194 Category:$$Hotel (…) ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisA7723 Category:$Motel (…) ID:$$$$$$$$alifparis1 Name:$$$$$$Hotel$Alif Price:$$$$$30€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$35192900912 Animals:$$$No Postal:$$$$LisbonA1233 Category:$$Hotel (…) ID:$$$$$$$$tivolilx1 Name:$$$$$$Hotel$Tivoli Price:$$$$$90€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$351923821911 Animals:$$$No Postal:$$$$LisbonA1000 Category:$$Hotel (…)

A Z City 400€ 0€ Price

Space is still partitionable

slide-26
SLIDE 26

Motivation: Multi-Dimensional Systems

ID:$$$$$$$$hiltonparis1 Name:$$$$$$Hotel$Hilton Price:$$$$$250€ City:$$$$$$Paris Stars:$$$$$5 Phone:$$$$$87162801020 Animals:$$$No Postal:$$$$ParisA1291 Category:$$Hotel (…) ID:$$$$$$$$ibisparis1 Name:$$$$$$Hotel$Ibis Price:$$$$$50€ City:$$$$$$Paris Stars:$$$$$3 Phone:$$$$$87188190021 Animals:$$$No Postal:$$$$ParisA1194 Category:$$Hotel (…) ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisA7723 Category:$Motel (…) ID:$$$$$$$$alifparis1 Name:$$$$$$Hotel$Alif Price:$$$$$30€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$35192900912 Animals:$$$No Postal:$$$$LisbonA1233 Category:$$Hotel (…) ID:$$$$$$$$tivolilx1 Name:$$$$$$Hotel$Tivoli Price:$$$$$90€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$351923821911 Animals:$$$No Postal:$$$$LisbonA1000 Category:$$Hotel (…)

A Z City 400€ 0€ Price

Space is still partitionable

slide-27
SLIDE 27

Motivation: Multi-Dimensional Systems

ID:$$$$$$$$hiltonparis1 Name:$$$$$$Hotel$Hilton Price:$$$$$250€ City:$$$$$$Paris Stars:$$$$$5 Phone:$$$$$87162801020 Animals:$$$No Postal:$$$$ParisA1291 Category:$$Hotel (…) ID:$$$$$$$$ibisparis1 Name:$$$$$$Hotel$Ibis Price:$$$$$50€ City:$$$$$$Paris Stars:$$$$$3 Phone:$$$$$87188190021 Animals:$$$No Postal:$$$$ParisA1194 Category:$$Hotel (…) ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisA7723 Category:$Motel (…) ID:$$$$$$$$alifparis1 Name:$$$$$$Hotel$Alif Price:$$$$$30€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$35192900912 Animals:$$$No Postal:$$$$LisbonA1233 Category:$$Hotel (…) ID:$$$$$$$$tivolilx1 Name:$$$$$$Hotel$Tivoli Price:$$$$$90€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$351923821911 Animals:$$$No Postal:$$$$LisbonA1000 Category:$$Hotel (…)

A Z City 400€ 0€ Price Paris Lisbon

Items are mapped along the several dimensions

slide-28
SLIDE 28

Motivation: Multi-Dimensional Systems

ID:$$$$$$$$hiltonparis1 Name:$$$$$$Hotel$Hilton Price:$$$$$250€ City:$$$$$$Paris Stars:$$$$$5 Phone:$$$$$87162801020 Animals:$$$No Postal:$$$$ParisA1291 Category:$$Hotel (…) ID:$$$$$$$$ibisparis1 Name:$$$$$$Hotel$Ibis Price:$$$$$50€ City:$$$$$$Paris Stars:$$$$$3 Phone:$$$$$87188190021 Animals:$$$No Postal:$$$$ParisA1194 Category:$$Hotel (…) ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisA7723 Category:$Motel (…) ID:$$$$$$$$alifparis1 Name:$$$$$$Hotel$Alif Price:$$$$$30€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$35192900912 Animals:$$$No Postal:$$$$LisbonA1233 Category:$$Hotel (…) ID:$$$$$$$$tivolilx1 Name:$$$$$$Hotel$Tivoli Price:$$$$$90€ City:$$$$$$Lisbon Stars:$$$$$4 Phone:$$$$$351923821911 Animals:$$$No Postal:$$$$LisbonA1000 Category:$$Hotel (…)

A Z City 400€ 0€ Price Paris Lisbon

Items are mapped along the several dimensions

One hash per dimension

slide-29
SLIDE 29

Motivation: Multi-Dimensional Systems

ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisC7723 Category:$$Motel (…)

A Z City 400€ 0€ Price Paris 10€

Efficient search by Price and City

slide-30
SLIDE 30

Configuring Multi-Dimensional Systems

A Z City 400€ 0€ Price

Two dimensional space

slide-31
SLIDE 31

Configuring Multi-Dimensional Systems

A Z City 400€ 0€ Price Stars

Three dimensional space

slide-32
SLIDE 32

Configuring Multi-Dimensional Systems

127 configurations for 7 attributes:

  • Price, Postal, Animals, Stars, Category, Name, Phone
  • Postal, Animals, Stars, Category, Name, Phone
  • Price, Animals, Stars, Name, Phone
  • Price, Postal, Animals, Category
  • Stars, Name
  • Price, Stars
  • Animals
  • Price
  • …
slide-33
SLIDE 33

Configuring Multi-Dimensional Systems

A Z City 400€ 0€ Price Stars Category Apartment Hotel

What about even more subspaces?

slide-34
SLIDE 34

Configuring Multi-Dimensional Systems

4149 configurations for 7 attributes:

  • <Price, Postal, Animals, Name, Phone>, <Stars>
  • <Postal, Stars, Category, Name, Phone>, <Price>
  • <Price, Animals, Stars, Name, Phone>
  • <Price, Postal, Animals, Category>
  • <Stars, Name>, <Postal, Animals>
  • <Animals>, <Stars, Name>
  • <Price>, <Stars>, <Animals>
  • …
slide-35
SLIDE 35

Performance is highly dependent on configuration

1 2 3 4 5 6 7 PL EQ PE desempenho (1000 ops / s) padrão de carga hyperspace 47x 27x 8x subspaces

RH BAL WH Workload Pattern throughput (1000 ops / sec)

smart choice naive choice

naive = one hyperspace with all attributes

slide-36
SLIDE 36

Objectives

  • 1. Understand multi-dimensional systems
  • 2. Automatize their configuration
  • 3. Case study of HyperDex [SIGCOMM12]
slide-37
SLIDE 37

Objectives

  • 1. Understand multi-dimensional systems
  • 2. Automatize their configuration
  • 3. Case study of HyperDex [SIGCOMM12]

Contributions

  • 1. Predictive Analytical Model for HyperDex
  • 2. Architecture for HyperDex reconfiguration
slide-38
SLIDE 38

Modelling: Search Operations

Intuition: cost proportional to the number of regions contacted

slide-39
SLIDE 39

Modelling: Search Operations

A Z City 400€ 0€ Price

slide-40
SLIDE 40

Modelling: Search Operations

A Z City 400€ 0€ Price Paris 10€

Fully specified space searches are cheap

slide-41
SLIDE 41

Modelling: Search Operations

Fully specified space searches are cheap

A Z City 400€ 0€ Price Paris 10€

slide-42
SLIDE 42

Modelling: Search Operations

Partly specified space searches contact more regions

A Z City 400€ 0€ Price Paris

slide-43
SLIDE 43

Modelling: Search Operations

Partly specified space searches contact more regions

A Z City 400€ 0€ Price Paris

slide-44
SLIDE 44

Modelling: Search Operations

Searches non-specified on subspaces are expensive.

A Z City 400€ 0€ Price

slide-45
SLIDE 45

Modelling: Search Operations

For any configuration Si and search query Qi

where

cost

  • expected number of contacted regions (CR):
  • expected throughput, function of CR:
slide-46
SLIDE 46

Modelling: Search Operations

For any configuration Si and search query Qi

where

cost

  • expected number of contacted regions (CR):
  • expected throughput, function of CR:
slide-47
SLIDE 47

Modelling: Update Operations

Intuition: cost proportional to chain that replicates the item

slide-48
SLIDE 48

Modelling: Update Operations

slide-49
SLIDE 49

Modelling: Update Operations

fetch first no blind writes

slide-50
SLIDE 50

Modelling: Update Operations

fetch first no blind writes length of subspace chain

slide-51
SLIDE 51

Modelling: Update Operations

fetch first no blind writes length of subspace chain replication degree

  • f the chain
slide-52
SLIDE 52

Modelling: Update Operations

A Z A Z City 400€ 0€ Price Stars Category Apartment Hotel

ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisC7723 Category:$$Motel (…)

slide-53
SLIDE 53

Modelling: Update Operations

A Z A Z City 400€ 0€ Price Stars Category Apartment Hotel

ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisC7723 Category:$$Motel (…)

slide-54
SLIDE 54

Modelling: Update Operations

A Z A Z City 400€ 0€ Price Stars Category Apartment Hotel

ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisC7723 Category:$$Motel (…)

slide-55
SLIDE 55

Modelling: Update Operations

A Z A Z City 400€ 0€ Price Stars Category Apartment Hotel

ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$1 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisC7723 Category:$$Motel (…)

slide-56
SLIDE 56

Modelling: Update Operations

A Z A Z City 400€ 0€ Price Stars Category Apartment Hotel

ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$5 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisC7723 Category:$$Motel (…)

slide-57
SLIDE 57

Modelling: Update Operations

A Z A Z City 400€ 0€ Price Stars Category Apartment Hotel

ID:$$$$$$$$sleazyparis1 Name:$$$$$$Motel$Sleazy Price:$$$$$10€ City:$$$$$$Paris Stars:$$$$$5 Phone:$$$$$871898283033 Animals:$$$Yes Postal:$$$$ParisC7723 Category:$$Motel (…)

slide-58
SLIDE 58

Precision of the Model

Workloads

  • A: specific searches
  • B: wide searches

Variants

  • RH: Read
  • BAL: Balanced
  • WH: Write
slide-59
SLIDE 59

Precision of the Model

Workloads

  • A: specific searches
  • B: wide searches

Variants

  • RH: Read
  • BAL: Balanced
  • WH: Write

Deployment similar to the original in SIGCOMM 2012

  • Data-set from

http://www.hotelsbase.org/

slide-60
SLIDE 60

Precision of the Model

Real Throughput (1000 ops / sec) Estimated Throughput (1000 ops / sec)

A-RH A-BAL A-WH A-RH A-BAL A-WH

slide-61
SLIDE 61

Re-configuring HyperDex

Analisador Oráculo Configurador Operações Parametros do modelo Perfil das

  • perações

Configuraçao seleccionada Operations Analyser Profile of the operations Oracle Parameters Model

a a

Selected configuration Configurator

slide-62
SLIDE 62

Evaluation

1 2 3 4 5 6 7 A-RH A-BAL A-WH B-RH B-BAL B-WH throughput (1000 ops / s) workload no-subspace dominant hyperspace automatic

slide-63
SLIDE 63

Evaluation

1 2 3 4 5 6 7 A-RH A-BAL A-WH B-RH B-BAL B-WH throughput (1000 ops / s) workload no-subspace dominant hyperspace automatic

slide-64
SLIDE 64

Conclusions

  • Solution to autonomically configure HyperDex
  • Predictive model of its throughput
  • Precision of the model of 92%
  • Easily wins over static heuristics
slide-65
SLIDE 65

Thank You

Questions?

Nuno Diegues, Muhammet Orazov, João Paiva, Luís Rodrigues, Paolo Romano

slide-66
SLIDE 66
slide-67
SLIDE 67