Databases 2 - Optional Presentation Andrea Gussoni Politecnico di - - PowerPoint PPT Presentation

▶
databases 2 optional presentation
SMART_READER_LITE
LIVE PREVIEW

Databases 2 - Optional Presentation Andrea Gussoni Politecnico di - - PowerPoint PPT Presentation

Feb 14, 2023 561 likes •1.04k views

Databases 2 - Optional Presentation Andrea Gussoni Politecnico di Milano July 15, 2016 Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 1 / 39 Coordination Avoidance Table of Contents Coordination

slide-1
SLIDE 1

Databases 2 - Optional Presentation

Andrea Gussoni

Politecnico di Milano

July 15, 2016

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 1 / 39

slide-2
SLIDE 2

Coordination Avoidance

Table of Contents

1

Coordination Avoidance

2

Trekking Through Siberia

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 2 / 39

slide-3
SLIDE 3

Coordination Avoidance

Some information on the paper

Title: Coordination Avoidance in Database Systems. Authors: Peter Bailis, Alan Fekete, Michael J. Franklin, Ali Ghodsi, Joseph M. Hellerstein, Ion Stoica. Presented at 2015 VLDB.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 3 / 39

slide-4
SLIDE 4

Coordination Avoidance

The Problem

At the present time, Database Systems in a distributed scenario are increasingly common. This means that the task of coordinating different entities is assuming a lot of importance.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 4 / 39

slide-5
SLIDE 5

Coordination Avoidance

The Problem

Usually concurrency control protocols are necessary because we want to guarantee the consistency of the application level data through the use of a database layer that check and solve the possible problems and conflicts. An example can be the use of a 2PL serialization technique that is often used in commercial DBMS.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 5 / 39

slide-6
SLIDE 6

Coordination Avoidance

The Problem

Mixing this with a distributed scenario means the necessity to introduce complex algorithms (such as 2PC) that coordinate the various entities involved in the transactions, introducing latency. Coordination also means that we cannot exploit all the parallel resources of a distributed environment, because we have a huge overhead introduced by the coordination phase.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 6 / 39

slide-7
SLIDE 7

Coordination Avoidance

The Problem

Usually we pay coordination overhead in term of: Increased latency. Decreased throughput. Unavailability (in case of failures).

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 7 / 39

slide-8
SLIDE 8

Coordination Avoidance

The Problem

Figure: Microbenchmark performance of coordinated and coordination-free execution on eight separate multi-core servers.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 8 / 39

slide-9
SLIDE 9

Coordination Avoidance

Invariant Confluence

The authors of the paper discuss this new technique (or better analysis framework) that if applied, it will reduce in a considerable way the need

  • f coordination between the Database entities, reducing the cost in terms
  • f bandwidth and latency, increasing considerably the overall throughput
  • f the system.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 9 / 39

slide-10
SLIDE 10

Coordination Avoidance

Invariant Confluence

The main idea here is not to introduce some new exotic way to improve the coordination task, but instead the authors predicate on the fact that there is a set of workloads that do not require coordination, and that can be executed in parallel. The programmer at the application level can then state in an explicit way the invariants, special attributes of the tables that need coordination in case of concurrent operations executing on them.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 10 / 39

slide-11
SLIDE 11

Coordination Avoidance

The Model

The main concepts introduced: Invariants

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 11 / 39

slide-12
SLIDE 12

Coordination Avoidance

The Model

The main concepts introduced: Invariants Transactions

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 11 / 39

slide-13
SLIDE 13

Coordination Avoidance

The Model

The main concepts introduced: Invariants Transactions Replicas

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 11 / 39

slide-14
SLIDE 14

Coordination Avoidance

The Model

The main concepts introduced: Invariants Transactions Replicas (I-)Convergence

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 11 / 39

slide-15
SLIDE 15

Coordination Avoidance

The Model

The main concepts introduced: Invariants Transactions Replicas (I-)Convergence Merging

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 11 / 39

slide-16
SLIDE 16

Coordination Avoidance

Convergence

This is a figure that explains the main concept behind the idea of convergence:

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 12 / 39

slide-17
SLIDE 17

Coordination Avoidance

Coordination-Free Execution

Here instead we show the basic evolution of a simple coordination free execution and the consequent merging operation:

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 13 / 39

slide-18
SLIDE 18

Coordination Avoidance

Invariants

It is important to note that coordination can only be avoided if all local commit decisions are globally valid.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 14 / 39

slide-19
SLIDE 19

Coordination Avoidance

Invariants

It is important to note that coordination can only be avoided if all local commit decisions are globally valid. So the best approach to guarantee the application level consistency is to apply a convergence analysis and then identify the true conflicts. The uncertain situations must be threated in a conservative approach.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 14 / 39

slide-20
SLIDE 20

Coordination Avoidance

Invariants

It is important to note that coordination can only be avoided if all local commit decisions are globally valid. So the best approach to guarantee the application level consistency is to apply a convergence analysis and then identify the true conflicts. The uncertain situations must be threated in a conservative approach. This means that we rely on the analysis done by the programmer at the application level to guarantee the correctness. This is clearly a drawback.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 14 / 39

slide-21
SLIDE 21

Coordination Avoidance

Invariants

Luckily there are some standard situations for the analysis of invariants that we can use as boilerplate in the building of the set of invariants of

  • ur application, this figure summarizes the main cases:

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 15 / 39

slide-22
SLIDE 22

Coordination Avoidance

Benchmarking

The authors then proceeded to implement this new framework and test it with a standard benchmark, the TPC-C benchmark, that is said to be “the gold standard for database concurrency control both in research and industry.” They also used RAMP transactions, that are transactions that “employ limited multi-versioning and metadata to ensure that readers and writers can always proceed concurrently.” The selected language for the prototype is Scala, used for reason of compactness of the code.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 16 / 39

slide-23
SLIDE 23

Coordination Avoidance

Benchmarking

In the next few slides there are some plots of the results obtained by the

  • authors. The New-Order label refers to the fact that the authors, when

an unique id assignment was needed, decided to assign a temp-ID, and

  • nly just before the commit, a sequential (as required from the

specifications of the benchamrk) real-ID is assigned, and a table mapping tmp-ID to real-ID is created.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 17 / 39

slide-24
SLIDE 24

Coordination Avoidance

Results

Figure: TPC-C New-Order throughput across eight servers.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 18 / 39

slide-25
SLIDE 25

Coordination Avoidance

Results

Figure: Coordination-avoiding New-Order scalability.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 19 / 39

slide-26
SLIDE 26

Coordination Avoidance

Conclusions

This paper demonstrates that ACID transactions and associated strong isolation levels dominated the field of database concurrency. This is a powerful abstraction that automatically guarantee consistency at the application level. In a distributed scenario where we want to achieve high scalability, we can sacrifice these abstractions and perform an I-Confluence analysis in order to exploit scalability through coordination-free transactions

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 20 / 39

slide-27
SLIDE 27

Trekking Through Siberia

Table of Contents

1

Coordination Avoidance

2

Trekking Through Siberia

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 21 / 39

slide-28
SLIDE 28

Trekking Through Siberia

Some information on the paper

Title: Trekking Through Siberia: Managing Cold Data in a Memory-Optimized Database. Authors: Ahmed Eldawy, Justin Levandoski, Per-Ake Larson. Presented at 2014 VLDB.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 22 / 39

slide-29
SLIDE 29

Trekking Through Siberia

Introduction

With the drop in memory prices a set of in main memory database

  • emerged. While for most of OLTP workloads often this solution is

reasonable, due to the fact that often databases exhibit a skewed access patterns that divide records in hot (frequently accessed) and cold (rarely accessed) it is still convenient to find a way to maintain the hot records in memory and the cold ones on for example flash storage, that is still a lot less expensive than memory.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 23 / 39

slide-30
SLIDE 30

Trekking Through Siberia

Introduction

In this paper it is presented Project Siberia, an extension to the Hekaton engine of Microsoft SQL Server that aims to pursue these objectives: Cold data classification. Cold data storage. Cold storage access reduction. Cold data access and migration mechanism (the focus of this paper is on this aspect).

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 24 / 39

slide-31
SLIDE 31

Trekking Through Siberia

Hekaton

This figure shows how the storage and indexing is done in Hekaton:

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 25 / 39

slide-32
SLIDE 32

Trekking Through Siberia

Hekaton

Hekaton utilizes optimistic multi-version concurrency control (MVCC), it mainly leverage these features of timestamps to obtain this: Commit/End Time (useful to determine the serialization order). Valid Time. Logical Read Time (start time of the transaction).

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 26 / 39

slide-33
SLIDE 33

Trekking Through Siberia

Some important data structures

Figure: Structure of a record in the cold store. Figure: Structure of a cached record. Figure: Structure of a timestamp notice in the update memo.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 27 / 39

slide-34
SLIDE 34

Trekking Through Siberia

The main operations on the cold storage

Insert: done always in the hot storage. Migration to cold storage: is the only way to move a record from the hot storage to the cold one. Delete. Updates: a delete operation on the cold storage followed by an insertion in the hot storage. Read. Update Memo and Cold Store Cleaning.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 28 / 39

slide-35
SLIDE 35

Trekking Through Siberia

Focus on migration

Figure: Contents of cold store, hot store, and update memo during migration of a record.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 29 / 39

slide-36
SLIDE 36

Trekking Through Siberia

Focus on delete

Figure: Effect on the cold store and update memo of a record deletion.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 30 / 39

slide-37
SLIDE 37

Trekking Through Siberia

Observations

We need a new phase called validation, that checks just before a commit action that all the records used during the transactions still exist, are valid and have not been modified by another concurrent transaction. There is no deletion in the strict sense of the term. The to-delete records have they end timestamps changed, and the garbage collection remove the unused records (when all the transactions alive begun after the deletion).

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 31 / 39

slide-38
SLIDE 38

Trekking Through Siberia

Benchmarks

The authors utilized two types of benchmarks: YCSB Benchmark (50GB single-table database, 1KB records) that is divided in:

Read-heavy: 90% reads and % updates. Write-heavy: 50% reads and 50% writes. Read-only: 100% reads.

Multi-step read/update workload (1GB single-table database, 56 bytes records) that is divided in:

Read-only. Update-only.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 32 / 39

slide-39
SLIDE 39

Trekking Through Siberia

In-Memory Cold storage

This analysis is done in order to isolate the overhead strictly caused by the Siberia framework, eliminating the latency of the I/O operations:

Figure: In-memory overhead of the Siberia framework.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 33 / 39

slide-40
SLIDE 40

Trekking Through Siberia

Migration

This analysis instead focuses on the performance degradation of various types of workload during a live migration to the cold storage of parts of the database:

Figure: In-memory overhead of the Siberia framework.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 34 / 39

slide-41
SLIDE 41

Trekking Through Siberia

Read-only workload with I/O

This analysis focuses on the performance degradation of a read-only workload with cold storage on flash (a similar analysis has been done for an update-only workload):

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 35 / 39

slide-42
SLIDE 42

Trekking Through Siberia

Read-only workload with I/O

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 36 / 39

slide-43
SLIDE 43

Trekking Through Siberia

YCSB benchmark

Figure: YCSB write-heavy workload.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 37 / 39

slide-44
SLIDE 44

Trekking Through Siberia

Conclusions

There is related research in progress in this direction: Buffer pool: page indirection on disk. HyPer: hybrid between OLTP and OLAP, optimizes data in chunks using different virtual memory pages, finds the cold data and compress for OLAP usage.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 38 / 39

slide-45
SLIDE 45

Trekking Through Siberia

Conclusions

There is related research in progress in this direction: Buffer pool: page indirection on disk. HyPer: hybrid between OLTP and OLAP, optimizes data in chunks using different virtual memory pages, finds the cold data and compress for OLAP usage. The approach used in Siberia has the great advantage to have an access at record level, and for databases where the cold storage is between 10% and 20% of the whole database it has the great advantage of not requiring additional structures in memory (except the compact bloom-filters) for the cold data.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 38 / 39

slide-46
SLIDE 46

Trekking Through Siberia

License

This work is licensed under a Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ or send a letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View, California, 94041, USA.

Andrea Gussoni (Politecnico di Milano) Databases 2 - Optional Presentation July 15, 2016 39 / 39