STRONG WHEN BAD THINGS DO NOT COME IN THREES ZECHAO SHANG JEFFREY - - PowerPoint PPT Presentation

▶

Jul 15, 2023 362 likes •577 views

MY WEAK CONSISTENCY IS STRONG WHEN BAD THINGS DO NOT COME IN THREES ZECHAO SHANG JEFFREY XU YU DISCLAIMER: NOT AN OLTP TALK HOW TO GET ALMOST EVERYTHING FOR NOTHING SHARED-MEMORY SYSTEM IS BACK shared data Fine-grained mini-jobs

SLIDE 1

MY WEAK CONSISTENCY IS

STRONG

WHEN BAD THINGS DO NOT COME IN THREES

ZECHAO SHANG JEFFREY XU YU

SLIDE 2

HOW TO GET ALMOST EVERYTHING FOR NOTHING

DISCLAIMER: NOT AN OLTP TALK

SLIDE 3

SHARED-MEMORY SYSTEM IS BACK

Fine-grained mini-jobs
Hard to batch
Low-latency in-place updates
Hard to partition the data space
Applications
Machine learning (SGD and others)
Graph computing (Vertex-centric systems)
Streaming (S-Store)

mini-jobs

in-place update compute shared data time Correct behavior Atomic and isolation Serializability theory

(transaction)

reads

SLIDE 4

THROUGHPUT

SCALABILITY LATENCY

DATA CONSISTENCY & JOB ISOLATION

SLIDE 5

Approach: remove data consistency controller
Pros: super-fast, yeeeeh!
Cons: could cause data consistency issues
HogWild! & Parameter Server & others
Correctness proofs rely on special properties
Convexity
Lipschitz-continuity
Bounded staleness
PBS: Probabilistic Bounded Staleness
Weak consistency actually provides strong semantics
Single key only
Probabilistic

DO WE NEED IT?

Correct Behavior Run the transactions like serial Atomic and isolation Serializability theory

SLIDE 6

THE DATABASE WAY

Fewer assumptions, more applications
Non-convex (deep learning)
Discrete & combinational (graph problems)

Correct Behavior Run the transactions like serial Atomic and isolation Serializability theory Run the transactions + anomalies Weak consistency This talk ? Behavior

SLIDE 7

DATA CONFLICT GRAPH

Each vertex represents a txn
An edge if two txns share data
Potential conflicts

1 7 5 4 2 6 3 8

SLIDE 8

GOOD AND BAD

Good No. 1: serial execution

time 1 7 5 4 2 6 3 8 1 2 8 7 6 5 4 3

SLIDE 9

GOOD AND BAD

Good No. 2: a nice scheduler
No direct edge in concurrent txns

time 1 7 5 4 2 6 3 8 1 2 8 7 6 5 4 3 1 2 8 7 6 5 4 3 dependency graph

SLIDE 10

GOOD AND BAD

Bad: potential conflict

1 7 5 4 2 6 3 8 time 1 3 4 8 6 7 5 2

Bad degree (for a transaction)
# of potential conflict transactions
Concurrent
Share same data (adjacent in graph)

BD=2 BD=1 1 2 8 7 6 5 4 3 1 2 8 7 6 5 4 3 time time BD=0 always

SLIDE 11

BAD DEGREE AND CORRECTNESSES

MAX BAD DEGREE CONCURRENCY CONTROL TXN SEMANTICS RESULTS ACCURACY NO SERIALIZABILITY CORRECT >0 NO NO DON’T KNOW YES SERIALIZABILITY CORRECT

SLIDE 12

BAD THINGS DO NOT COME IN 3 (BN3)

BN3: bad degree ≤1 for all transactions

MAX BAD DEGREE CONCURRENCY CONTROL TXN SEMANTICS RESULTS ACCURACY NO SERIALIZABILITY CORRECT 1 (BN3) >1 NO NO DON’T KNOW YES SERIALIZABILITY CORRECT

SLIDE 13

IS BN3 TRUE?

Depends on
Data conflict severity: the density of data conflict graph |"|

|#| $

Job type
Access pattern

GRAPH NAME |V| (in 106) |E| (in 106) DENSITY (in 10-4) Web Graphs uk-2007-05 106 3,739 4.2 uk-2014 787 47,614 4.7 eu-2015 1,070 91,792 5.8 claw-2012 3,563 128,736 1.4 Social Networks wise 59 265 4.0 friendster 66 1,806 0.7 TPC-C New Order >1000 1 7 5 4 2 6 3 8

SLIDE 14

BAD DEGREE DISTRIBUTION

≦1 BN3 (bad degree ≦ 1) 0BD (bad degree = 0)

0.90 0.92 0.94 0.96 0.98 0.99 1.00 1 2 4 8 1 6 3 2 6 4 1 2 8 2 5 6 5 1 2 1 2 4 Probability Number of Cores

conflict graph density:10-6 conflict graph density:10-5 conflict graph density:10-4

0.90 0.92 0.94 0.96 0.98 0.99 1.00 1 2 4 8 1 6 3 2 6 4 1 2 8 2 5 6 5 1 2 1 2 4 Probability Number of Cores

conflict graph density: 10-6 conflict graph density: 10-5 conflict graph density: 10-4

SLIDE 15

WHAT GOOD IS BN3?

THE TRANSACTIONS EXECUTED WITHOUT ANY CONSISTENCY MECHANISM IS UNDER SNAPSHOT ISOLATION (SI)

SLIDE 16

PROOF: A TWO-STEP APPROACH

0. BN3 restricts the size of “mafia”
Two crews (vertices) at most
1. Only two bad transactions case
Proof by enumerating the type of edges
2. Other good transactions
Does not cause more cycles
Adjacent (non-bad) vertices: behind or after
Non-adjacent vertices: none of their business

1 7 4 2 3 8 5 6 bad edge

SLIDE 17

BAD DEGREE AND CORRECTNESSES

MAX BAD DEGREE CONCURRENCY CONTROL TXN SEMANTICS RESULTS ACCURACY NO SERIALIZABILITY CORRECT 1 (BN3) NO SNAPSHOT ISOLATION WRITE-SKEW ANY NO NO DON’T KNOW YES SERIALIZABILITY CORRECT

SLIDE 18

10-4 10-3 2*10-3 1*10-4 6*10-5 5*10-5 Residual Conflict graph density No Consistency Read Uncommitted Read Committed Serializability

128 cores

10-4 10-3 2*10-3 1*10-4 6*10-5 5*10-5 Residual Conflict graph density No Consistency Read Uncommitted Read Committed Serializability

0.90 0.92 0.94 0.96 0.98 0.99 1.00 1 2 4 8 1 6 3 2 6 4 1 2 8 2 5 6 5 1 2 1 2 4 Probability Number of Cores

density:2*10-3 density:1*10-4 density:6*10-5 density:5*10-5

256 cores

x: vary the conflict graph density lines: vary isolation levels y: residual after 50 iterations of Page Rank

“BN3-ness”

SLIDE 19

TAKE HOME MESSAGES

Life is not just all-or-nothing
Flawlessness costs a lot
It is possible to have almost everything for free
BN3: realistic assumption, practical conclusion
Some future works
Runtime: monitor the BN3-ness
BN3 as a new consistency level
Mixed concurrency control

SLIDE 20

Thank you

SLIDE 21

EXPERIMENTAL STUDIES (THROUGHPUT)

40 60 2*10-3 1*10-4 6*10-5 5*10-5 Throughout (* 106) Conflict graph density No Consistency Read Uncommitted Read Committed Serializability 40 60 80 2*10-3 1*10-4 6*10-5 5*10-5 Throughout (* 106) Conflict graph density No Consistency Read Uncommitted Read Committed Serializability