Early Scheduling in Parallel State Machine Replication Eduardo - PowerPoint PPT Presentation

� 1 Early Scheduling in Parallel State Machine Replication Eduardo Alchieri, Fernando Dotti, and Fernando Pedone Universidade de Brasilia, Pontifica Universidade Católica do Rio Grande do Sul, and University of Lugano

� 2 State Machine Replication (SMR) Fundamental approach to fault tolerance Google Spanner Apache Zookeeper Windows Azure Storage MySQL Group Replication Galera Cluster Blockchain, …

� 3 SMR is intuitive and simple same order deterministic execution R0 R1 R2 R0 R1 R2 R0 R1 R2 Clients Servers

� 4 Parallel State Machine Replication Key observation Independent requests can execute concurrently Conflicting requests must be serialized and executed in the same order by the replicas Two requests conflict if they access common state and at least one of them updates the state

� 5 Parallel State Machine Replication Late scheduling R1 worker R5 R3 R0 Scheduling happens after 
 requests are ordered scheduler R4 R2 worker Early scheduling Replica Scheduling decisions happen 
 before requests are ordered R 5 (x) R 3 (x) R 2 (x) R 0 (x) worker t x E.g., worker t x executes 
 requests on X, worker t y 
 R 4 (y) R 1 (y) scheduler executes requests on Y worker t y Replica

� 6 Scheduling tradeo ff Ideal High Late Scheduling Concurrency This paper Early Early Scheduling Scheduling Classic SMR Low Low High Synchronization Overhead

� 7 Our contributions Generalization of Early Scheduling Classes of requests: expressing application concurrency How to automatically map classes to worker threads How the resulting technique compares to late scheduling

� 8 Classes of requests C R Readers and writers External Class C R : read requests conflict Internal C W Class C W : write requests conflict

� 9 Mapping classes to workers Define workers that execute requests 
 t 0 , …, t k C R in the class Concurrent Define class type Sequential: one request at a time t k , …, t n C W Sequential Concurrent: requests executed 
 concurrently

� 10 Early Scheduling execution model Class C is CONCURRENT: request assigned to t 0 OR t 1 class ➝ workers worker t 0 mapping R 6 R 4 R 3 R 1 ordered requests R 1 , R 2 ,… in class C scheduler worker t 1 R 7 R 5 R 2 Replica

� 11 Early Scheduling execution model Class C is SEQUENTIAL: request assigned to t 0 AND t 1 class ➝ workers worker t 0 mapping R 2 R 1 ordered requests barrier R 1 , R 2 ,… in class C scheduler worker t 1 R 2 R 1 Replica

� 12 Mapping classes to workers Rule #5 Rule #3 Rule #4 Rule #2 Rule #1 Sequential C 1 Concurrent C 1 t 0 ,t 1 ,t 2 t 0 ,t 1 C 1 Sequential t 0 ,t 1 ,t 2 C 1 ➝ C W C 1 Sequential or t 3 C 2 ➝ Sequential C 2 C 2 t 2 ,t 3 ,t 4 t 0 ,t 1 ,t 2 Sequential C 2 Sequential If C 1 and C 2 conflict, and are If C 1 and C 2 conflict, C 1 is If C has internal conflicts, Every class must have at If C 1 and C 2 conflict, at concurrent, and C 2 is sequential, sequential, then C 1 and C 2 must least one must be sequential workers of C 1 are workers of C 2 then it must be sequential have one worker in common least one worker thread

� 13 Mapping classes to workers Sequential t 0 , t 1 , t 2 , t 3 C Wg Concurrent Concurrent Local reads most 
 C R1 C R2 t 2 , t 3 t 0 , t 1 common requests Workers: t 0 ,t 1 ,t 2 ,t 3 Sequential Sequential C W1 C W2 Synchronized t 0 , t 2 , t 3 t 0 , t 1 , t 2 C Rg Concurrent t 0 , t 2

� 14 Optimizing scheduling O 1a : Minimize workers in sequential classes O 1b : Maximize workers in concurrent classes O 2 : Assign workers to concurrent classes in proportion to class weight (i.e., more work, more workers) O 3 : Minimize unnecessary synchronization among classes

� 15 Optimization model . . . Described in AMPL Solved with KNitro

� 16 Naive vs Optimized mapping Sequential t 0 , t 1 , t 2 , t 3 C Wg Concurrent Concurrent Local reads most 
 C R1 C R2 t 0 , t 1 t 2 , t 3 common requests Workers: t 0 ,t 1 ,t 2 ,t 3 Sequential Sequential C W1 C W2 parallel t 0 , t 1 t 2 , t 3 C Rg Sequential t 0 , t 2

� 17 Experimental evaluation Prototype in BFT-SMaRt environment Early scheduling and late scheduling Configured to crash failures (not BFT) Linked-list application Single- and multi-shard deployments Light, moderate, and heavy execution costs Uniform and skewed workloads

� 18 Single-shard, reads, moderate

� 19 Multi-shard, mixed, moderate

� 20 http://www.inf.usi.ch/faculty/ pedone/