Byzantine Fault Tolerance and Partial Synchrony Stefan Stattelmann - - PowerPoint PPT Presentation

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance

Byzantine Fault Tolerance and Partial Synchrony

Stefan Stattelmann

Seminar Advanced Topics in Distributed Computing WS 2007/2008, MPI-SWS (Saarland University), Petr Kuznetsov

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance

1 Introduction

Consensus and Synchrony Notation

2 Consensus and Partial Synchrony

Basic Protocol Protocols for partial synchrony Results

3 Practical Byzantine Fault Tolerance

BFT protocol View Changes and Recovery Results

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance Consensus and Synchrony Notation

Protocols to achieve agreement in synchronous distributed system are well-known Synchrony does not hold in practice Agreement in an asynchronous system is impossible Solution: Partial synchrony

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance Consensus and Synchrony Notation

Notation: N: total number of processors/servers participating in protocol f : maximum number of faulty participants N ≥ 3f + 1 holds All protocols achieve achieve optimal resiliency: N = 3f + 1

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance Basic Protocol Protocols for partial synchrony Results

First Paper

Cynthia Dwork, Nancy A. Lynch, Larry J. Stockmeyer: Consensus in the presence of partial synchrony. Content Agreement protocols for various fault types and different notions of synchrony Presentation only covers (authenticated) byzantine faults Reasons for partial synchrony Delay ∆ for message transmission Some processors are Φ times faster than others ∆ and Φ have to be bounded ∆ and Φ might not hold all the time exact value can be unknown

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance Basic Protocol Protocols for partial synchrony Results

Every processor pi stores some initial value v out of a value domain V PROPER ⊆ V , set of proper values locks on some values a message buffer Proper Values: if there is exactly one initial value, this is the only proper one

• therwise every v ∈ V is proper

A value v is called acceptable for pj if pj does not have a lock on v. Every processor signs its message with an unforgeable signature and attaches its PROPER set and its initial value.

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance Basic Protocol Protocols for partial synchrony Results

Message exchange in synchronous rounds, divided in three subrounds: send: every processor can send messages to any number of other processors receive: receive a subset of the messages sent in previous subround compute: state transition based on the received messages Assumption: There is some round GST (global stabilization time) such that all messages that are sent during or after GST arrive in the same round they were sent. Lost message are not considered as faulty behaviour.

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance Basic Protocol Protocols for partial synchrony Results

Protocol is divided into phases with two subphases trying: 3 rounds lock-release 1 round

• ne phase takes exactly 4 rounds

In phase k processor pi with i = k mod N is the processor to propose a value. Locks on values have an associated phase number. If pj has lock on value v for phase k ⇒ pj thinks pi might decide v.

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance Basic Protocol Protocols for partial synchrony Results

Round 1

Processors send a list of all values that are proper and acceptable. If a processor pj receives claims from t + 1 other processors that a value v is in their PROPER set, it adds v to PROPER. If pj receives intial values from more than 2t + 1 processors among there are not t + 1 equal values, pj sets PROPER = V .

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance Basic Protocol Protocols for partial synchrony Results

Round 1

Processors send a list of all values that are proper and acceptable. If a processor pj receives claims from t + 1 other processors that a value v is in their PROPER set, it adds v to PROPER. If pj receives intial values from more than 2t + 1 processors among there are not t + 1 equal values, pj sets PROPER = V .

Round 2

Processor pi proposes a value v that is acceptable for N − f processors (proof of acceptability included) Processor receiving a valid proposal lock v, previous locks on v are released

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance Basic Protocol Protocols for partial synchrony Results

Round 2

Processor pi proposes a value v that is acceptable for N − f processors (proof of acceptability included) Processor receiving a valid proposal lock v, previous locks on v are released

Round 3

Every processor that locked v sends an acknowledged message to pi pi decides v if it receives more than 2f + 1 replies

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance Basic Protocol Protocols for partial synchrony Results

Round 2

Processor pi proposes a value v that is acceptable for N − f processors (proof of acceptability included) Processor receiving a valid proposal lock v, previous locks on v are released

Round 3

Every processor that locked v sends an acknowledged message to pi pi decides v if it receives more than 2f + 1 replies

Round 4

Every processor multicasts on which values it has locks. A processors releases a lock if another processor has a lock on a different value.

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance Basic Protocol Protocols for partial synchrony Results

∆ holds eventually:

• ne round in synchronous algorithm is simulated by N + ∆ steps

first N steps: send messages as previously last ∆ steps: receive messages each round gets a unique identifier, attached to every sent message (easy: processors share a common clock) messages from earlier rounds are ignored ⇒ as soon as ∆ holds, message are delivered in the same round ⇒ after agreement was reached, ∆ can stop holding

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance Basic Protocol Protocols for partial synchrony Results

∆ holds, but is unknown: round r in synchronous algorithm is simulated by N + r steps first N steps: send messages as previously last r steps: receive messages each round gets a unique identifier, attached to every sent message (easy: processors share a common clock) messages from earlier rounds are ignored ⇒ as soon as ∆ >= r holds, messages are delivered in the same round

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance Basic Protocol Protocols for partial synchrony Results

Partially synchronous communication and processors: Processors have to agree on a (approximately) common notion of time Global clock simulated by private clocks Processors exchange with ticks and claims (with proofs) about ticks Private clocks are increased if there are enough valid claims to do so ⇒ creates a lot of overhead (2

3 of the rounds)

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance Basic Protocol Protocols for partial synchrony Results

Results: all protocols reach agreement in polynomial time all protocols have optimal resiliency (N = 3f + 1) But: practical application questionable specification of the protols could be more exact Consensus is possible in partially synchronous models!

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

Second and Third paper

Miguel Castro and Barbara Liskov: Proactive Recovery in a Byzantine-Fault-Tolerant System. Miguel Castro, Barbara Liskov: Practical Byzantine Fault Tolerance and Proactive Recovery. Content state machine replication algorithm for distributed services with complex operations basic version (BFT) for byzantine-fault-tolerant services extended version (BFT-PR) with automatic recovery use of symmetric cryptography for better performance

• ptimizations for frequent recovery

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

Assumptions about the replicas: all correct replicas executed the same service each replica identified by an integer common collision resistant cryptographic hash function to compute message digests message authentication codes (MACs) are used to authenticate all messages message log with garbage collection MACs: replicas share a secret session key with every other replica and every active client (refreshed dynamically) multicast messages use a vector of MACs (authenticators) MACs are update periodically messages with an old authentication are ignored

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

Assumptions for recovery replica stores a secure private key in hardware correct service code can be restored on reboot Assumptions about synchrony: message delay bounded by a linear function

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

Basic idea: client sends a request to all replicas all non-faulty replicas execute the request operation and reply with the result client waits for f+1 replies with same result Problem: Requests have to be executed in the same order by all replicas! ⇒ replicas agree on a sequence number for each request

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

Basic idea: client sends a request to all replicas all non-faulty replicas execute the request operation and reply with the result client waits for f+1 replies with same result Problem: Requests have to be executed in the same order by all replicas! ⇒ replicas agree on a sequence number for each request Solution: Primary-backup mechanism replicas move through a succession of views replica pi is the primary for view v if i = v mod N, the others are the backups primary picks the ordering But: Primary could be faulty! ⇒ change view

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

Client sends request to all replicas.

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

Primary multicasts a pre-prepare message containing: its current view digest of request sequence number This information is also added to all

• ther messages related to finding a

sequence number.

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

Backup ’acknowledges’ by multicasting a prepare message if it has not accepted pre-prepared for this view and sequence number sequence number is between a lower and upper bound

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

Request is prepared if replica has collected 2f + 1 pre-prepare messages 2f prepare messages ⇒ multicast commit message

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

All replicas execute request and send result. Client waits fo f + 1 identical replies.

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

Certificates weak certificate: set of f + 1 signed messages quorum certificate: set of 2f + 1 signed messages Replicas and clients can construct certificates. Exchange not possible (easily)! ⇒ MACs

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

View change challenges ensure total order of requests (details omitted) convince backups of view change hard ⇒ no certificate exchange

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

View Change: Backup considers primary faulty: enter view v + 1 multicast view-change message attaches 2 sets for prepared and pre-prepared requests

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

View Change: Replicas collect view-change messages for view v + 1. Accepted only if attached sets contain requests for view v or earlier. Acknowledge view-change to new primary ⇒ primary can prove authenticity

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

View Change: New primary collects view-change and view-change-ack messages ”reconstructs” the next sequence number from attached sets (details: Section 4.5 of [3]) constructs quorum certificate S for view-changes S = { (i, d) | replica i sent view-change with digest d} multicast new-view message that contains this state and S

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

View Change: Replicas wait for new-view wait for view-change messages in certificate check state of primary correct: use state faulty: change view

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

BFT-PR: proactive recovery force replicas to reboot periodically save state on disk, eventually trigger a view-change reboot with correct code from read-only hard-drive renegotiate session keys check and update state Critical for performance reboot (LinuxBIOS, now coreboot) state transfer (”hash tree” ⇒ blackboard)

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

Properties of the algorithm safety and liveness safety for arbitrary number of faulty clients:

• perations performed by faulty clients are observed in a consistent

way by non-faulty clients formal correctness proof for safety exists liveness: correct clients eventually receive a reply privacy not guaranteed (faulty replica might leak information)

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

Practical results BFT and BFT-PR were implemented in a general purpose library BFS: Byzantine-Fault-Tolerant File System comparison of BFS with other NFS implementations showed good performance (overhead ≤ 30%)

• nly tested for f = 1 and f = 2

reboots were only simulated ⇒ room for improvements!

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

Thank you for your attention! Questions?

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony

Introduction Consensus and Partial Synchrony Practical Byzantine Fault Tolerance BFT protocol View Changes and Recovery Results

Cynthia Dwork, Nancy A. Lynch, Larry J. Stockmeyer: Consensus in the presence of partial synchrony. J. ACM 35(2): 288-323 (1988) Miguel Castro and Barbara Liskov. Proactive Recovery in a Byzantine-Fault-Tolerant System. OSDI 2000. Miguel Castro, Barbara Liskov. Practical Byzantine Fault Tolerance and Proactive Recovery. ACM TOCS, 2002. Miguel Castro. Practical Byzantine Fault Tolerance (Ph. D. thesis)

Stefan Stattelmann Byzantine Fault Tolerance and Partial Synchrony