Bloxy: P roviding Transparent and Generic BFT-Based Ordering - - PowerPoint PPT Presentation

Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains

Symposium on Reliable Distributed Systems, October 4, 2019

Signe Rüsch, Kai Bleeke, Rüdiger Kapitza

ruesch@ibr.cs.tu-bs.de Technische Universität Braunschweig, Germany

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Blockchain Technology for Industry Usage

Transactions are stored decentralised and tamper-proof Permissioned blockchains:

Number of users and access to data regulated Interesting for companies and organisations Several use cases: supply chain management, distributed storage, …

Block n Hash h(n-1) Block n+1 Hash h(n) Block n+2 Hash h(n+1) tx11 tx12 … tx21 tx22 … tx1 tx2 …

2019-10-04 Signe Rüsch Page 2 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Blockchain Technology for Industry Usage

Transactions are stored decentralised and tamper-proof Permissioned blockchains:

Number of users and access to data regulated Interesting for companies and organisations Several use cases: supply chain management, distributed storage, …

Byzantine fault tolerant (BFT) protocols:

Consensus even with participants showing arbitrarily wrong behaviour Suitable for block creation

Block n Hash h(n-1) Block n+1 Hash h(n) Block n+2 Hash h(n+1) tx11 tx12 … tx21 tx22 … tx1 tx2 …

2019-10-04 Signe Rüsch Page 2 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Blockchain Technology for Industry Usage

Transactions are stored decentralised and tamper-proof Permissioned blockchains:

Number of users and access to data regulated Interesting for companies and organisations Several use cases: supply chain management, distributed storage, …

Byzantine fault tolerant (BFT) protocols:

Consensus even with participants showing arbitrarily wrong behaviour Suitable for block creation

Different use cases lead to different requirements for consensus!

Block n Hash h(n-1) Block n+1 Hash h(n) Block n+2 Hash h(n+1) tx11 tx12 … tx21 tx22 … tx1 tx2 …

2019-10-04 Signe Rüsch Page 2 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

BFT Protocols

Common structure:

Client sends request Replicas order and execute request Client performs majority voting on replies

2019-10-04 Signe Rüsch Page 3 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

BFT Protocols

Common structure:

Client sends request Replicas order and execute request Client performs majority voting on replies

Optimized for different aspects:

Scalability: Algorand [1] Throughput: OmniLedger [2] Latency: FastBFT [3] (A)Synchrony: HoneyBadgerBFT [4]

Algorand HoneyBadgerBFT FastBFT OmniLedger

Latency Throughput Synchrony Scalability

[1] Gilad et al., SOSP’17 [2] Kokoris-Kogias et al., IEEE S&P’18 [3] Liu et al., 2016 [4] Miller et al., CCS’16

2019-10-04 Signe Rüsch Page 3 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

BFT Protocols

Common structure:

Client sends request Replicas order and execute request Client performs majority voting on replies

Optimized for different aspects:

Scalability: Algorand [1] Throughput: OmniLedger [2] Latency: FastBFT [3] (A)Synchrony: HoneyBadgerBFT [4]

No one-size-fits-all BFT solution!

Algorand HoneyBadgerBFT FastBFT OmniLedger

Latency Throughput Synchrony Scalability

[1] Gilad et al., SOSP’17 [2] Kokoris-Kogias et al., IEEE S&P’18 [3] Liu et al., 2016 [4] Miller et al., CCS’16

2019-10-04 Signe Rüsch Page 3 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

BFT Protocols

Common structure:

Client sends request Replicas order and execute request Client performs majority voting on replies

Optimized for different aspects:

Scalability: Algorand [1] Throughput: OmniLedger [2] Latency: FastBFT [3] (A)Synchrony: HoneyBadgerBFT [4]

No one-size-fits-all BFT solution!

Algorand HoneyBadgerBFT FastBFT OmniLedger

Latency Throughput Synchrony Scalability

[1] Gilad et al., SOSP’17 [2] Kokoris-Kogias et al., IEEE S&P’18 [3] Liu et al., 2016 [4] Miller et al., CCS’16

Allow operators to choose protocol fitting to use case!

2019-10-04 Signe Rüsch Page 3 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Hyperledger Fabric

Permissioned blockchain platform [Androulaki et al., EuroSys’18] Modular design with pluggable consensus

Ordering Service 1 2 3 4 5 Client Peers OSNs

2019-10-04 Signe Rüsch Page 4 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Hyperledger Fabric

Permissioned blockchain platform [Androulaki et al., EuroSys’18] Modular design with pluggable consensus Transaction flow:

1

Peers execute transaction

2

Client collects endorsements

3

Client sends transaction to be

• rdered

4

Ordering Service Nodes (OSNs)

distribute blocks

5

Peers validate transactions

Ordering Service 1 2 3 4 5 Client Peers OSNs

2019-10-04 Signe Rüsch Page 4 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Ordering Services in Fabric

No fault tolerance: Solo Crash-fault tolerance: Apache Kafka Byzantine fault tolerance: BFT-SMaRt

[Sousa et al., DSN’18]

C P P P OSN R R R R O O O O C P P P OSN R ? R ? R ? R ? O O O O P: Peers C: Client R: Replica O: Orderer F: Frontend

2019-10-04 Signe Rüsch Page 5 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Ordering Services in Fabric

No fault tolerance: Solo Crash-fault tolerance: Apache Kafka Byzantine fault tolerance: BFT-SMaRt

[Sousa et al., DSN’18]

BFT integration showed Fabric’s shortcomings:

Client tasks compromise pluggability “Frontend”: Fabric orderer now part of peer

C F P F P F P F R R R R OSN C P P P OSN R ? R ? R ? R ? O O O O P: Peers C: Client R: Replica O: Orderer F: Frontend

2019-10-04 Signe Rüsch Page 5 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Ordering Services in Fabric

No fault tolerance: Solo Crash-fault tolerance: Apache Kafka Byzantine fault tolerance: BFT-SMaRt

[Sousa et al., DSN’18]

BFT integration showed Fabric’s shortcomings:

Client tasks compromise pluggability “Frontend”: Fabric orderer now part of peer Positioned on all peer nodes → Severely reduced modularity

C F P F P F P F R R R R OSN C P P P OSN R ? R ? R ? R ? O O O O P: Peers C: Client R: Replica O: Orderer F: Frontend

2019-10-04 Signe Rüsch Page 5 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Ordering Services in Fabric

No fault tolerance: Solo Crash-fault tolerance: Apache Kafka Byzantine fault tolerance: BFT-SMaRt

[Sousa et al., DSN’18]

BFT integration showed Fabric’s shortcomings:

Client tasks compromise pluggability “Frontend”: Fabric orderer now part of peer Positioned on all peer nodes → Severely reduced modularity Re-implementation for all protocols needed

C F P F P F P F R R R R OSN C P P P OSN R ? R ? R ? R ? O O O O P: Peers C: Client R: Replica O: Orderer F: Frontend

2019-10-04 Signe Rüsch Page 5 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Ordering Services in Fabric

No fault tolerance: Solo Crash-fault tolerance: Apache Kafka Byzantine fault tolerance: BFT-SMaRt

[Sousa et al., DSN’18]

BFT integration showed Fabric’s shortcomings:

Client tasks compromise pluggability “Frontend”: Fabric orderer now part of peer Positioned on all peer nodes → Severely reduced modularity Re-implementation for all protocols needed

C F P F P F P F R R R R OSN C P P P OSN R ? R ? R ? R ? O O O O

Goal: Shift BFT client functionality to replica nodes to maintain modularity!

P: Peers C: Client R: Replica O: Orderer F: Frontend

2019-10-04 Signe Rüsch Page 5 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Bloxy: Blockchain-Aware Trusted Proxy

Consensus proxy for Hyperledger Fabric Encapsulates BFT client functionality Transparent access to a generic ordering service

C P P P OSN R B R B R B R B O O O O

Ordering Checkpointing View Change Replica Secure Communication Majority Voting Request Invocation Block Dissemination BLOXY

2019-10-04 Signe Rüsch Page 6 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Bloxy: Blockchain-Aware Trusted Proxy

Consensus proxy for Hyperledger Fabric Encapsulates BFT client functionality Transparent access to a generic ordering service Leverage trusted execution environment (TEE)

Hybrid fault model: TEE can only fail by crashing

C P P P OSN R B R B R B R B O O O O

Ordering Checkpointing View Change Replica Secure Communication Majority Voting Request Invocation Block Dissemination BLOXY

2019-10-04 Signe Rüsch Page 6 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Bloxy: Blockchain-Aware Trusted Proxy

Consensus proxy for Hyperledger Fabric Encapsulates BFT client functionality Transparent access to a generic ordering service Leverage trusted execution environment (TEE)

Hybrid fault model: TEE can only fail by crashing

Integration of two BFT protocols:

PBFT [Castro et al., OSDI’99] Hybrid protocol Hybster [Behl et al., EuroSys’17]

C P P P OSN R B R B R B R B O O O O

Ordering Checkpointing View Change Replica Secure Communication Majority Voting Request Invocation Block Dissemination BLOXY

2019-10-04 Signe Rüsch Page 6 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Bloxy: Blockchain-Aware Trusted Proxy

Consensus proxy for Hyperledger Fabric Encapsulates BFT client functionality Transparent access to a generic ordering service Leverage trusted execution environment (TEE)

Hybrid fault model: TEE can only fail by crashing

Integration of two BFT protocols:

PBFT [Castro et al., OSDI’99] Hybrid protocol Hybster [Behl et al., EuroSys’17]

Functionality:

• 1. Establishing secure connections
• 2. Forwarding transactions as BFT requests
• 3. Majority voting on replies
• 4. Block dissemination

C P P P OSN R B R B R B R B O O O O

Ordering Checkpointing View Change Replica Secure Communication Majority Voting Request Invocation Block Dissemination BLOXY

2019-10-04 Signe Rüsch Page 6 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Trusted Execution: Intel SGX in a Nutshell

x86 instruction set extension introduced with Skylake architecture Creation of TEEs → enclaves Execution and data inside enclaves protected from privileged software Hardware-based memory integrity protection and encryption Remote attestation of enclaves Only CPU is trusted Application TEE / Enclave Operating System Hardware CPU

2019-10-04 Signe Rüsch Page 7 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Trusted Execution: Intel SGX in a Nutshell

x86 instruction set extension introduced with Skylake architecture Creation of TEEs → enclaves Execution and data inside enclaves protected from privileged software Hardware-based memory integrity protection and encryption Remote attestation of enclaves Only CPU is trusted Application TEE / Enclave Operating System Hardware CPU Intel SGX allows the creation of enclaves, trusted execution environments (TEEs) protected by hardware.

2019-10-04 Signe Rüsch Page 7 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Bloxy Transaction Flow

Orderer Peer Peer Replica BLOXY Replica BLOXY Orderer Peer Peer Orderer Replica BLOXY Client Peer 4 4 1 1 4 2 3 4

1

Transaction forwarded to Bloxy via orderer

2

Replicas order translated BFT request from Bloxy

3

Bloxies get block, exchange hashes, perform

majority voting

4

Blocks are disseminated to all connected nodes

2019-10-04 Signe Rüsch Page 8 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Bloxy Transaction Flow

Orderer Peer Peer Replica BLOXY Replica BLOXY Orderer Peer Peer Orderer Replica BLOXY Client Peer 4 4 1 1 4 2 3 4

1

Transaction forwarded to Bloxy via orderer

2

Replicas order translated BFT request from Bloxy

3

Bloxies get block, exchange hashes, perform

majority voting

4

Blocks are disseminated to all connected nodes

→ Transmit only hashes and one block per orderer

2019-10-04 Signe Rüsch Page 8 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Bloxy Transaction Flow

Orderer Peer Peer Replica BLOXY Replica BLOXY Orderer Peer Peer Orderer Replica BLOXY Client Peer 4 4 1 1 4 2 3 4

1

Transaction forwarded to Bloxy via orderer

2

Replicas order translated BFT request from Bloxy

3

Bloxies get block, exchange hashes, perform

majority voting

4

Blocks are disseminated to all connected nodes

→ Transmit only hashes and one block per orderer Lower overhead due to less and shorter messages!

2019-10-04 Signe Rüsch Page 8 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Bloxy Short-Term Block Storage

Fulfil Fabric’s requirements for ordering services No Skipping: all peers need all blocks

Byzantine orderers cannot manipulate blocks signed by Bloxy How to ensure Byzantine orderers deliver all blocks?

2019-10-04 Signe Rüsch Page 9 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Bloxy Short-Term Block Storage

Fulfil Fabric’s requirements for ordering services No Skipping: all peers need all blocks

Byzantine orderers cannot manipulate blocks signed by Bloxy How to ensure Byzantine orderers deliver all blocks?

Bloxy needs to ensure at least one correct orderer receives each block Block stored on replicas until f + 1 orderers acknowledge reception

2019-10-04 Signe Rüsch Page 9 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Bloxy Short-Term Block Storage

Fulfil Fabric’s requirements for ordering services No Skipping: all peers need all blocks

Byzantine orderers cannot manipulate blocks signed by Bloxy How to ensure Byzantine orderers deliver all blocks?

Bloxy needs to ensure at least one correct orderer receives each block Block stored on replicas until f + 1 orderers acknowledge reception Orderer can query Short-Term Block Storage:

Receive missing block → full round of BFT communication Receive list of acknowledged orderers to query

2019-10-04 Signe Rüsch Page 9 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Bloxy Short-Term Block Storage

Fulfil Fabric’s requirements for ordering services No Skipping: all peers need all blocks

Byzantine orderers cannot manipulate blocks signed by Bloxy How to ensure Byzantine orderers deliver all blocks?

Bloxy needs to ensure at least one correct orderer receives each block Block stored on replicas until f + 1 orderers acknowledge reception Orderer can query Short-Term Block Storage:

Receive missing block → full round of BFT communication Receive list of acknowledged orderers to query

Performance vs. fault tolerance → orderer’s behaviour is configurable

2019-10-04 Signe Rüsch Page 9 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Bloxy Short-Term Block Storage

Fulfil Fabric’s requirements for ordering services No Skipping: all peers need all blocks

Byzantine orderers cannot manipulate blocks signed by Bloxy How to ensure Byzantine orderers deliver all blocks?

Bloxy needs to ensure at least one correct orderer receives each block Block stored on replicas until f + 1 orderers acknowledge reception Orderer can query Short-Term Block Storage:

Receive missing block → full round of BFT communication Receive list of acknowledged orderers to query

Performance vs. fault tolerance → orderer’s behaviour is configurable Bloxy can guarantee all requirements for ordering services!

2019-10-04 Signe Rüsch Page 9 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Generic Consensus

Generality: Bloxy is not limited to one specific protocol Bloxy implemented as a library BFT frameworks can include the Bloxy library

2019-10-04 Signe Rüsch Page 10 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Generic Consensus

Generality: Bloxy is not limited to one specific protocol Bloxy implemented as a library BFT frameworks can include the Bloxy library Call Bloxy for trusted operations shared by most BFT frameworks:

Connection establishment Request translation Majority voting

No functional changes to BFT framework necessary

Core aspects are unchanged: ordering, checkpointing, view change

2019-10-04 Signe Rüsch Page 10 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Generic Consensus

Generality: Bloxy is not limited to one specific protocol Bloxy implemented as a library BFT frameworks can include the Bloxy library Call Bloxy for trusted operations shared by most BFT frameworks:

Connection establishment Request translation Majority voting

No functional changes to BFT framework necessary

Core aspects are unchanged: ordering, checkpointing, view change

Bloxy allows easy integration of arbitrary BFT protocols!

2019-10-04 Signe Rüsch Page 10 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Evaluation of Bloxy

4 machines for ordering service

SGX-capable 4-core Xeon v5 CPUs, 32GB RAM

5 machines as peer and orderer nodes

non-SGX 4-core Xeon v2 CPUs, 16GB RAM

10 Gbps interconnection (switched network) Ordering Service Component:

What is Bloxy’s impact on latency? What throughput can Bloxy achieve?

Complete Fabric network:

Does Bloxy affect the performance of Fabric?

2019-10-04 Signe Rüsch Page 11 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Throughput Evaluation of Ordering Services

1,000 5,000 10,000 15,000 20,000 5,000 10,000 15,000 20,000

+71%

• 20%

Send Rate [rps] Throughput [rps] Target Solo BFT-SMaRt Bloxy+Hybster Bloxy+PBFT

2019-10-04 Signe Rüsch Page 12 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Latency Evaluation of Ordering Services

1,000 5,000 10,000 15,000 20,000 100 200

• 14.3%

+99.5%

Send Rate [rps] Latency [ms] Solo BFT-SMaRt Bloxy+Hybster Bloxy+PBFT

2019-10-04 Signe Rüsch Page 13 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Latency Evaluation of Ordering Services

1,000 5,000 10,000 15,000 20,000 100 200

• 14.3%

+99.5%

Send Rate [rps] Latency [ms] Solo BFT-SMaRt Bloxy+Hybster Bloxy+PBFT

Lower communication overhead of Bloxy protocols, improved throughput and latency

2019-10-04 Signe Rüsch Page 13 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Throughput Evaluation of Fabric Network

100 500 1,000 1,500 500 1,000 1,500 Send Rate [rps] Throughput [rps] Target Solo BFT-SMaRt Bloxy+Hybster Bloxy+PBFT

Hyperledger Caliper with full Fabric network

12 clients, 2 peers, 3-4 replicas in ordering service Bloxy-backed protocols give comparable performance

2019-10-04 Signe Rüsch Page 14 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Latency Evaluation of Fabric Network

100 500 1,000 1,500 1 2 3 4 5 Send Rate [rps] Latency [s] Solo BFT-SMaRt Bloxy+Hybster Bloxy+PBFT

2019-10-04 Signe Rüsch Page 15 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Latency Evaluation of Fabric Network

100 500 1,000 1,500 1 2 3 4 5 Send Rate [rps] Latency [s] Solo BFT-SMaRt Bloxy+Hybster Bloxy+PBFT

Ordering service has no noticeable impact on Fabric’s performance Generic consensus at no performance cost!

→ Full evaluation of all ordering services in paper!

2019-10-04 Signe Rüsch Page 15 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Related Work

Other uses of Intel SGX as trusted proxy Troxy [Li et al., DSN’18]

Shift BFT client functionality to replica side Aid adoption of BFT protocols e. g. for SMTP and HTTP No changes to common protocols and client implementations necessary

LibSeal [Aublin et al., EuroSys’18]

Audit library for detecting integrity violations

VeritasDB [Sinha et al., IACR Cryptology ePrint’18]

Network proxy for key-value stores with integrity guarantees

2019-10-04 Signe Rüsch Page 16 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Related Work

Other uses of Intel SGX as trusted proxy Troxy [Li et al., DSN’18]

Shift BFT client functionality to replica side Aid adoption of BFT protocols e. g. for SMTP and HTTP No changes to common protocols and client implementations necessary

LibSeal [Aublin et al., EuroSys’18]

Audit library for detecting integrity violations

VeritasDB [Sinha et al., IACR Cryptology ePrint’18]

Network proxy for key-value stores with integrity guarantees

Bloxy tailored for blockchain use case and communication, integration into a permissioned blockchain with ordering service

2019-10-04 Signe Rüsch Page 16 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Conclusion

Bloxy’s contributions: Consensus proxy for Hyperledger Fabric Encapsulation of BFT client functionality

C P P P OSN R B R B R B R B O O O O

Orderer Peer Peer Replica BLOXY Replica BLOXY Orderer Peer Peer Orderer Replica BLOXY Client Peer 4 4 1 1 4 2 3 4

2019-10-04 Signe Rüsch Page 17 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Conclusion

Bloxy’s contributions: Consensus proxy for Hyperledger Fabric Encapsulation of BFT client functionality Enabling easy integration of ordering services Allowing operators to choose suitable protocols

C P P P OSN R B R B R B R B O O O O

Orderer Peer Peer Replica BLOXY Replica BLOXY Orderer Peer Peer Orderer Replica BLOXY Client Peer 4 4 1 1 4 2 3 4

2019-10-04 Signe Rüsch Page 17 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Conclusion

Bloxy’s contributions: Consensus proxy for Hyperledger Fabric Encapsulation of BFT client functionality Enabling easy integration of ordering services Allowing operators to choose suitable protocols Bloxy maintains Fabric’s modularity Good performance compared to directly integrated BFT protocols

C P P P OSN R B R B R B R B O O O O

Orderer Peer Peer Replica BLOXY Replica BLOXY Orderer Peer Peer Orderer Replica BLOXY Client Peer 4 4 1 1 4 2 3 4

2019-10-04 Signe Rüsch Page 17 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Conclusion

Bloxy’s contributions: Consensus proxy for Hyperledger Fabric Encapsulation of BFT client functionality Enabling easy integration of ordering services Allowing operators to choose suitable protocols Bloxy maintains Fabric’s modularity Good performance compared to directly integrated BFT protocols

C P P P OSN R B R B R B R B O O O O

Orderer Peer Peer Replica BLOXY Replica BLOXY Orderer Peer Peer Orderer Replica BLOXY Client Peer 4 4 1 1 4 2 3 4

Thank you for your attention! Questions? ruesch@ibr.cs.tu-bs.de

2019-10-04 Signe Rüsch Page 17 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion 2019-10-04 Signe Rüsch Page 18 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Requirements of Fabric’s Ordering Services

Agreement:

Blocks with same sequence number should be identical No support of forks

Hashchain integrity:

Blocks contain hash of predecessor Blocks form non-manipulatable hashchain

No skipping:

No peer can skip a block Has to receive all previous blocks

No creation of transactions Liveness:

Transactions submitted to the ordering service will eventually be contained in block

2019-10-04 Signe Rüsch Page 19 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Bloxy Short-Term Block Storage

C P P P OSN R B R B R B R B O O O O BS BS BS BS

P: Peers C: Client R: Replica O: Orderer B: Bloxy BS: Block Storage

2019-10-04 Signe Rüsch Page 20 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Bloxy Short-Term Block Storage: Evaluation Results

Bloxy (iii) Orderer (i) Storage Hit (ii) Storage Miss Duration 21.79 ms 7.19 ms 7.28 ms (i) Query Short-Term Block Storage (ii) Query Block Storage, query orderer from acknowledged list (iii) Randomly query orderers

2019-10-04 Signe Rüsch Page 21 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Public API of Bloxy

Connection establishment Orderer:

Submit transaction for ordering Receive voted block

Replicas:

Receive forwarded and translated transaction for ordering Submit block for majority voting

2019-10-04 Signe Rüsch Page 22 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Ordering Service

5,000 10,000 15,000 20,000 Throughput [rps] Target Solo Kafka BFT-SMaRt Bloxy+Hybster Bloxy+PBFT 1,000 5,000 10,000 15,000 20,000 100 200 Send Rate [rps] Latency [ms]

2019-10-04 Signe Rüsch Page 23 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Ordering Service

Throughput:

71 % (Hybster) and 51 % (PBFT) higher compared to BFT-SMaRt 20 % (Hybster) and 29.5 % (PBFT) lower compared to Solo

Latency:

99 % (Hybster) and 125 % (PBFT) higher than Solo 14.3 % (Hybster) lower than BFT-SMaRt

2019-10-04 Signe Rüsch Page 24 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks

Motivation Hyperledger Fabric Bloxy Evaluation Related Work Conclusion

Full Fabric Network

500 1,000 1,500 Throughput [rps] Target Solo Kafka BFT-SMaRt Bloxy+Hybster Bloxy+PBFT 100 500 1,000 1,500 1 2 3 4 5 Send Rate [rps] Latency [s]

2019-10-04 Signe Rüsch Page 25 Bloxy: Providing Transparent and Generic BFT-Based Ordering Services for Blockchains Institute of Operating Systems and Computer Networks