ISIT 2020 Topology-Aware Cooperative Data Protection in Blockchain-Based Decentralized Storage Networks Siyi Yang 1 , Ahmed Hareedy 2 , Robert Calderbank 2 , Lara Dolecek 1 1 Electrical and Computer Engineering Department, UCLA 2 Electrical and Computer Engineering Department, Duke University 06/2020
Outline w Introduction Ø Decentralized storage networks Ø Existing work w Cooperative Data Protection Ø Preliminaries Ø ECC hierarchy Ø Single-level cooperation w Multi-level cooperation Ø Cooperation graphs and compatible graphs Ø Construction over compatible graphs w Conclusion
Outline w Introduction Ø Decentralized storage networks Ø Existing work w Cooperative Data Protection Ø Preliminaries Ø ECC hierarchy Ø Single-level cooperation w Multi-level cooperation Ø Cooperation graphs and compatible graphs Ø Construction over compatible graphs w Conclusion
Decentralized Storage Networks w Invention of Blockchain technology makes the concept of “decentralization” a popular term Ø Higher privacy Ø Better scalability and flexibility w Decentralization has potential to universally revolutionize various applications Ø Blockchain-based decentralized storage networks (DSNs) Ø Masterless coded distributed computation Ø Federated learning Ø Wireless sensor networks 1
Decentralized Storage Networks w Invention of Blockchain technology makes the concept of “decentralization” a popular term Ø Higher privacy Ø Better scalability and flexibility w Decentralization has potential to universally revolutionize various applications Ø Blockchain-based decentralized storage networks (DSNs) Ø Masterless coded distributed computation Ø Federated learning Ø Wireless sensor networks 1
Decentralized Storage Networks w Typical Structures Ø Nodes form multiple clusters (clouds), each of which has a representative (master) node Ø Master nodes form a distributed network Ø Nodes in each cluster form a centralized network around the representative master node Messages are encoded and stored distributively among non-master nodes in each cloud Ce Centralized Storage De Decentralized Storage Di Distributed St Storage 2
Practical Concerns in DSNs w Hierarchical erasure correction Ø Each node provides different levels of robustness for codeword stored at it through accessing different set of nodes ● Failures in reading nodes result in erased symbols ● Extra information improves the reliability, at a cost of higher latency w Topology-awareness Ø Scheme optimized for DSNs with a specific topology can result in bad performance in DSNs with other topologies ● Different reading frequencies (hot/cold) and reliabilities among nodes ● Inter-cloud transmission is much slower than intra-cloud transmission Our goal is to construct topology-aware coding schemes that also provide hierarchical erasure correction at each node 3
Existing Literature w Distributed storage [1] Ø No explicit consideration of clustering nature of network nodes w Multi-rack storage [2-8] Ø Network topologies are typically not considered to be general Ø Capacities of the communication links are typically considered to be homogeneous [1] A. G. Dimakis et al., “ Network coding for distributed storage systems ”, IEEE Trans. Inf. Theory, vol. 56, no. 9, pp. 4539-4551, 2010 [2] Z. Kong et al., “ Decentralized coding algorithms for distributed storage in wireless sensor networks ”, IEEE JSAC, vol. 28, no. 2, pp. 261- 267, 2010 [3] M. Ye et al., “ Cooperative repair: Constructions of optimal MDS codes for all admissible parameters ”, IEEE Trans. Inf. Theory, vol. 65, no. 3, pp. 1639-1656, 2018 [4] N. Prakash et al., “ The storage versus repair-bandwidth trade-off for clustered storage systems ”, IEEE Trans. Inf. Theory, vol. 64, no. 8, pp. 5783-5805, 2018 [5] J. Li et al., “ Tree-structured data regeneration in distributed storage systems with regenerating codes ”, IEEE INFOCOM, 2010 [6] Y. Wang et al., “ Non-homogeneous two-rack model for distributed storage systems ”, IEEE INFOCOM, 2014 [7] H. Hou et al., “ Rack-aware regenerating codes for data centers ”, IEEE Trans. Inf. Theory, 2019 [8] Z. Chen et al., “ Explicit constructions of MSR codes for clustered distributed storage: the rack-aware storage model ”, [Online]. Available: https://arxiv.org/abs/1901.04419, 2019 4
Outline w Introduction Ø Decentralized storage networks Ø Existing work w Cooperative Data Protection Ø Preliminaries Ø ECC hierarchy Ø Single-level cooperation w Multi-level cooperation Ø Cooperation graphs and compatible graphs Ø Construction over compatible graphs w Conclusion
Notation w A DSN is modeled as a graph 𝐻(𝑊, 𝐹) Ø Sets of master nodes and edges between them: 𝑊 , 𝐹 The set consisting of all neighbors (master nodes) of a node 𝑤 ( is called the neighborhood of this node, and is denoted by 𝒪 ( Ø Neighborhood of 𝑤 ( : 𝒪 ( = 𝑤 + , , 𝑤 + - , 𝑤 + . Ø Codeword stored at 𝑤 ( : 𝐝 ( Ø Message stored at 𝑤 ( : 𝐧 ( Ø Number of nodes: 𝑞 Ø 𝐧 = 𝐧 2 ,𝐧 3 , ⋯ , 𝐧 5 ( m j 2 , c j 2 ) v j 3 ( m j 3 , c j 3 ) e i,j 3 e i,j 2 v j 2 Ø 𝐝 = 𝐝 2 , 𝐝 3 , ⋯, 𝐝 5 ( m i , c i ) Ø Length of 𝐧 ( , 𝐝 ( : 𝑙 ( , 𝑜 ( e i,j 1 Ø Redundancy of 𝐝 ( : 𝑠 ( = 𝑜 ( − 𝑙 ( v j 1 ( m j 1 , c j 1 ) 5
ECC Hierarchy ECC hierarchy describes the erasure correction (EC) capabilities of nodes while cooperating with different sets of other nodes λ i, 3; B 3 d i, 3 i λ i, 2; B 2 d i, 2 i λ i, 1; B 1 d i, 1 λ i, 3; ∅ i d i, 0 λ i, 2; ∅ λ i, 1; ∅ 6
ECC Hierarchy ECC hierarchy describes the erasure correction (EC) capabilities of nodes while cooperating with different sets of other nodes w Cooperation at each node Ø Sets associated with nodes utilizing 𝑚 -th level cooperation 2 ⊂ ( B = ∅ > ? ⊆ 𝑊, ℬ ( 3 ⊂ ⋯ ⊂ ( B 2 ∩ ℬ ( ● ∅ ⊂ ( 2CBC> ? , ( λ i, 3; B 3 d i, 3 i λ i, 2; B 2 d i, 2 i λ i, 1; B 1 d i, 1 λ i, 3; ∅ i d i, 0 λ i, 2; ∅ λ i, 1; ∅ 6
ECC Hierarchy ECC hierarchy describes the erasure correction (EC) capabilities of nodes while cooperating with different sets of other nodes w Cooperation at each node Ø Sets associated with nodes utilizing 𝑚 -th level cooperation 2 ⊂ ( B = ∅ > ? ⊆ 𝑊, ℬ ( 3 ⊂ ⋯ ⊂ ( B 2 ∩ ℬ ( ● ∅ ⊂ ( 2CBC> ? , ( Ø ECC hierarchy at 𝑤 ( : 𝐞 ( = 𝑒 (,G ,𝑒 (,2 ,⋯ , 𝑒 (,> ? ; depth of 𝐞 ( : 𝑀 ( B are recovered and involved in the decoding process ● Nodes in ( ● 𝑒 (,B : EC capability with the 𝑚 -th level cooperation at 𝑤 ( λ i, 3; B 3 d i, 3 i λ i, 2; B 2 d i, 2 i λ i, 1; B 1 d i, 1 λ i, 3; ∅ i d i, 0 λ i, 2; ∅ λ i, 1; ∅ 6
ECC Hierarchy ECC hierarchy describes the erasure correction (EC) capabilities of nodes while cooperating with different sets of other nodes w Cooperation at each node Ø Sets associated with nodes utilizing 𝑚 -th level cooperation 2 ⊂ ( B = ∅ > ? ⊆ 𝑊, ℬ ( 3 ⊂ ⋯ ⊂ ( B 2 ∩ ℬ ( ● ∅ ⊂ ( 2CBC> ? , ( Ø ECC hierarchy at 𝑤 ( : 𝐞 ( = 𝑒 (,G ,𝑒 (,2 ,⋯ , 𝑒 (,> ? ; depth of 𝐞 ( : 𝑀 ( B are recovered and involved in the decoding process ● Nodes in ( ● 𝑒 (,B : EC capability with the 𝑚 -th level cooperation at 𝑤 ( λ i, 3; B 3 d i, 3 Ø Elaborated EC capability: 𝑒 (,B = 𝜇 (,B;𝒳 ∅⊆𝒳⊆ℬ ? i M ● 𝜇 (,B;𝒳 : EC capability while nodes in 𝒳 are λ i, 2; B 2 d i, 2 also recovered and involved in the i 𝑚 -th level cooperation at 𝑤 ( λ i, 1; B 1 d i, 1 λ i, 3; ∅ i d i, 0 λ i, 2; ∅ λ i, 1; ∅ 6
ECC Hierarchy w Example 2 are neighbors of 𝑤 3 and are required to be locally- Ø Nodes in 3 recoverable to remove cross parities from the parity part of 𝐝 3 ● 𝑤 3 tolerates 𝑠 3 erasures in this scenario v 1 v 5 v 9 v 2 v 8 A 1 2 v 3 v 10 B 1 v 12 2 v 4 v 7 v 6 v 11 7
ECC Hierarchy w Example 2 are neighbors of 𝑤 3 and are required to be locally- Ø Nodes in 3 recoverable to remove cross parities from the parity part of 𝐝 3 ● 𝑤 3 tolerates 𝑠 3 erasures in this scenario 2 are neighbors of nodes in 3 2 except for 𝑤 3 and Ø Nodes in ℬ 3 2 themselves nodes in 3 ● 𝑤 3 obtains extra parities from 𝑤 N if 𝑤 O is also recovered, i.e., 𝑤 O ⊆ 𝒳 ● 𝑤 3 obtains extra parities from 𝑤 P if 𝑤 O ,𝑤 Q ,𝑤 R ⊆ 𝒳 ● 𝑤 3 obtains extra parities from 𝑤 2 v 1 v 5 v 9 v 2 v 8 A 1 2 v 3 v 10 B 1 v 12 2 v 4 v 7 v 6 v 11 7
ECC Hierarchy w Example 2 are neighbors of 𝑤 3 and are required to be locally- Ø Nodes in 3 recoverable to remove cross parities from the parity part of 𝐝 3 ● 𝑤 3 tolerates 𝑠 3 erasures in this scenario 2 are neighbors of nodes in 3 2 except for 𝑤 3 and Ø Nodes in ℬ 3 2 themselves nodes in 3 ● 𝑤 3 obtains extra parities from 𝑤 N if 𝑤 O is also recovered, i.e., 𝑤 O ⊆ 𝒳 ● 𝑤 3 obtains extra parities from 𝑤 P if 𝑤 O ,𝑤 Q ,𝑤 R ⊆ 𝒳 ● 𝑤 3 obtains extra parities from 𝑤 2 v 1 Ø All possible 𝜇 3,2;𝒳 v 5 v 9 v 2 ● 𝜇 3,2;𝒳 = 𝜇 3,2;∅ ,𝒳 ⊆ 𝑤 Q ,𝑤 R v 8 A 1 ● 𝜇 3,2;𝒳 = 𝜇 3,2; S T , 2 v 3 v 10 𝑤 O ⊆ 𝒳 ⊂ 𝑤 O ,𝑤 Q ,𝑤 R B 1 l v 12 2 ● 𝜇 3,2;𝒳 = 𝜇 3,2; S T ,S U ,S V = 𝑒 3,2 , v 4 v 7 v 6 l 𝒳 = 𝑤 O ,𝑤 Q ,𝑤 R v 11 7
Example: Single-Level Cooperation w Parity part of the generator matrix of a single-level accessible code based on CRS codes v 1 v 5 v 9 v 2 v 8 v 3 v 10 v 12 v 4 v 7 v 6 v 11 8
Recommend
More recommend
