Problems of Network Coding in P2P - and how to overcome it - PowerPoint PPT Presentation

Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with Christian Ortolf & Arne Vater presented in SPAA 09 & 10 Albert-Ludwig University Freiburg Department of Computer Science Computer Networks and Telematics

Global Internet Traffic Shares 1993-2004 CacheLogic Research Trends of Internet Protocols 1993-2004 70 60 50 Share of Internet traffic 40 30 E-Mail 20 FTP Peer-to-Peer 10 Web 0 Source: CacheLogic 2005 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2

P2P Share Germany 2007 Quelle: Ipoque 2007 3

What Germans Download 2007 by Volume Quelle: Ipoque 2007 4

Internet Traffic of a Geman ISP August 2009 Download HTTP 44.4 % Skype 0.8 % RTSP 1.2 % eDonkey 4 % BitTorrent 24.1 % RTMP 5 % Upload SHOUTcast 6.4 % NNTP 14.2 % HTTP 14.6 % BitTorrent 64.3 % Skype 3 % RTSP 0.1 % eDonkey 16.3 % RTMP 0.4 % SHOUTcast 0.7 % NNTP 0.7 % 5

P2P Systems Germany 2007 by Volume Quelle: Ipoque 2007 6

Motivation  Peer-to-peer networks - distributed system - equal participants (peers) - no client/server structure - used for • communication (i.e. Skype) • data storage (i.e. OceanStore) • file sharing 7

Block-Based  Bram Cohen: BitTorrent file X  Block-based file sharing system x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 - divide file into blocks - efficient download - efficient usage of upload - usage of several sources - fairness amongst peers x 2 - uses implicit multicast trees x 3 for the distribution of blocks x 1 x 4 x 8 x 5 x 7 x 6 8

BitTorrent Block Selection  Coupon-Collector-Problem 0 x 1 - reason for unevenly distributed blocks 1 x 2 - if blocks chosen randomly 5 x 3  Measures: Policies 4 x 4 - heuristics to select blocks for distribution 1 x 5 - different policies depending on progress 3 x 6 • random first 0 x 7 • rarest first 2 x 8 • endgame - very successful for popular files 9

Solution: Network Coding  Optimal solution for Coupon-Collector-Problem / Policy - optimal network flow • Ahlswede, Cai, Li, and Yeung, "Network Information Flow" - practical network coding • Gkantsidis, and Rodriguez, "Network coding for large scale content distribution“  Method - sender transmits code blocks as linear combinations of the file‘s blocks - receiver collects code blocks and reconstructs the original file 10

Coding and Decoding (I) x 1 x 2 x 3 encoding x 4 b i b 2 b 3 b 6 b 9 b 12 b 17 b 18 b 20 x 5 x 6 decoding x 7 x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 8 11

Coding and Decoding (II)  file X = (x 1 , x 2 , ..., x n )  linear coefficients c ij  code blocks b 1 , b 2 , ..., b n  if the matrix is invertable then 12

Problems of Network Coding  Overhead of storing linear coefficients - one per block - e.g. 4 GB file with 100 KB blocks • 4 GB / 100 KB = 40 KB per block • overhead 40% - better: 4 GB file and 1 MB blocks • 4 KB overhead = 0.4% Overhead of decoding  - Inversion of an (n × n)-matrix needs time O(n 3 )  Read/write accesses - writing n blocks requires reading each part n times: O(n 2 ) - disk access is much slower than memory access 13

Scenario  BitTorrent is optimal regarding disk access and computation overhead, - but it suffers from the coupon collector problem (availability). 14

Scenario  Network Coding is optimal regarding availability - but it has a high computational overhead as well as high disk access overhead 15

Our Goal  We want to find a coding scheme that - performs better than BitTorrent regarding availability, and - requires less read/write accesses than Network Coding. 16

Paircoding  Paircoding - is a reduced form of Network Coding - combines only two original blocks into one code block • p i,j = c i x i + c j y j x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 . . . p 2,3 p 4,7 p 1,2 p 4,5 p 5,7 p 3,6 p 4,6 p 1,8 p i,j 17

Advantages  Easy code block creation - only two original blocks must be read  Recoding - new code blocks can be created from different code blocks - no prior decoding necessary  Decoding - requires little computation (sparse matrix inversion) - can be done lazy  Coding alleviates the coupon collector problem 18

Scenario Paircoding outperforms  BitTorrent - with less overhead than Network Coding 19

Simulation (I)  one seed  one downloading peer  seeder sends one random block in each round 20

Policy  Policy - algorithmic choice of creating and uploading a code block depending on • receiving peer • current network configuration • availability • progress  Policy of BitTorrent - optimize throughput and fairness Policy of Practical Network Coding  - always send linearly independent code blocks - optimal 21

Outperforming A file sharing system A is at least as good as B, A ≥ B if for every scenario and every policy of B there is a policy in A such that A performs at least as well as B. If A ≥ B and there exists a scenario in which A has larger progress than B, A outperforms B. A > B 22

Paircoding - Results  Theorems - Paircoding outperforms BitTorrent • For all scenarios and any BitTorrent policy, Paircoing is at least as efficient as BitTorrent. • For some scenarios Paircoding is more efficient than Bittorrent, i.e. Paircoding outperforms BitTorrent. - Encoding and decoding can be performed with an almost linear number of read/write operations: O(n • α (n)). • α (n) is the inverse Ackerman function 23

Treecoding( κ )  tree structure - fixed linear coefficients for all blocks x i - Xor of two nodes creates parent node  κ different trees - with linearly independent linear coefficients  root node is equivalent to a network coding block  leaves are equivalent to uncoded blocks 24

Treecoding - Results  Performance hierarchy - Treecoding( κ + 1 ) > Treecoding( κ )  Treecoding performs as well as forward error correction Treecoding( κ ) ≥ FEC( κ ) -  Treecoding outperforms Fixed Paircoding ∪ κ Treecoding( κ ) > FixedPaircoding - - if the number of trees is arbitrary Treecoding and Paircoding are incomparable  Treecoding has read/write cost of  O(n), if κ = 1 - O( κ n log 2 n), for any κ - 25

Class Hierarchy 26

Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with Christian Ortolf & Arne Vater presented in SPAA 09 & 10 Albert-Ludwig University Freiburg Department of Computer Science Computer Networks and Telematics