Antfarm: Efficient Content Distribution with Managed Swarms Ryan S. - - PowerPoint PPT Presentation

▶
antfarm efficient content distribution with managed swarms
SMART_READER_LITE
LIVE PREVIEW

Antfarm: Efficient Content Distribution with Managed Swarms Ryan S. - - PowerPoint PPT Presentation

Nov 25, 2022 129 likes •519 views

Antfarm: Efficient Content Distribution with Managed Swarms Ryan S. Peterson and Emin Gn Sirer Department of Computer Science, Cornell University United Networks, LLC April 22, 2009 Problem Domain What is the most efficient way to

slide-1
SLIDE 1

Antfarm: Efficient Content Distribution with Managed Swarms

Ryan S. Peterson and Emin Gün Sirer Department of Computer Science, Cornell University United Networks, LLC April 22, 2009

slide-2
SLIDE 2

Problem Domain

What is the most efficient way to disseminate a large set of files to a large set of clients?

slide-3
SLIDE 3

Client-Server

server clients

slide-4
SLIDE 4

Client-Server

clients

Inefficient High cost of ownership

slide-5
SLIDE 5

Peer-to-Peer

peer block transfer

slide-6
SLIDE 6

Peer-to-Peer

Limited information No control or performance guarantees

slide-7
SLIDE 7

Peer-to-Peer

swarm

slide-8
SLIDE 8

Antfarm Goals

  • High performance
  • Low cost of deployment
  • Performance guarantees
  • Administrator control over swarm performance
  • Accounting
  • Enables different resource contribution policies
slide-9
SLIDE 9

Antfarm Approach

  • Key insight: view content distribution as

an optimization problem

  • Hybrid architecture
  • P2P swarming with a logically centralized

coordinator

  • Clean slate protocol
slide-10
SLIDE 10

Antfarm System Model

coordinator seeder altruist

slide-11
SLIDE 11

Antfarm System Model

coordinator seeder

Coordinator optimally allocates total seeder bandwidth B

slide-12
SLIDE 12

Antfarm

Overview The System Evaluation

slide-13
SLIDE 13

Antfarm

Overview The System Evaluation

slide-14
SLIDE 14

Strawman Coordinator

  • One could schedule every data transfer

in the system

  • All packets for all time
  • Unscalable, impractical!
  • Antfarm coordinator makes critical

decisions based on observed dynamics

slide-15
SLIDE 15

Antfarm Coordinator

  • Models swarm dynamics
  • Measures and extracts key parameters
  • Formulates optimization problem
  • Calculates optimal bandwidth allocation
  • Enacts allocation decisions
  • Maximizes aggregate bandwidth
  • Minimizes average download time
slide-16
SLIDE 16

Antfarm Formalization

Maximize system-wide aggregate bandwidth subject to a bandwidth constraint

slide-17
SLIDE 17

Swarm aggregate bandwidth Seeder bandwidth

Response Curves

s l

  • p

e = 1 slope = 0

slide-18
SLIDE 18

Response Curves

Swarm aggregate bandwidth (KB/s)

1500 1000 500

Seeder bandwidth (KB/s)

25 75 100 50

slide-19
SLIDE 19

Swarm Dynamics

Swarms exhibit different dynamics based on size, peer resources, network conditions. . .

slide-20
SLIDE 20

Swarm Dynamics

slide-21
SLIDE 21

Antfarm Optimization

Swarm aggregate bandwidth Seeder bandwidth

A B C

slide-22
SLIDE 22

Antfarm Optimization

Swarm aggregate bandwidth Seeder bandwidth

A C B A + B + C = B A B C

slide-23
SLIDE 23

Performance Control

  • Can provide swarm performance

guarantees

  • Guarantee minimum level of service
  • Prioritize swarms
slide-24
SLIDE 24

Swarm aggregate bandwidth Seeder bandwidth

Antfarm Allocation

A C B A+ B+ C= B A B C

slide-25
SLIDE 25

Adapting to Change

  • Swarm dynamics change
  • Churn
  • Network conditions
  • Antfarm updates response curves
  • Coordinator explores around point of
  • peration
slide-26
SLIDE 26

peer A

purse ledger

Wire Protocol

  • Coordinator mints small, unforgeable tokens
  • Peers trade each other tokens for blocks
  • Peers return spent tokens to the coordinator

as proof of contribution

coordinator

peer B

purse ledger

slide-27
SLIDE 27

Antfarm

Overview The System Evaluation

slide-28
SLIDE 28

Antfarm

Overview The System Evaluation

slide-29
SLIDE 29

Antfarm Performance

Zipf, 60 KB/s seeder Zipf, 200 KB/s seeder 1000 2000 3000 4000 Aggregate bandwidth (KB/s)

Client-server BitTorrent Antfarm

slide-30
SLIDE 30

Swarm Starvation

10 20 30 BitTorrent Antfarm Avg bandwidth per peer (KB/s)

self-sufficient swarm singleton swarm

BitTorrent starves the singleton swarm

slide-31
SLIDE 31

BitTorrent: Starves New Swarm

total seeder bandwidth avg bandwidth per peer

5 10 15 20 25 Bandwidth (KB/s) Swarms, ordered largest to smallest

new self-sufficient singleton

slide-32
SLIDE 32

total seeder bandwidth avg bandwidth per peer

5 10 15 20 25 Bandwidth (KB/s)

Antfarm: Seeds New Swarm

Swarms, ordered largest to smallest

new self-sufficient singleton

slide-33
SLIDE 33

Scalability

Number of peers

20K 40K 60K 80K

1-machine coordinator 4-machine coordinator 2-machine coordinator 8-machine coordinator

5 GB/s 1 GB/s 2 GB/s 3 GB/s 4 GB/s Aggregate bandwidth

slide-34
SLIDE 34

Scalability

Number of peers

20K 40K 60K 80K

1-machine coordinator 4-machine coordinator 2-machine coordinator 8-machine coordinator

5 GB/s 1 GB/s 2 GB/s 3 GB/s 4 GB/s Aggregate bandwidth

Single PC can compute allocations for 10,000 swarms with 1,000,000 peers in 6 seconds

slide-35
SLIDE 35

Antfarm Implications

  • No fine-tuning
  • Subsumes hacks devised for BitTorrent
  • Share ratio
  • Manual pruning
slide-36
SLIDE 36

Related Work

  • Content Distribution Networks
  • Akamai, CoBlitz, CoDeeN, ECHOS, Coral, Slurpie,

YouTube, Hulu, GridCast, Tribler, Joost, Huang et al. 2008, ...

  • P2P Swarming
  • BitTorrent, BitTyrant, PropShare, BitTornado, BASS,

Annapureddy et al. 2007, Guo et al. 2005, ...

  • Incentives and microcurrencies
  • Dandelion, BAR Gossip, Samsara, Karma, SHARP, PPay,

Kash et al. 2007, ...

slide-37
SLIDE 37

Conclusions

  • Model swarm dynamics and allocate

bandwidth optimally

  • Novel hybrid architecture
  • PlanetLab deployment shows that Antfarm
  • utperforms client-server and P2P
slide-38
SLIDE 38

Questions?