Automatic Deployment of Peer-to-Peer Network Infrastructures PhD - - PowerPoint PPT Presentation

▶

Jul 13, 2023 408 likes •769 views

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Automatic Deployment of Peer-to-Peer Network Infrastructures PhD Scientific Report R azvan Deaconescu Supervisor: prof. dr.

SLIDE 1

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Automatic Deployment of Peer-to-Peer Network Infrastructures

PhD Scientific Report R˘ azvan Deaconescu Supervisor: prof. dr. ing. Nicolae T

, ˘

apus

Computer Science and Engineering Department

January 31, 2011, Bucharest

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 2

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Outline

1 Context 2 Virtualized Infrastructure 3 Automated Framework 4 Data Acquisition 5 Setup and Results 6 Conclusion 7 Questions

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 3

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Peer-to-Peer Systems

Peer-to-Peer vs. client-server continuous evolution since 2000 (Napster) definition

Does it treat variable connectivity and temporary addresses as a natural occurrence? Does it provide autonomy to all nodes, even those on the edge

f the network?

streaming technology making its way

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 4

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

BitTorrent

flagship of BitTorrent

most core traffic is still BiTorrent HTTP rapidly catching up (video streaming)

multiple implementations rarest-piece-first, tit-for-tat, optimistic-unchoke

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 5

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Measurements in Peer-to-Peer Systems

testbeds (PlanetLab) simulations real world scenarios

data collection and storage trackers

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 6

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Virtualization

hardware/server virtualization multiple instances of separate OSs running on same hardware hypervisor (VMM) consolidation, reliability, security full virtualization, paravirtualization, OS-level virtualization

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 7

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Goals

automated infrastructure for BitTorrent clients scalable and flexible (virtualization) complete control over swarm environment (private infrastructure) intensive client information (client logging) performance evaluation and comparison internal protocol analysis

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 8

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Outline

1 Context 2 Virtualized Infrastructure 3 Automated Framework 4 Data Acquisition 5 Setup and Results 6 Conclusion 7 Questions

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 9

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Virtualization Solutions

constraints: Linux environment, automation, low overhead KVM: native virtualization, not easily automated built, some

verhead

Xen: paravirtualization (good performance), separation,

verhead

OS-level virtualization

LXC: mainline kernel support, lack of documentation OpenVZ: good spread, good documentation

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 10

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

OpenVZ

OS-level virtualization low overhead (extended chroot) – runs directly on top of

perating system

templates CLI-interaction (vzctl) – automation resource limitation (user beancounters)

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 11

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Overall View

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 12

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Tools Employed

vzctl + templates shell scripting SSH tc (traffic control) iptables brctl sysstat

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 13

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Virtualization Evaluation

efficiency (scalability) – how many virtual machines/containers may be deployed on a virtual host and allow proper simulation of an environment; isolation – how well are virtual machines’ resources separated; reliability – how many software crashes happen for a given solution; this may be due to implementation or to resourse

veruse/abuse.

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 14

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Virtualization Evaluation (2)

Eff = f (HW , SW , VS) (1) Eff = f (RAM, HDD, CPU, NET, OS, PS, BT, VS, NVM) (2) Eff = VMB HNB (3) Iso(normalprocesses) < Iso(chroot) < Iso(OpenVZ, LXC) < Iso(Xen, KVM) (4) Rel = f (HW , SW , VS) (5) Rel = f (RAM, HDD, CPU, OS(filesystem), PS, BT, VS, NVM) (6)

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 15

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Open Issues

formal evaluation of virtualization solutions and BitTorrent processes LXC vs. OpenVZ

OS-level virtualization efficiency/scalaibility

traffic control in OpenVZ scalability evaluation

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 16

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Outline

1 Context 2 Virtualized Infrastructure 3 Automated Framework 4 Data Acquisition 5 Setup and Results 6 Conclusion 7 Questions

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 17

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Architecture

Commander Station OpenVZ Container Tribler Hrktorrent Transmission Server bootstrap Server (SSH) command a c k / e r r

XML Configuration Files R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 18

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Goals

automation complete control (internal swarms vs. external swarms) full client information

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 19

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Client Instrumentation

pen-source clients

update for status logging instrument for verbose logging wrappers for starting/stopping/collecting results

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 20

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Client Management

Commander station service running on each container (start, stop, status)

specialized protocol

XML-based node-description and swarm-description

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 21

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Outline

1 Context 2 Virtualized Infrastructure 3 Automated Framework 4 Data Acquisition 5 Setup and Results 6 Conclusion 7 Questions

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 22

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Client Logging

status messages

periodic (usually each second) client state upload speed, download speed, eta, download percentage lightweight may be monitored

verbose log messages

per-protocol/client behavior internal events (chokes, unchokes, have, piece transmission) heavyweight cannot be monitored to be collected, parsed and disseminated

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 23

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Log Processing Engine

SQLite Database Verbose Log Status Log BitTorrent Swarm Verbose Parser Status Parser Database Access Module Rendering Engine

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 24

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Evaluating Performance

Eval(hw, sys, impl, swarm, net) = (protomsg, speed, conn, ruse) (7) FS =

DSt (8) PDSt =      ds1,1 = 0 ds1,2 · · · ds1,NP ds2,1 ds2,2 = 0 · · · ds2,NP . . . . . . ... . . . dsNP,1 dsNP,2 · · · dsNP,NP = 0      (9)

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 25

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Outline

1 Context 2 Virtualized Infrastructure 3 Automated Framework 4 Data Acquisition 5 Setup and Results 6 Conclusion 7 Questions

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 26

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Experimental Setup

10 commodity hardware systems

dual core CPU, 3GHz 2GB RAM 300GB HDD

1Gbit connections 10–20 OpenVZ containers per system “hidden network”

all containers are part of the 10.0.0.0/8 network uses Linux bridging on host system

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 27

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Monitoring

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 28

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Monitoring (2)

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 29

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Performance Evaluation

Client Test1 Test2 Test3 Test4 file size 908MB 4.1GB 1.09GB 1.09GB seeders 2900 761 521 496 leechers 2700 117 49 51 aria2c 1h17m 53m53s 8m 10m23s azureus 32m41s 38m33s N/A 7m bittorrent 4h53m 60m39s 26m 14m libtorrent 9m41s 15m13s 2m30s 2m14s transmission 40m46s 53m 7m 5m tribler 34m 21m N/A N/A

Tabela: Test Swarms Results

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 30

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Rendering Engine

: : 5 : : 1 : : 1 5 : : 2

time (s)

100 100 200 300 400 500

KB/s

ds Client 3 acc_ds Client 3

0.0 0.2 0.4 0.6 0.8 1.0 CHOKE UNCHOKE INTERESTED NOT_INTERESTED HAVE BITFIELD REQUEST PIECE CANCEL DHT_PORT 2

1 39 1 638 721

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 31

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Outline

1 Context 2 Virtualized Infrastructure 3 Automated Framework 4 Data Acquisition 5 Setup and Results 6 Conclusion 7 Questions

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 32

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Conclusion

virtualized infrastructure allows scalability and flexibility framework allows automation verbose logging for extensive information result processing and analysis performance evaluation

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 33

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Planning

deploy multiple swarm scenarios

swarm diversity peer characteristics P2P network setup BitTorrent implementation

automate P2P video streaming implementations classical distribution vs. video streaming formalize performance measurements provide feedback/advice to implementation

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 34

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Keywords

Peer-to-Peer BitTorrent virtualization automation infrastructure scenarios instrumentation OpenVZ status log messages verbose log messages performance evaluation rendering engine SSH, iptables, tc, brctl

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

SLIDE 35

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions

Outline

1 Context 2 Virtualized Infrastructure 3 Automated Framework 4 Data Acquisition 5 Setup and Results 6 Conclusion 7 Questions

R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures