Mathematical Analysis of Scheduling Policies in Peer-to-Peer Video - - PowerPoint PPT Presentation

▶
mathematical analysis of scheduling policies in peer to
SMART_READER_LITE
LIVE PREVIEW

Mathematical Analysis of Scheduling Policies in Peer-to-Peer Video - - PowerPoint PPT Presentation

May 23, 2023 139 likes •653 views

Motivation Contributions Video on-demand Live Streaming Conclusions Mathematical Analysis of Scheduling Policies in Peer-to-Peer Video Streaming Networks Pablo Romero Facultad de Ingeniera, Universidad de la Repblica. PEDECIBA

slide-1
SLIDE 1

Motivation Contributions Video on-demand Live Streaming Conclusions

Mathematical Analysis of Scheduling Policies in Peer-to-Peer Video Streaming Networks

Pablo Romero

Facultad de Ingeniería, Universidad de la República. PEDECIBA Informática, Montevideo, Uruguay.

Advisors: Dr. Ing. Franco Robledo (Universidad de la República)

  • Dr. Ing. Pablo Rodríguez-Bocca (Universidad de la República)

Ph.D. Thesis Defense, November 19th, 2012

1 / 47

slide-2
SLIDE 2

Motivation Contributions Video on-demand Live Streaming Conclusions

Outline

1

Motivation

2

Contributions

3

Video on-demand

4

Live Streaming

5

Conclusions

2 / 47

slide-3
SLIDE 3

Motivation Contributions Video on-demand Live Streaming Conclusions

Outline

1

Motivation

2

Contributions

3

Video on-demand

4

Live Streaming

5

Conclusions

3 / 47

slide-4
SLIDE 4

Motivation Contributions Video on-demand Live Streaming Conclusions Services

Video Distribution in Internet

Video streaming modes File sharing: full download is mandatory before playback. Video on-demand: progressive and asymmetric playback. Live Streaming: simultaneous generation, distribution and synchronized playback. Hint In this thesis we focus on the most challenging streaming modes: Video on-demand (VoD) and Live streaming (Live).

4 / 47

slide-5
SLIDE 5

Motivation Contributions Video on-demand Live Streaming Conclusions Services

Video Distribution in Internet

Video streaming modes File sharing: full download is mandatory before playback. Video on-demand: progressive and asymmetric playback. Live Streaming: simultaneous generation, distribution and synchronized playback. Hint In this thesis we focus on the most challenging streaming modes: Video on-demand (VoD) and Live streaming (Live).

4 / 47

slide-6
SLIDE 6

Motivation Contributions Video on-demand Live Streaming Conclusions Video On-Demand

Video on-demand

Context A 10% of Internet Traffic is due to YouTube videos. Google pays more than 1 million dollars per day for bandwidth access. YouTube still does not exploit idle resources from end-users. Hint A mathematical analysis of user-assistance in VoD services is attractive.

5 / 47

slide-7
SLIDE 7

Motivation Contributions Video on-demand Live Streaming Conclusions Video On-Demand

Video on-demand

Context A 10% of Internet Traffic is due to YouTube videos. Google pays more than 1 million dollars per day for bandwidth access. YouTube still does not exploit idle resources from end-users. Hint A mathematical analysis of user-assistance in VoD services is attractive.

5 / 47

slide-8
SLIDE 8

Motivation Contributions Video on-demand Live Streaming Conclusions Live Streaming

Live Streaming

Context BitTorrent is suitable for offline, but live services... The most successful P2P systems are BitTorrent-based. Pull-mesh systems represent the most promising distribution engine. Hint A mathematical analysis of pull-mesh cooperative services is attractive.

6 / 47

slide-9
SLIDE 9

Motivation Contributions Video on-demand Live Streaming Conclusions Live Streaming

Live Streaming

Context BitTorrent is suitable for offline, but live services... The most successful P2P systems are BitTorrent-based. Pull-mesh systems represent the most promising distribution engine. Hint A mathematical analysis of pull-mesh cooperative services is attractive.

6 / 47

slide-10
SLIDE 10

Motivation Contributions Video on-demand Live Streaming Conclusions

Outline

1

Motivation

2

Contributions

3

Video on-demand

4

Live Streaming

5

Conclusions

7 / 47

slide-11
SLIDE 11

Motivation Contributions Video on-demand Live Streaming Conclusions Main Contributions

Video On-Demand

Video On-Demand

1

Mathematical modeling of user-assisted VoD systems.

2

Analysis of expected evolution.

3

Analysis of global stability (sequential VoD systems).

4

Cooperative systems always outperform raw CDN technology.

5

Combinatorial specification of a Caching Problem.

6

Resolution and application in YouTube (real-world simulation).

8 / 47

slide-12
SLIDE 12

Motivation Contributions Video on-demand Live Streaming Conclusions Main Contributions

Live Streaming

Live Streaming

1

Mathematical analysis of chunk scheduling policies (pull-mesh cooperative model).

2

Design of the best policies so far.

3

Introduction of feasible policies in GoalBit.

4

Design of a Multi-Class model, regarding free-riding and heterogeneity.

9 / 47

slide-13
SLIDE 13

Motivation Contributions Video on-demand Live Streaming Conclusions

Outline

1

Motivation

2

Contributions

3

Video on-demand

4

Live Streaming

5

Conclusions

10 / 47

slide-14
SLIDE 14

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Objective

The Goal Consider a set of video items, super-peers (resourceful stable peers) with repositories and joining users requesting videos on demand. We want to find the best video assignment into the repositories, in order to offer a minimal download time to end-users. Note A special treatment to popular videos and large files is required.

11 / 47

slide-15
SLIDE 15

Motivation Contributions Video on-demand Live Streaming Conclusions Model

VoD components (1/2)

Content: video to be distributed on demand. Peer: end-user that wants to watch one or several video items (downloader or seeder) Super-peers: resourceful stable peers managed by the op- erator Repository: limited storage space where to save VoD con- tents to be distributed by Super-Peers. Tracker: server entity that knows all Peers and Super- Peers that are sharing a content (seeding or downloading).

12 / 47

slide-16
SLIDE 16

Motivation Contributions Video on-demand Live Streaming Conclusions Model

VoD components (2/2)

13 / 47

slide-17
SLIDE 17

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Definitions

As proposed in related literature, we model the system as a Markov chain with following details: K video items with sizes s1, ..., sK (measured in Kbits). Each peer can download multiple streams at time t. Peers are grouped into classes: {C1, C2, . . . , CK} (peers in class Ci are downloading i videos simultaneously). Peers set’s cardinalities:

xi

j (t): downloaders in class Ci downloading video j at time t

yi

j (t): seeders in class Ci seeding video j at time t

zi

j (t): super-peers in class Ci seeding video j at time t

Markov chain hypothesis:

Peers join the network following a poissonian process, and abort the system with an exponential law. Seeders depart the system exponentially.

14 / 47

slide-18
SLIDE 18

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Other model parameters

λi

j

arrival rate for downloaders in class Ci requesting video j θi

j

departure rate of downloaders in class Ci downloading video j γi

j

departure rate of seeders in class Ci requesting video j ci

j

download bandwidth for each peer in the class Ci requesting video j (in kbps) µi

j

upload bandwidth for each peer in the class Ci seeding video j (in kbps) ρi

j

upload bandwidth for each super-peers in the class Ci seeding video j (in kbps) η video sharing effectiveness between peers (η ∈ [0, 1])

15 / 47

slide-19
SLIDE 19

Motivation Contributions Video on-demand Live Streaming Conclusions Model

General Fluid Model (GFM)

Modeling the expected peers’ behavior as a deterministic fluid model, we get the General Fluid Model (peers evolution: xi

j (t) and yi j (t)):

GFM        ˙ xi

j = λi j − θi jxi j − min{ci j xi j , ηµi jxi j +

  • k

(µk

j yk j + ρk j zk j )},

˙ yi

j = min{ci j xi j , ηµi jxi j +

  • k

(µk

j yk j + ρk j zk j )} − γi j yi j .

where i, j ∈ {1, . . . , K}. The complexity and number of variables involved makes the GFM hard to treat analytically.

16 / 47

slide-20
SLIDE 20

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Concurrent Fluid Model (P2P-CFM)

BitTorrent-based Assumptions

1

“Fair transmission”: the resources are equally distributed in the different concurrent videos:

2

“Tit-for-tat”: proportional downloading according to the level of altruism.

3

“Fair Seeders”: seeders send a rate proportional to download bandwidth and population.

4

“Fair Super-peers”: super-peers send a rate proportional to download bandwidth and population.

5

“Peers Departures”: peers and seeders depart following the Zipf law.

17 / 47

slide-21
SLIDE 21

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Video on-demand

Sub-Models and Relations

GFM P2P-CFM P2P-SFM CDN-CFM CDN-SFM

Fairness No Cooperation Single item No Cooperation Single item

18 / 47

slide-22
SLIDE 22

Motivation Contributions Video on-demand Live Streaming Conclusions Sequential fluid model

P2P-SFM - Rest Point

We find the only rest point solving the system

dxi

j

dt = dyi

j

dt = 0:

Rest Point for the P2P-SFM xj

P2P SFM = max

  • λjsj

θsj + c , λj(γsj − µ) − γρzj θ(γsj − µ) + ηγµ

  • yj

P2P SFM = λj − θxj P2P SFM

γj . The rest point for the CDN-SFM is just xj

CDN SFM = xj P2P SFM|µ=0.

Global Stability Theorem The P2P-SFM is Globally Stable (whenever γ > 0).

19 / 47

slide-23
SLIDE 23

Motivation Contributions Video on-demand Live Streaming Conclusions Sequential fluid model

Expected waiting time

The expected waiting time for any downloader under regime can be computed applying Little’s law: E{Tj} = xj

λj .

Then, the expected waiting time of a user T SFM

P2P and T SFM CDN are:

Expected waiting time T P2P

SFM = 1

λ

  • j

xj

P2P SFM,

T CDN

SFM = 1

λ

  • j

xj

CDN SFM,

where λ =

  • j

λj. Domination Theorem T P2P

SFM ≤ T CDN SFM .

20 / 47

slide-24
SLIDE 24

Motivation Contributions Video on-demand Live Streaming Conclusions Sequential fluid model

Multiple Video Cache

Caching Problem min

E

T P2P

SFM(γ = ∞)

s.t.          E × s ≤ S Et × uP = z z ≥ 2uK E(p, j) ∈ {0, 1} ∀p ∈ [P], j ∈ [K]. Complexity Theorem The Caching Problem is NP-Complete.

21 / 47

slide-25
SLIDE 25

Motivation Contributions Video on-demand Live Streaming Conclusions Sequential fluid model

Empirical Results (1/2)

We captured YouTube traces with a passive crawler, and stressed P2P and raw CDN systems to analyze their scalability. P2P vs CDN

1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 1 2 3 4 5 6 7 8 9 10

Estimated download time Estimated download time Popularity factor (10^i) Popularity factor (10^i)

T_P2P T_P2P T_CDN T_CDN 22 / 47

slide-26
SLIDE 26

Motivation Contributions Video on-demand Live Streaming Conclusions Sequential fluid model

Empirical Results (2/2)

In the same scenario, we modified the number of source nodes to measure robustness to single-point failures. Sensibility to resources

1 2 3 3 4 5 6 7 8 9 10

Estimated download time Estimated download time Number of super-peers Number of super-peers

T_P2P T_P2P T_CDN T_CDN 23 / 47

slide-27
SLIDE 27

Motivation Contributions Video on-demand Live Streaming Conclusions

Outline

1

Motivation

2

Contributions

3

Video on-demand

4

Live Streaming

5

Conclusions

24 / 47

slide-28
SLIDE 28

Motivation Contributions Video on-demand Live Streaming Conclusions Goal

Objective

The Goal Define high-performance chunk scheduling policies in BitTorrent-based networks, maximizing the quality of experience of end-users. Extend the BitTorrent’s success for live services.

25 / 47

slide-29
SLIDE 29

Motivation Contributions Video on-demand Live Streaming Conclusions Goal

Live Streaming

Research Process

VIDEO STREAMING QoE Continuity Buffering Time Combinatorial Optimization Problem ACO

Model Interpretation

26 / 47

slide-30
SLIDE 30

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Pull-Mesh Cooperative Model

Assumptions

1

Single server: periodically chooses one gifted peer.

2

M identical peers with buffer capacity N.

3

Closed system.

4

Fully connected network.

5

Non-gifted peers pull at most one video-chunk per slot.

27 / 47

slide-31
SLIDE 31

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Scenario

Cooperative Scenario

S

P1 P2 P3 P4 PM

TS1 TS2

1 N 1 N

28 / 47

slide-32
SLIDE 32

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Metrology

1

Bitmap (p1, . . . , pN): occupancy probability of buffer cells.

2

Strategic sequence s1, . . . , sN−1: probability to reach cell i during a request.

3

In a massive network under regime, the following recursive bitmap holds:

  • p1

= 1

M

pi+1 = pi + (1 − pi)pisi, ∀i ∈ [N − 1]

4

The playback continuity is pN.

5

The latency or buffering time is L = N

i=1 pi.

29 / 47

slide-33
SLIDE 33

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Naive Policies

Rarest First: sRF

i

= 1 − pi

Rarest First starts here 1 N

Greedy: sG

i = 1 − 1/M − (pN − pi)

Greedy starts here 1 N 30 / 47

slide-34
SLIDE 34

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Family of Permutation-based policies

Definition

1

Denote Π the set of permutation of the objects {1, . . . , N − 1}

2

We can consider a chunk scheduling policy for each permutation, and its strategic sequence is: sπ(i) = (1 − 1 M )

i−1

  • j=1
  • 1 − pπ(j)(1 − pπ(j))
  • 3

Are we able to find the best one?

1 N

1 2 3

31 / 47

slide-35
SLIDE 35

Motivation Contributions Video on-demand Live Streaming Conclusions Model

W-Shaped Policies

By means of an ideal analysis, a distinguished polynomial Sub-Family of policies was obtained: Definition For each pair of naturals (I, J) : I + J ≤ N − 1, there is one permutation of the W-Shaped Policies, that can be expressed as follows: π(i) = N − i, i = 1, . . . , I, π(I + j) = j, j = 1, . . . , J π(I + J + k) = N + J − I 2

  • +

k 2

  • (−1)k+1,

k = 1, . . . , N − I − J − 1.

32 / 47

slide-36
SLIDE 36

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Single Score

The expected steps in a successful request following policy π is: E(Xπ) = M M − 1

N−1

  • i=1

π(i)(pi+1 − pi). Properties E(Xπ) is monotonically increasing with p(N). What is more, a joining peer will have longer requests when resourceless peers cooperate under regime. Definition The Score for the Cooperative Network Game (CNG) with a permutation π is the expected number of steps in a successful request: E(Xπ).

33 / 47

slide-37
SLIDE 37

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Cooperative Network Game

Cooperative Network Game (CNG) max

π∈ΠN−1

E(Xπ) s.t.                p1 = 1 M pi+1 = pi + pi(1 − pi)si, ∀i ∈ {1, . . . , N − 1} sπ(1) = 1 − 1 M sπ(i+1) = sπ(i) + pπ(i) − pπ(i)+1, ∀i ∈ {1, . . . , N − 2}

34 / 47

slide-38
SLIDE 38

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Ant Colony Optimization (ACO)

Blocks I ATSP ← CNG II π ← AntWorkers(ATSP) III πout ← LocalSearch(π)

Table: Performance of different chunk scheduling policies.

Policy Continuity Latency Rarest First 0.9571 21.0011 Greedy 0.9020 4.1094 Mixture 0.9970 14.4798 Average V-Shaped 0.9670 17.6683 ACO 0.9998 7.9821

35 / 47

slide-39
SLIDE 39

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Performance in GoalBit (1/3)

Buffering Time

The W-shaped policy presents the lowest buffering times:

0.2 0.4 0.6 0.8 1 20 40 60 80 100 120 140 160 180 P(L<=t) Time (s)

W-Shaped Rarest First Greedy 36 / 47

slide-40
SLIDE 40

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Performance in GoalBit (2/3)

Number of Cuts

The W-shaped policy clearly outperforms Greedy:

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 2 4 6 8 10 P(C<=n) Number of Cuts

W-Shaped Rarest First Greedy 37 / 47

slide-41
SLIDE 41

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Performance en GoalBit (3/3)

Re-Buffering time

The W-shaped policy has competitive re-buffering times:

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 20 40 60 80 100 120 140 160 180 200 P(RB<=t) Re-Buffering Time (s)

W-Shaped Rarest First Greedy 38 / 47

slide-42
SLIDE 42

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Extended Model

The extended model is represented by a fully connected closed system with 2 classes of peers collecting chunks with pulls. Entities Free-riders: selfish peers that do not cooperate.

F

Peer: normal user with unit upload.

P

Double-peers: users with double upload.

2P

Super-peers: resourceful peers with infinite upload.

SP

Server: pushes chunks to one gifted peer.

S

39 / 47

slide-43
SLIDE 43

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Mesh as a dynamic tree-structure

The best self-organization is a dynamic tree

SP SP SP S

2P 2P 2P

P P F F F P P P

40 / 47

slide-44
SLIDE 44

Motivation Contributions Video on-demand Live Streaming Conclusions Model

Extended Model

Remarkable Results

1

The system is highly scalable under full-knowledge.

2

Free-riders should never be gifted-peers.

3

If the server cannot recognize different entities, the performance dramatically deteriorates when free-riders are within the system.

4

The information travels in a natural dynamic tree structure.

5

The trade-off between full knowledge and overhead needs further research.

41 / 47

slide-45
SLIDE 45

Motivation Contributions Video on-demand Live Streaming Conclusions

Outline

1

Motivation

2

Contributions

3

Video on-demand

4

Live Streaming

5

Conclusions

42 / 47

slide-46
SLIDE 46

Motivation Contributions Video on-demand Live Streaming Conclusions Conclusions

Conclusions

1

A mathematical analysis of the two most challenging streaming modes was provided.

2

The inclusion of user-assistance in VoD systems is promising.

3

The Rarest First policy from BitTorrent is not suitable for live streaming.

4

We just give hints of new chunk scheduling policies.

5

However, the playback-delivery trade-off is not well understood yet.

43 / 47

slide-47
SLIDE 47

Motivation Contributions Video on-demand Live Streaming Conclusions Publications

Publications (1/3)

Live “Systematic Procedure for Improving Continuity and Latency on a P2P Streaming Protocol”. Bertinat, Padula, Robledo, Romero, De Vera, Rodríguez-Bocca. LATINCOM-09. Medellín, Colombia. Live “A COP for Cooperation in a P2P Streaming Protocol”. Bertinat, Padula, Robledo, Romero, De Vera, Rodríguez-Bocca, Rubino. ICUMT-09. St. Petersburg. Live “Optimum Piece Selection Strategy in GoalBit, a BitTorrent-based streaming system”. Bertinat, Padula,

  • Romero. MACI-09. Rosario.

Live “GoalBit: The First Free and Open Source Peer-to-Peer Streaming Network”. Bertinat, De Vera, Padula, Robledo, Rodríguez-Bocca, Romero, Rubino. LANC-09. Pelotas. Live “A Cooperative Network Game Efficiently Solved via an Ant Colony Optimization approach”. ANTS-10. Brussels.

44 / 47

slide-48
SLIDE 48

Motivation Contributions Video on-demand Live Streaming Conclusions Publications

Publications (2/3)

P4P “A Simple Proactive Provider Participation Technique in a Mesh-Based Peer-to-Peer Streaming Service”. Bertinat, Padula, Robledo, Rodríguez-Bocca, Romero. HAIS-11. Wroclaw. P4P “Optimal Bandwidth Allocation in Mesh-Based Peer-to-Peer Streaming Networks”. Romero, Robledo, Rodríguez-Bocca, Bertinat, Padula. INOC-11. Hamburg. Live “A Cooperative Model For Multi-Class Peer-to-Peer Streaming Networks”. ICORES-12. Algarve. VoD “Stability and Capacity of Peer-to-Peer Assisted Video-on-Demand Applications”. Robledo, Rodríguez-Bocca, Romero, Rostagnol. ICUMT-12. VoD “A new caching policy for cloud assisted Peer-to-Peer video-on-demand services”. P2PCONF-12. Tenerife.

45 / 47

slide-49
SLIDE 49

Motivation Contributions Video on-demand Live Streaming Conclusions Publications

Publications (3/3)

VoD “Analysis and Design of Peer-Assisted Video On-Demand Services”. Pablo Romero, Franco Robledo, Pablo Rodríguez-Bocca, Claudia Rostagnol. To appear in International Transactions in Operational Research. Live “An Ant-Colony approach for the design of optimal Chunk Scheduling Policies in live Peer-to-peer networks”. Pablo Romero, Franco Robledo, Pablo Rodríguez-Bocca. To appear in International Journal of Metaheuristics (IJMHeur). VoD “Optimum Piece Selection Strategies for A Peer-to-Peer Video Streaming Platform”. Pablo Romero, Franco Robledo, Pablo Rodríguez-Bocca. To appear in Computers & Operations Research.

46 / 47

slide-50
SLIDE 50

Motivation Contributions Video on-demand Live Streaming Conclusions Publications

Thanks!

UdelaR: http://www.fing.edu.uy/ Publications available at: http://goalbit.sourceforge.net/

47 / 47