Distributed Algorithms for Content Allocation in Interconnected - - PowerPoint PPT Presentation

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Distributed Algorithms for Content Allocation in Interconnected Content Distribution Networks

Valentino Pacifici, Gy¨

• rgy D´

an

Laboratory for Communication Networks KTH Royal Institute of Technology Stockholm, Sweden

Hong Kong, April 30, 2015

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 1 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Booming Content Delivery Market

Content Distribution in the Internet

❼ 2009: P2P traffic → up to 70 % of Internet traffic ❼ 2013: Netflix + YouTube → 50 % fixed network traffic ❼ 2017: Video traffic → 80 % of IP traffic ❼ ❼ ❼ ❼ ❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 2 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Booming Content Delivery Market

Content Distribution in the Internet

❼ 2009: P2P traffic → up to 70 % of Internet traffic ❼ 2013: Netflix + YouTube → 50 % fixed network traffic ❼ 2017: Video traffic → 80 % of IP traffic

Over-the-top Content providers

❼ ↑ Quality of delivered content ❼ ↑ User demand for content ❼ ↑ Revenue ❼ ❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 2 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Booming Content Delivery Market

Content Distribution in the Internet

❼ 2009: P2P traffic → up to 70 % of Internet traffic ❼ 2013: Netflix + YouTube → 50 % fixed network traffic ❼ 2017: Video traffic → 80 % of IP traffic

Over-the-top Content providers

❼ ↑ Quality of delivered content ❼ ↑ User demand for content ❼ ↑ Revenue

Content Delivery Networks (CDNs)

❼ ↑ Delivered content on behalf of OTT providers ❼ ↑ 2017: delivery of 2

3 of total video traffic

❼ ↑ Revenue

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 2 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Network Operator Managed CDNs

Network operators

❼ Content distribution stresses network infrastructure (OTT, P2P) ❼ Network operators not part of revenue chain

↓

❼ ❼ ❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 3 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Network Operator Managed CDNs

Network operators

❼ Content distribution stresses network infrastructure (OTT, P2P) ❼ Network operators not part of revenue chain

↓

Network Operator Managed CDNs (nCDNs)

❼ Storage of content close to the customers ❼ Objectives:

1 Improve user’s QoE → decrease latency 2 Decrease traffic cost → decrease network traffic

❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 3 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Network Operator Managed CDNs

Network operators

❼ Content distribution stresses network infrastructure (OTT, P2P) ❼ Network operators not part of revenue chain

↓

Network Operator Managed CDNs (nCDNs)

❼ Storage of content close to the customers ❼ Objectives:

1 Improve user’s QoE → decrease latency 2 Decrease traffic cost → decrease network traffic

Content Allocation Problem

❼ nCDNs periodically update content allocation based on predicted demands ❼ Pre-fetching: nCDN i decides on allocation Ai ∈ O and fetches the content

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 3 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Autonomous Content Delivery Networks

• 2O

A1½O O

ISP 1

❼ nCDN optimized for local performance ❼

❼

❼ ❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 4 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Autonomous Content Delivery Networks

A1

O ISP 1 ❼ nCDN optimized for local performance ❼

❼

❼ ❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 4 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Autonomous Content Delivery Networks

A2 A3 A1

O ISP 3 ISP 2 ISP 1 ❼ nCDN optimized for local performance Interconnected nCDNs ❼ Maximize users’ QoE

❼ Retrieve items from connected nCDNs with lowest latency

❼ ❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 4 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Autonomous Content Delivery Networks

A2 A3 A1

O ISP 3 ISP 2 ISP 1 ❼ nCDN optimized for local performance Interconnected nCDNs ❼ Maximize users’ QoE

❼ Retrieve items from connected nCDNs with lowest latency

Cost for serving one request (latency) ❼ αi from own nCDN i ❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 4 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Autonomous Content Delivery Networks

A2 A3 A1

O ISP 3 ISP 2 ISP 1 ❼ nCDN optimized for local performance Interconnected nCDNs ❼ Maximize users’ QoE

❼ Retrieve items from connected nCDNs with lowest latency

Cost for serving one request (latency) ❼ αi from own nCDN i ❼ βj

i from connected nCDN j

❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 4 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Autonomous Content Delivery Networks

A2 A3 A1

O ISP 3 ISP 2 ISP 1 ❼ nCDN optimized for local performance Interconnected nCDNs ❼ Maximize users’ QoE

❼ Retrieve items from connected nCDNs with lowest latency

Cost for serving one request (latency) ❼ αi from own nCDN i ❼ βj

i from connected nCDN j

❼ γi from content provider αi ≤ βj

i < γi

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 4 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Interconnected nCDNs - The Cost Model

❼ Ki storage capacity of nCDN i ❼ wo

i ∈ R+ demand for item o ∈ O at nCDN i

❼ Ri =

j∈N(i) Aj → content available from connected nCDNs

❼ βo

i (A−i) minj∈N(i){βj i |o ∈ Aj} → lowest latency to retrieve item o

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 5 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Interconnected nCDNs - The Cost Model

❼ Ki storage capacity of nCDN i ❼ wo

i ∈ R+ demand for item o ∈ O at nCDN i

❼ Ri =

j∈N(i) Aj → content available from connected nCDNs

❼ βo

i (A−i) minj∈N(i){βj i |o ∈ Aj} → lowest latency to retrieve item o

Average latency experienced by customers of operator i :

Ci(Ai, A−i) =

• Ai

wo

i αi

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 5 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Interconnected nCDNs - The Cost Model

❼ Ki storage capacity of nCDN i ❼ wo

i ∈ R+ demand for item o ∈ O at nCDN i

❼ Ri =

j∈N(i) Aj → content available from connected nCDNs

❼ βo

i (A−i) minj∈N(i){βj i |o ∈ Aj} → lowest latency to retrieve item o

Average latency experienced by customers of operator i :

Ci(Ai, A−i) =

• Ai

wo

i αi +

• Ri

wo

i βo i (A−i)

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 5 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Interconnected nCDNs - The Cost Model

❼ Ki storage capacity of nCDN i ❼ wo

i ∈ R+ demand for item o ∈ O at nCDN i

❼ Ri =

j∈N(i) Aj → content available from connected nCDNs

❼ βo

i (A−i) minj∈N(i){βj i |o ∈ Aj} → lowest latency to retrieve item o

Average latency experienced by customers of operator i :

Ci(Ai, A−i) =

• Ai

wo

i αi +

• Ri

wo

i βo i (A−i) +

• O{Ai∪Ri}

wo

i γi

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 5 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Objective

Distributed algorithm - desiderata

1 nCDN i exchange information only with connected N(i)

❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 6 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Objective

Distributed algorithm - desiderata

1 nCDN i exchange information only with connected N(i) 2 Reveals little information about wo i

❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 6 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Objective

Distributed algorithm - desiderata

1 nCDN i exchange information only with connected N(i) 2 Reveals little information about wo i 3 Leads to content allocation ¯

A = (Ai)i∈N that is individually rational

❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 6 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Objective

Distributed algorithm - desiderata

1 nCDN i exchange information only with connected N(i) 2 Reveals little information about wo i 3 Leads to content allocation ¯

A = (Ai)i∈N that is individually rational

Individual rationality

❼ ri(A) cost saving ratio for nCDN i in allocation A

ri(A) = cost saving coop. cost saving no coop.

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 6 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Objective

Distributed algorithm - desiderata

1 nCDN i exchange information only with connected N(i) 2 Reveals little information about wo i 3 Leads to content allocation ¯

A = (Ai)i∈N that is individually rational

Individual rationality

❼ ri(A) cost saving ratio for nCDN i in allocation A

ri(A) = cost saving coop. cost saving no coop. ri(A) > 1 nCDN i benefits from cooperation

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 6 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Objective

Distributed algorithm - desiderata

1 nCDN i exchange information only with connected N(i) 2 Reveals little information about wo i 3 Leads to content allocation ¯

A = (Ai)i∈N that is individually rational

Individual rationality

❼ ri(A) cost saving ratio for nCDN i in allocation A

ri(A) = cost saving coop. cost saving no coop. ri(A) > 1 nCDN i benefits from cooperation ri(A) ≥ 1 Ai individually rational

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 6 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Self-enforcing Content Allocations

❼ Every nCDN i allocates Ai to minimize Ci(Ai, A−i) given A−i ❼ Compatible with operators’ selfish interests ❼ ❼ ❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 7 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Self-enforcing Content Allocations

❼ Every nCDN i allocates Ai to minimize Ci(Ai, A−i) given A−i ❼ Compatible with operators’ selfish interests ❼ Enforced without bilateral payments → self-enforcing allocation

(A∗

i )i∈N s.t. A∗ i ∈ arg minAi Ci(Ai, A∗ −i).

⇓ Nash Equilibrium of strategic game Γ =< N, (Ai)i∈N, (Ci)i∈N >

❼ ❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 7 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Self-enforcing Content Allocations

❼ Every nCDN i allocates Ai to minimize Ci(Ai, A−i) given A−i ❼ Compatible with operators’ selfish interests ❼ Enforced without bilateral payments → self-enforcing allocation

(A∗

i )i∈N s.t. A∗ i ∈ arg minAi Ci(Ai, A∗ −i).

⇓ Nash Equilibrium of strategic game Γ =< N, (Ai)i∈N, (Ci)i∈N >

Distributed Local-Greedy Algorithm

❼ For each nCDN i ❼ Compute cost saving for each o ∈ O given A−i CSo

i (1, A−i) =

wo

i [γi − αi]

if o / ∈ Ri wo

i [βo i (A−i) − αi]

if o ∈ Ri ❼ Store Ki items with highest cost saving

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 7 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Failure of Distributed Local-Greedy

Open questions ❼ ∃ self-enforcing content allocation A∗ = (A∗

i )i∈N

❼ Convergence of Distributed Local-Greedy to A∗ ❼ ❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 8 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Failure of Distributed Local-Greedy

Open questions ❼ ∃ self-enforcing content allocation A∗ = (A∗

i )i∈N

❼ Convergence of Distributed Local-Greedy to A∗ Example ❼ nCDNs N = {1, . . . , 5} ❼ Content items O = {a, b, c, d} ❼ ∃ αi, βj

i , γi and wo i

1 2 3 4 5

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 8 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Failure of Distributed Local-Greedy

Open questions ❼ ∃ self-enforcing content allocation A∗ = (A∗

i )i∈N

❼ Convergence of Distributed Local-Greedy to A∗ Example ❼ nCDNs N = {1, . . . , 5} ❼ Content items O = {a, b, c, d} ❼ ∃ αi, βj

i , γi and wo i

1 2 3 4 5 (a, b, c, d) − →

4

(a, b, c, b) − →

3

(a, b, d, b) − →

2

(a, c, d, b) − →

1

(b, c, d, b) − →

4

(b, c, d, d) − →

3

(b, c, c, d) − →

2

(b, b, c, d) − →

1

(a, b, c, d)

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 8 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Failure of Distributed Local-Greedy

Open questions ❼ ∃ self-enforcing content allocation A∗ = (A∗

i )i∈N

❼ Convergence of Distributed Local-Greedy to A∗ Example ❼ nCDNs N = {1, . . . , 5} ❼ Content items O = {a, b, c, d} ❼ ∃ αi, βj

i , γi and wo i

1 2 3 4 5 (a, b, c, d) − →

4

(a, b, c, b) − →

3

(a, b, d, b) − →

2

(a, c, d, b) − →

1

(b, c, d, b) − →

4

(b, c, d, d) − →

3

(b, c, c, d) − →

2

(b, b, c, d) − →

1

(a, b, c, d)

Theorem: Self-enforcing content allocations might not exist

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 8 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Failure of Distributed Local-Greedy

Open questions ❼ ∃ self-enforcing content allocation A∗ = (A∗

i )i∈N

❼ Convergence of Distributed Local-Greedy to A∗ Example ❼ nCDNs N = {1, . . . , 5} ❼ Content items O = {a, b, c, d} ❼ ∃ αi, βj

i , γi and wo i

1 2 3 4 5 (a, b, c, d) − →

4

(a, b, c, b) − →

3

(a, b, d, b) − →

2

(a, c, d, b) − →

1

(b, c, d, b) − →

4

(b, c, d, d) − →

3

(b, c, c, d) − →

2

(b, b, c, d) − →

1

(a, b, c, d)

Theorem: Self-enforcing content allocations might not exist Corollary: αi = α,γi = γ, and βj

i = βi j do not help!

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 8 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Bilateral Compensation-based Algorithms

❼ Introduce periodic bilateral payments among nCDNs ❼ Open question: ∃ stable content allocation A ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 9 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Bilateral Compensation-based Algorithms

❼ Introduce periodic bilateral payments among nCDNs ❼ Open question: ∃ stable content allocation A

Bilateral compensation-based algorithms in a nutshell

❼ At every time step t

1 nCDNs in set Nt allowed to propose allocation update 2 Connected nCDNs j ∈ N(Nt) can offer payments to dissuade Nt

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 9 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Bilateral Compensation-based Algorithms

❼ Introduce periodic bilateral payments among nCDNs ❼ Open question: ∃ stable content allocation A

Bilateral compensation-based algorithms in a nutshell

❼ At every time step t

1 nCDNs in set Nt allowed to propose allocation update 2 Connected nCDNs j ∈ N(Nt) can offer payments to dissuade Nt

Synchronization schemes

Asynchronous |Nt| = 1 Plesiochronous Nt ⊂ N Synchronous Nt = N

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 9 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Aggregate-value Compensation Algorithm (AC)

❼ At every time step t

1 ∀ nCDN i ∈ Nt computes Ai(t) that decreases its cost, i.e. ∆Ci < 0

❼

❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 10 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Aggregate-value Compensation Algorithm (AC)

❼ At every time step t

1 ∀ nCDN i ∈ Nt computes Ai(t) that decreases its cost, i.e. ∆Ci < 0 2 nCDN i communicates Ai(t) to all neighboring nCDNs j ∈ N(i)

❼

❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 10 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Aggregate-value Compensation Algorithm (AC)

❼ At every time step t

1 ∀ nCDN i ∈ Nt computes Ai(t) that decreases its cost, i.e. ∆Ci < 0 2 nCDN i communicates Ai(t) to all neighboring nCDNs j ∈ N(i) 3 All nCDNs j ∈ N(i) s.t. ∆Cj > 0 offer a compensation pi

j = ∆Cj

❼

❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 10 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Aggregate-value Compensation Algorithm (AC)

❼ At every time step t

1 ∀ nCDN i ∈ Nt computes Ai(t) that decreases its cost, i.e. ∆Ci < 0 2 nCDN i communicates Ai(t) to all neighboring nCDNs j ∈ N(i) 3 All nCDNs j ∈ N(i) s.t. ∆Cj > 0 offer a compensation pi

j = ∆Cj

4 nCDN i updates its content allocation to Ai(t) only if

• j∈N (i)

pi

j < −∆Ci(t)

❼

❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 10 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Aggregate-value Compensation Algorithm (AC)

❼ At every time step t

1 ∀ nCDN i ∈ Nt computes Ai(t) that decreases its cost, i.e. ∆Ci < 0 2 nCDN i communicates Ai(t) to all neighboring nCDNs j ∈ N(i) 3 All nCDNs j ∈ N(i) s.t. ∆Cj > 0 offer a compensation pi

j = ∆Cj

4 nCDN i updates its content allocation to Ai(t) only if

• j∈N (i)

pi

j < −∆Ci(t)

Asynchronous operation, |Nt| = 1 Theorem: AC algorithm terminates in a finite number of steps (1-AC) ❼

❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 10 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Aggregate-value Compensation Algorithm (AC)

❼ At every time step t

1 ∀ nCDN i ∈ Nt computes Ai(t) that decreases its cost, i.e. ∆Ci < 0 2 nCDN i communicates Ai(t) to all neighboring nCDNs j ∈ N(i) 3 All nCDNs j ∈ N(i) s.t. ∆Cj > 0 offer a compensation pi

j = ∆Cj

4 nCDN i updates its content allocation to Ai(t) only if

• j∈N (i)

pi

j < −∆Ci(t)

Asynchronous operation, |Nt| = 1 Theorem: AC algorithm terminates in a finite number of steps (1-AC) Plesiochronous operation, Nt ⊂ N Theorem: If Nt is 2-independent set of G ⇒ AC algorithm terminates in a finite number of steps (I2-AC) ❼ Distance between any two vertexes of Nt is at least 3

❼ Few nodes update at each time step ❼ Needs coordination scheme over 2-hops neighbours t

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 10 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Object-value Compensation Algorithm (OC)

❼ At every time step t

1 ∀ nCDN i ∈ Nt computes Ai(t) that decreases its cost, i.e. ∆Ci < 0 2 nCDN i communicates Ai(t) to all neighboring nCDNs j ∈ N(i)

❼ ❼

❼ ❫ ❼ ❴ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 11 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Object-value Compensation Algorithm (OC)

❼ At every time step t

1 ∀ nCDN i ∈ Nt computes Ai(t) that decreases its cost, i.e. ∆Ci < 0 2 nCDN i communicates Ai(t) to all neighboring nCDNs j ∈ N(i) 3 j ∈ N(Nt) offers compensation pk

j,o = ∆Co j for each individual

item o to nCDN k ∈ Nt s.t

❼ ❼

❼ ❫ ❼ ❴ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 11 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Object-value Compensation Algorithm (OC)

❼ At every time step t

1 ∀ nCDN i ∈ Nt computes Ai(t) that decreases its cost, i.e. ∆Ci < 0 2 nCDN i communicates Ai(t) to all neighboring nCDNs j ∈ N(i) 3 j ∈ N(Nt) offers compensation pk

j,o = ∆Co j for each individual

item o to nCDN k ∈ Nt s.t

❼ nCDN k evicting item o and ❼ lowest latency, i.e. k = mini∈N (j){βi

j|o ∈ Ai}

❼ ❫ ❼ ❴ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 11 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Object-value Compensation Algorithm (OC)

❼ At every time step t

1 ∀ nCDN i ∈ Nt computes Ai(t) that decreases its cost, i.e. ∆Ci < 0 2 nCDN i communicates Ai(t) to all neighboring nCDNs j ∈ N(i) 3 j ∈ N(Nt) offers compensation pk

j,o = ∆Co j for each individual

item o to nCDN k ∈ Nt s.t

❼ nCDN k evicting item o and ❼ lowest latency, i.e. k = mini∈N (j){βi

j|o ∈ Ai}

4 nCDN i updates its content allocation to Ai(t) only if

• j∈N (i)
• ∈O

pi

j,o < −∆Ci(t)

❼ ❫ ❼ ❴ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 11 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Object-value Compensation Algorithm (OC)

❼ At every time step t

1 ∀ nCDN i ∈ Nt computes Ai(t) that decreases its cost, i.e. ∆Ci < 0 2 nCDN i communicates Ai(t) to all neighboring nCDNs j ∈ N(i) 3 j ∈ N(Nt) offers compensation pk

j,o = ∆Co j for each individual

item o to nCDN k ∈ Nt s.t

❼ nCDN k evicting item o and ❼ lowest latency, i.e. k = mini∈N (j){βi

j|o ∈ Ai}

4 nCDN i updates its content allocation to Ai(t) only if

• j∈N (i)
• ∈O

pi

j,o < −∆Ci(t)

Plesiochronous operation, Nt ⊂ N Theorem: If Nt is 1-independent set of G ⇒ OC algorithm terminates in a finite number of steps (I1-OC) ❼ ❫ ❼ ❴ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 11 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Object-value Compensation Algorithm (OC)

❼ At every time step t

1 ∀ nCDN i ∈ Nt computes Ai(t) that decreases its cost, i.e. ∆Ci < 0 2 nCDN i communicates Ai(t) to all neighboring nCDNs j ∈ N(i) 3 j ∈ N(Nt) offers compensation pk

j,o = ∆Co j for each individual

item o to nCDN k ∈ Nt s.t

❼ nCDN k evicting item o and ❼ lowest latency, i.e. k = mini∈N (j){βi

j|o ∈ Ai}

4 nCDN i updates its content allocation to Ai(t) only if

• j∈N (i)
• ∈O

pi

j,o < −∆Ci(t)

Plesiochronous operation, Nt ⊂ N Theorem: If Nt is 1-independent set of G ⇒ OC algorithm terminates in a finite number of steps (I1-OC) ❼ ❫ On average |I1| >> |I2| ❼ ❴ I1-OC reveals more information about wo

i

❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 11 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Object-value Compensation Algorithm (OC)

❼ At every time step t

1 ∀ nCDN i ∈ Nt computes Ai(t) that decreases its cost, i.e. ∆Ci < 0 2 nCDN i communicates Ai(t) to all neighboring nCDNs j ∈ N(i) 3 j ∈ N(Nt) offers compensation pk

j,o = ∆Co j for each individual

item o to nCDN k ∈ Nt s.t

❼ nCDN k evicting item o and ❼ lowest latency, i.e. k = mini∈N (j){βi

j|o ∈ Ai}

4 nCDN i updates its content allocation to Ai(t) only if

• j∈N (i)
• ∈O

pi

j,o < −∆Ci(t)

Plesiochronous operation, Nt ⊂ N Theorem: If Nt is 1-independent set of G ⇒ OC algorithm terminates in a finite number of steps (I1-OC) ❼ ❫ On average |I1| >> |I2| ❼ ❴ I1-OC reveals more information about wo

i

❼ Neither AC nor OC ensure individual rationality!

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 11 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Achieving Individual Rationality

Opt Out scheme

❼ Set Ncoop ← N, until individual rationality achieved do

1 Run algorithm AC or OC ← termination in A 2 Exlcude nCDNs k s.t. rk(A) < 1,

❼ Ncoop ← {i|ri(A) ≥ 1} ❼ Excluded nCDNs k do not cooperate, i.e. AI

k

❼ ❼ ❼ ❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 12 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Achieving Individual Rationality

Opt Out scheme

❼ Set Ncoop ← N, until individual rationality achieved do

1 Run algorithm AC or OC ← termination in A 2 Exlcude nCDNs k s.t. rk(A) < 1,

❼ Ncoop ← {i|ri(A) ≥ 1} ❼ Excluded nCDNs k do not cooperate, i.e. AI

k

Numerical Results - Impact of graph topology

❼ CAIDA dataset - AS-level topology ❼ European ASes with > 216 alloc. IPs ❼ CAIDA: |N| = 638, avg. node degree 10.8 ❼ CAIDA-ER: Erd˝

• s-R´

enyi

❼ CAIDA-BA: Barab´

asi-Albert

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 12 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Achieving Individual Rationality

Opt Out scheme

❼ Set Ncoop ← N, until individual rationality achieved do

1 Run algorithm AC or OC ← termination in A 2 Exlcude nCDNs k s.t. rk(A) < 1,

❼ Ncoop ← {i|ri(A) ≥ 1} ❼ Excluded nCDNs k do not cooperate, i.e. AI

k

Numerical Results - Impact of graph topology

❼ CAIDA dataset - AS-level topology ❼ European ASes with > 216 alloc. IPs ❼ CAIDA: |N| = 638, avg. node degree 10.8 ❼ CAIDA-ER: Erd˝

• s-R´

enyi

❼ CAIDA-BA: Barab´

asi-Albert

0.5 1 1.5 2 2.5 0.5 1 1.5 2 2.5 3 3.5 4 4.5

Cost saving ratio (ri( ¯ A)) Probability density estimate 1-AC I 2-AC I 1-OC CAIDA CAIDA-BA CAIDA-ER

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 12 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Achieving Individual Rationality

Opt Out scheme

❼ Set Ncoop ← N, until individual rationality achieved do

1 Run algorithm AC or OC ← termination in A 2 Exlcude nCDNs k s.t. rk(A) < 1,

❼ Ncoop ← {i|ri(A) ≥ 1} ❼ Excluded nCDNs k do not cooperate, i.e. AI

k

Numerical Results - Impact of graph topology

❼ CAIDA dataset - AS-level topology ❼ European ASes with > 216 alloc. IPs ❼ CAIDA: |N| = 638, avg. node degree 10.8 ❼ CAIDA-ER: Erd˝

• s-R´

enyi

❼ CAIDA-BA: Barab´

asi-Albert

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 480 500 520 540 560 580 600 620 640

• Avg. # of cooperating nCDNs at ¯

A Edge ratio (d)

1 1.1 1.2 1.3

• Avg. cost saving ratio (ri( ¯

A)) 1-AC I 2-AC I 1-OC # coop. nCDNs

• Avg. ri( ¯

C)

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 12 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Numerical Results - Convergence Rate

1000 2000 3000 4000 5000 6000 7000 8000 9000 10

−2

10

−1

10

Number of time steps to reach Aℓ CCDF 1-AC I 2-AC I 1-OC CAIDA CAIDA-BA CAIDA-ER

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 13 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Numerical Results - Convergence Rate

1000 2000 3000 4000 5000 6000 7000 8000 9000 10

−2

10

−1

10

Number of time steps to reach Aℓ CCDF 1-AC I 2-AC I 1-OC CAIDA CAIDA-BA CAIDA-ER

Graph I1 sets avg. size I2 sets avg. size CAIDA 39.8 2.9 CAIDA-BA 63.8 4.9 CAIDA-ER 79.8 17.8

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 13 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Numerical Results - Scaling for Storage Capacity

❼ Increasing storage capacity Ki at every nCDN i ∈ N ❼ ❼ ❼ ❼ ❫

❫

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 14 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Numerical Results - Scaling for Storage Capacity

❼ Increasing storage capacity Ki at every nCDN i ∈ N

10

1

10

2

10

3

10

1

10

2

10

3

10

4

Storage capacity (Ki) Number of time steps to reach Aℓ 1-AC I 2-AC I 1-OC CAIDA CAIDA-BA

❼ ❼ ❼ ❼ ❫

❫

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 14 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Numerical Results - Scaling for Storage Capacity

❼ Increasing storage capacity Ki at every nCDN i ∈ N

10

1

10

2

10

3

10

1

10

2

10

3

10

4

Storage capacity (Ki) Number of time steps to reach Aℓ 1-AC I 2-AC I 1-OC CAIDA CAIDA-BA

❼ Convergence rate insensitive to Ki ❼ ❼ ❼ ❫

❫

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 14 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Numerical Results - Scaling for Storage Capacity

❼ Increasing storage capacity Ki at every nCDN i ∈ N

10

1

10

2

10

3

10

1

10

2

10

3

10

4

Storage capacity (Ki) Number of time steps to reach Aℓ 1-AC I 2-AC I 1-OC CAIDA CAIDA-BA

10

1

10

2

10

3

10

2.7

10

2.8

Storage capacity (Ki) Number of item updates to reach Aℓ 1-AC I2-AC I1-OC CAIDA CAIDA-BA

❼ Convergence rate insensitive to Ki ❼ ❼ ❼ ❫

❫

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 14 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Numerical Results - Scaling for Storage Capacity

❼ Increasing storage capacity Ki at every nCDN i ∈ N

10

1

10

2

10

3

10

1

10

2

10

3

10

4

Storage capacity (Ki) Number of time steps to reach Aℓ 1-AC I 2-AC I 1-OC CAIDA CAIDA-BA

10

1

10

2

10

3

10

2.7

10

2.8

Storage capacity (Ki) Number of item updates to reach Aℓ 1-AC I2-AC I1-OC CAIDA CAIDA-BA

❼ Convergence rate insensitive to Ki ❼ ∗-AC: Same # of item updates ❼ I1-OC more efficient ❼ ❫ # item updates = ❫ conv. rate

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 14 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Conclusion

Interconnected Operator Managed CDNs ❼ Retrieval of content from connected nCDNs ❼ Individually rational content allocation ❼ ❼ ❼ ❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 15 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Conclusion

Interconnected Operator Managed CDNs ❼ Retrieval of content from connected nCDNs ❼ Individually rational content allocation Self-enforcing Content Allocations ❼ Distributed greedy cost minimization not suitable ❼ ❼ ❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 15 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Conclusion

Interconnected Operator Managed CDNs ❼ Retrieval of content from connected nCDNs ❼ Individually rational content allocation Self-enforcing Content Allocations ❼ Distributed greedy cost minimization not suitable Stable Allocations ❼ Bilateral compensation-based algorithms reach stable allocations ❼ Faster convergence at the cost of revealing information ❼ ❼

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 15 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Conclusion

Interconnected Operator Managed CDNs ❼ Retrieval of content from connected nCDNs ❼ Individually rational content allocation Self-enforcing Content Allocations ❼ Distributed greedy cost minimization not suitable Stable Allocations ❼ Bilateral compensation-based algorithms reach stable allocations ❼ Faster convergence at the cost of revealing information Individual Rationality ❼ Individual rationality of stable allocations depends on graph topology ❼ At least 80% of nCDNs have incentive to cooperate

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 15 / 16

Problem Definition Failure of Distributed Greedy Bilateral Compensation-based Algorithms Conclusion

Distributed Algorithms for Content Allocation in Interconnected Content Distribution Networks

Valentino Pacifici, Gy¨

• rgy D´

an

Laboratory for Communication Networks KTH Royal Institute of Technology Stockholm, Sweden

Hong Kong, April 30, 2015

• V. Pacifici, G. D´

an (EE,KTH) Replication in Interconnected nCDNs Apr 30, 2015 16 / 16