MURI 2009: Heavy Tails and Long Range Dependence in Networks Barlas - - PowerPoint PPT Presentation

MURI 2009: Heavy Tails and Long Range Dependence in Networks

Barlas O˘ guz

Department of Electrical Engineering and Computer Sciences University of California, Berkeley Berkeley, California barlas@berkeley.edu

September 9, 2009

Overview

Movie distribution in hybrid P2P networks, heavy tails in demand (Barlas O˘ guz) Stochastic Approximation with LRD noise (V. Anantharam, V.S. Borkar)

Barlas O˘ guz (UCB) MURI 2009 September 2009 2 / 29

Catalog Sizing in Hybrid P2P Networks

Single server VoD

Server

Barlas O˘ guz (UCB) MURI 2009 September 2009 3 / 29

Catalog Sizing in Hybrid P2P Networks

Single server VoD

Server

Reliable, not scalable

Barlas O˘ guz (UCB) MURI 2009 September 2009 3 / 29

Catalog Sizing in Hybrid P2P Networks

Single server VoD

Server

Reliable, not scalable P2P network

Barlas O˘ guz (UCB) MURI 2009 September 2009 3 / 29

Catalog Sizing in Hybrid P2P Networks

Single server VoD

Server

Reliable, not scalable P2P network Scalable, not reliable

Barlas O˘ guz (UCB) MURI 2009 September 2009 3 / 29

Catalog Sizing in Hybrid P2P Networks

Single server VoD

Server

Reliable, not scalable P2P network Scalable, not reliable Push 2 Peer

Server

Barlas O˘ guz (UCB) MURI 2009 September 2009 3 / 29

Catalog Sizing in Hybrid P2P Networks

Single server VoD

Server

Reliable, not scalable P2P network Scalable, not reliable Push 2 Peer

Server

Scalable and reliable?

Barlas O˘ guz (UCB) MURI 2009 September 2009 3 / 29

Push to Peer VoD system

Push phase: Server ’pushes’ content onto peers during low-traffic hours. Pull phase: peers download content from eachother without further help from server

Server

Barlas O˘ guz (UCB) MURI 2009 September 2009 4 / 29

Push to Peer VoD: formulation

M nodes (peers) {i : 1, . . . , M} N movies (catalog size) {j : 1, . . . , N}. Movie j has

length Lj bits rate Rj bits/s duration Tj s

Ci, storage capacity of node i. Bupi, uplink BW of node i. dij: 1( node i demands movie j )

• i dij = dj, total demand for movie j.

Barlas O˘ guz (UCB) MURI 2009 September 2009 5 / 29

Push to Peer VoD: Push Phase MOVIES NODES

Barlas O˘ guz (UCB) MURI 2009 September 2009 6 / 29

Push to Peer VoD: Push Phase MOVIES NODES

Barlas O˘ guz (UCB) MURI 2009 September 2009 6 / 29

Push to Peer VoD: Push Phase MOVIES NODES

Barlas O˘ guz (UCB) MURI 2009 September 2009 6 / 29

Push to Peer VoD: Push Phase MOVIES NODES

Barlas O˘ guz (UCB) MURI 2009 September 2009 6 / 29

Push strategy

More formally, the push strategy should satisfy:

Push Strategy

At least 1 copy of each movie is stored

• i

cij ≥ Lj We don’t violate storage contstraints at each node

• j

cij ≤ Ci

Barlas O˘ guz (UCB) MURI 2009 September 2009 7 / 29

Pull Phase

Barlas O˘ guz (UCB) MURI 2009 September 2009 8 / 29

Pull Phase

Barlas O˘ guz (UCB) MURI 2009 September 2009 8 / 29

Pull Phase

Barlas O˘ guz (UCB) MURI 2009 September 2009 8 / 29

Pull Phase

Barlas O˘ guz (UCB) MURI 2009 September 2009 8 / 29

Pull Phase

Barlas O˘ guz (UCB) MURI 2009 September 2009 8 / 29

Pull Phase

Barlas O˘ guz (UCB) MURI 2009 September 2009 8 / 29

Pull Phase

Barlas O˘ guz (UCB) MURI 2009 September 2009 8 / 29

Pull Strategy

The pull strategy should satisfy:

Pull Strategy

rj

ik, rate at which node i downloads movie j from node k

• j,i=k rj

ik ≤ Bupk, ∀i, upload capacities are respected

sj

ik, amount of bits, node i downloads of movie j from node k

rj

ik ≥ dijsj ik, all downloads have a minimum rate condition

sj

ik ≤ cij, k sj ik ≥ Lj

Barlas O˘ guz (UCB) MURI 2009 September 2009 9 / 29

No Constraints

• Full striping optimal
• Can scale catalog size with M

Barlas O˘ guz (UCB) MURI 2009 September 2009 10 / 29

Push to Peer VoD: Scalability?

Barlas O˘ guz (UCB) MURI 2009 September 2009 11 / 29

Push to Peer VoD: Scalability?

Barlas O˘ guz (UCB) MURI 2009 September 2009 11 / 29

Push to Peer VoD: Scalability?

Barlas O˘ guz (UCB) MURI 2009 September 2009 11 / 29

Push to Peer VoD: Scalability?

The preceding problem assumes a node can connect to all its peers. Not scalable as M becomes large. Put constraint on number of incoming connections.

Connections constraint

• k=i,j

|rj

ik|0 ≤ y

Barlas O˘ guz (UCB) MURI 2009 September 2009 12 / 29

Scaling Catalog Size

Mathieu, L. et. al., 2008 [?] consider the problem of scalability of catalog size in a P2P VoD system. Given the number of nodes M, we want to achieve reliability with scaling demand. i.e. O(M) simultaneous downloads, or stability under rate O(M) per movie. How to maximally scale N with M? For arbitrary demand profiles (or adversarial), catalog scales poorly.

Barlas O˘ guz (UCB) MURI 2009 September 2009 13 / 29

What kind of distribution?

Netflix Prize Database

Pareto principle: top 15% claims 85% of demand. “The Long Tail”, niche titles also have some demand. Should use this information.

Barlas O˘ guz (UCB) MURI 2009 September 2009 14 / 29

Scaling Catalog with Power Law Demands

Can show better results if we assume power law in demand. Let Nj ∼ Poisson(Mνj) Mean parameter νj come from a power law, e.g. νj ∼ 1

jα

Come up with a good push strategy. Look at probability of sustaining random demand {Nj}.

Probability of sustaining demand

PC =

• n

1(C can sustain n).pN(N = n)

Barlas O˘ guz (UCB) MURI 2009 September 2009 15 / 29

Results

Let popularity for movie j decay as jα, where total demand increases as ρM.

Asymptotic probability of sustaining demand

(a) If α < 1, then lim

M→∞ P(all requests are satisfied) = 0

(b) If α > 1 and (yBup−1

ρ

− 4

α)(α − 1) > 1, then

lim

M→∞ P(all requests are satisfied) = 1

Barlas O˘ guz (UCB) MURI 2009 September 2009 16 / 29

Stochastic Approximation with Long Range Dependent and Heavy Tailed Noise

Venkat Anantharam1 and Vivek S. Borkar2

• 1. Department of Electrical Engineering and Computer Sciences,

University of California, Berkeley

• 2. School of Technology and Computer Science, Tata Institute of

Fundamental Research, Homi Bhabha Road, Mumbai, India

Stochastic Approximation

Given a function h, we seek to find a solution to h(x∗) = 0. However, we

• nly observe h(xn) in noise. Use the following recursion.

Algorithm

x[n + 1] = an [h(xn) + Mn+1] where originally, M is mean zero, uncorrelated, bounded variance noise.

Barlas O˘ guz (UCB) MURI 2009 September 2009 18 / 29

Stochastic Approximation

Under suitable stability conditions (e.g. sup |xn| < K), the recursion can be approximated by ODE ˙ x(t) = h(x(t)) Which can be shown to converge if an = ∞ a2

n < ∞

Barlas O˘ guz (UCB) MURI 2009 September 2009 19 / 29

Applications

Many DSP applications, including adaptive filtering Network control Adaptive routing Service time control in queuing networks In network applications, we wish to run control algorithms based on the values of the flows. However, these might not be directly observed, might be available as noisy estimates. It has been observed empirically that queues and flows in large computer networks exhibit heavy tailed distributions or long range dependence.

Barlas O˘ guz (UCB) MURI 2009 September 2009 20 / 29

Alpha stable Levy motion

Take Xi i.i.d. symmetric, P(|X1| > x) = x−αL(x) then Snt (nL(n))

1 α

→d SαS (symmetric α-stable Levy motion)

Barlas O˘ guz (UCB) MURI 2009 September 2009 21 / 29

Alpha-stable Levy Motion properties

stationary, α-stable, i.i.d. increments. Distribution of Snt

√n → ∞ (long range dependence)

Var(St) = ∞ Self-similarity: Snt =d n

1 α St

Samorodnitsky, Taqqu. “Stable Non-Gaussian Random Processes: Stochastic models with infinite variance”

Barlas O˘ guz (UCB) MURI 2009 September 2009 22 / 29

Fractional Brownian Motion

Fractional Brownian Motion is the unique Gaussian H-sssi process. Cov(BH(t1), BH(t2)) = 1

2

• |t1|2H + |t2|2H − |t1 − t2|2H

Var(BH(1)) H-sssi

fBM limit

Let Cov(X1, Xn) = n−αL(n) regularly varying. And {Xi} zero-mean Gaussian. Then, Snt

nH →d BH(t), where H = (1 − α 2 ).

Barlas O˘ guz (UCB) MURI 2009 September 2009 23 / 29

Fractional Brownian Motion

fractional Brownian motion − parameter: 0.9 Barlas O˘ guz (UCB) MURI 2009 September 2009 24 / 29

Stochastic Approximation with LRD and Heavy Tailed Noise

xn+1 = xn + a(n)[h(xn) + Mn+1 + R(n)Bn+1 + D(n)Sn+1 + ξn+1], where h : Rd → Rd is Lipschitz, Bn+1 := ˜ B(n + 1) − ˜ B(n), where ˜ B(t), t ≥ 0, is a fractional Brownian motion with Hurst parameter ν ∈ (0, 1), Sn+1 := ˜ S(n + 1) − ˜ S(n), where ˜ S(t), t ≥ 0, is a symmetric α-stable process with 1 < α < 2, ξn is an ’error’ process satisfying supn ||ξn|| ≤ K0 < ∞ a.s. and ξn → 0 a.s.,

Barlas O˘ guz (UCB) MURI 2009 September 2009 25 / 29

R(n), D(n) are bounded sequences of d by d random matrices adapted to Fn, for Fn := σ(xi, Bi, Mi, Si, ξi, i ≤ n). Mn is a martingale difference sequence w.r.t. Fn satisfying E[||Mn+1||2|Fn] ≤ K1(1 + ||xn||2) a(n) are positive non-increasing stepsizes which are Θ(n−κ) for some κ ∈ (1, 1]. In particular, they satisfy:

• n

a(n) = ∞,

• n

a(n)2 < ∞ which are standard conditions for stochastic approximation. We need the additional restriction: κ > ν

Barlas O˘ guz (UCB) MURI 2009 September 2009 26 / 29

Main Result

Consider the related o.d.e.: ˙ x(t) = h(x(t)) Assume this o.d.e. has unique asymptotically stable equilibrium x , with continuously differentiable Liapunov function. V : Rd → R+ satisfying lim||x||→∞ V (x) = ∞ and ∇V , h(x) < 0 for x = x. In turn, existence of such a V implies global asymptotic stability of x. Then,

Theorem 1

Suppose (∗)K2 := sup

n

E[xn] < ∞ for some ξ ∈ [1, α) . Then for 1 < ξ′ < ξ, E[||xn − x||ξ′] → 0 e.g. h = −∇F for some F to be minimized.

Barlas O˘ guz (UCB) MURI 2009 September 2009 27 / 29

Sufficient Condition

Let hc(x) := h(cx)

c

for c > 0. We assume h∞(x) := lim

c→∞ hc(x)

• exists. Consider the o.d.e.

˙ xc(t) = hc(xc(t)) (**) for 0 < c ≤ ∞. The following condition is sufficient for (∗): For c = ∞, (**) has the origin as the globally exponentially stable equilibrium.

Barlas O˘ guz (UCB) MURI 2009 September 2009 28 / 29

a.s. convergence

Theorem 2

If either D(n) ≡ 0, or κ > 1

α and

sup

n

||xn|| < ∞ then xn → x∗ a.s.

Barlas O˘ guz (UCB) MURI 2009 September 2009 29 / 29