MURI 2009: Heavy Tails and Long Range Dependence in Networks Barlas - PowerPoint PPT Presentation

MURI 2009: Heavy Tails and Long Range Dependence in Networks Barlas O˘ guz Department of Electrical Engineering and Computer Sciences University of California, Berkeley Berkeley, California barlas@berkeley.edu September 9, 2009

Overview Movie distribution in hybrid P2P networks, heavy tails in demand (Barlas O˘ guz) Stochastic Approximation with LRD noise (V. Anantharam, V.S. Borkar) Barlas O˘ guz (UCB) MURI 2009 September 2009 2 / 29

Catalog Sizing in Hybrid P2P Networks Single server VoD Server Barlas O˘ guz (UCB) MURI 2009 September 2009 3 / 29

Catalog Sizing in Hybrid P2P Networks Single server VoD Server Reliable, not scalable Barlas O˘ guz (UCB) MURI 2009 September 2009 3 / 29

Catalog Sizing in Hybrid P2P Networks Single server VoD P2P network Server Reliable, not scalable Barlas O˘ guz (UCB) MURI 2009 September 2009 3 / 29

Catalog Sizing in Hybrid P2P Networks Single server VoD P2P network Server Reliable, not scalable Scalable, not reliable Barlas O˘ guz (UCB) MURI 2009 September 2009 3 / 29

Catalog Sizing in Hybrid P2P Networks Single server VoD P2P network Push 2 Peer Server Server Reliable, not scalable Scalable, not reliable Barlas O˘ guz (UCB) MURI 2009 September 2009 3 / 29

Catalog Sizing in Hybrid P2P Networks Single server VoD P2P network Push 2 Peer Server Server Reliable, not scalable Scalable, not reliable Scalable and reliable? Barlas O˘ guz (UCB) MURI 2009 September 2009 3 / 29

Push to Peer VoD system Push phase: Server ’pushes’ content onto Server peers during low-traffic hours. Pull phase: peers download content from eachother without further help from server Barlas O˘ guz (UCB) MURI 2009 September 2009 4 / 29

Push to Peer VoD: formulation M nodes (peers) { i : 1 , . . . , M } N movies (catalog size) { j : 1 , . . . , N } . Movie j has length L j bits rate R j bits/s duration T j s C i , storage capacity of node i . B up i , uplink BW of node i . d ij : 1( node i demands movie j ) � i d ij = d j , total demand for movie j . Barlas O˘ guz (UCB) MURI 2009 September 2009 5 / 29

Push to Peer VoD: Push Phase MOVIES NODES Barlas O˘ guz (UCB) MURI 2009 September 2009 6 / 29

Push to Peer VoD: Push Phase MOVIES NODES Barlas O˘ guz (UCB) MURI 2009 September 2009 6 / 29

Push to Peer VoD: Push Phase MOVIES NODES Barlas O˘ guz (UCB) MURI 2009 September 2009 6 / 29

Push to Peer VoD: Push Phase MOVIES NODES Barlas O˘ guz (UCB) MURI 2009 September 2009 6 / 29

Push strategy More formally, the push strategy should satisfy: Push Strategy At least 1 copy of each movie is stored � c ij ≥ L j i We don’t violate storage contstraints at each node � c ij ≤ C i j Barlas O˘ guz (UCB) MURI 2009 September 2009 7 / 29

Pull Phase Barlas O˘ guz (UCB) MURI 2009 September 2009 8 / 29

Pull Phase Barlas O˘ guz (UCB) MURI 2009 September 2009 8 / 29

Pull Phase Barlas O˘ guz (UCB) MURI 2009 September 2009 8 / 29

Pull Phase Barlas O˘ guz (UCB) MURI 2009 September 2009 8 / 29

Pull Phase Barlas O˘ guz (UCB) MURI 2009 September 2009 8 / 29

Pull Phase Barlas O˘ guz (UCB) MURI 2009 September 2009 8 / 29

Pull Phase Barlas O˘ guz (UCB) MURI 2009 September 2009 8 / 29

Pull Strategy The pull strategy should satisfy: Pull Strategy r j ik , rate at which node i downloads movie j from node k j , i � = k r j � ik ≤ B up k , ∀ i , upload capacities are respected s j ik , amount of bits, node i downloads of movie j from node k r j ik ≥ d ij s j ik , all downloads have a minimum rate condition s j k s j ik ≤ c ij , � ik ≥ L j Barlas O˘ guz (UCB) MURI 2009 September 2009 9 / 29

No Constraints - Full striping optimal - Can scale catalog size with M Barlas O˘ guz (UCB) MURI 2009 September 2009 10 / 29

Push to Peer VoD: Scalability? Barlas O˘ guz (UCB) MURI 2009 September 2009 11 / 29

Push to Peer VoD: Scalability? Barlas O˘ guz (UCB) MURI 2009 September 2009 11 / 29

Push to Peer VoD: Scalability? Barlas O˘ guz (UCB) MURI 2009 September 2009 11 / 29

Push to Peer VoD: Scalability? The preceding problem assumes a node can connect to all its peers. Not scalable as M becomes large. Put constraint on number of incoming connections. Connections constraint � | r j ik | 0 ≤ y k � = i , j Barlas O˘ guz (UCB) MURI 2009 September 2009 12 / 29

Scaling Catalog Size Mathieu, L. et. al., 2008 [ ? ] consider the problem of scalability of catalog size in a P2P VoD system. Given the number of nodes M , we want to achieve reliability with scaling demand. i.e. O ( M ) simultaneous downloads, or stability under rate O ( M ) per movie. How to maximally scale N with M ? For arbitrary demand profiles (or adversarial), catalog scales poorly. Barlas O˘ guz (UCB) MURI 2009 September 2009 13 / 29

What kind of distribution? Netflix Prize Database Pareto principle: top 15% claims 85% of demand. “The Long Tail”, niche titles also have some demand. Should use this information. Barlas O˘ guz (UCB) MURI 2009 September 2009 14 / 29

Scaling Catalog with Power Law Demands Can show better results if we assume power law in demand. Let N j ∼ Poisson( M ν j ) Mean parameter ν j come from a power law, e.g. ν j ∼ 1 j α Come up with a good push strategy. Look at probability of sustaining random demand { N j } . Probability of sustaining demand � P C = 1 ( C can sustain n ) . p N ( N = n ) n Barlas O˘ guz (UCB) MURI 2009 September 2009 15 / 29

Results Let popularity for movie j decay as j α , where total demand increases as ρ M . Asymptotic probability of sustaining demand (a) If α < 1, then M →∞ P (all requests are satisfied) = 0 lim (b) If α > 1 and ( yBup − 1 − 4 α )( α − 1) > 1, then ρ M →∞ P (all requests are satisfied) = 1 lim Barlas O˘ guz (UCB) MURI 2009 September 2009 16 / 29

Stochastic Approximation with Long Range Dependent and Heavy Tailed Noise Venkat Anantharam 1 and Vivek S. Borkar 2 1. Department of Electrical Engineering and Computer Sciences, University of California, Berkeley 2. School of Technology and Computer Science, Tata Institute of Fundamental Research, Homi Bhabha Road, Mumbai, India

Stochastic Approximation Given a function h , we seek to find a solution to h ( x ∗ ) = 0. However, we only observe h ( x n ) in noise. Use the following recursion. Algorithm x [ n + 1] = a n [ h ( x n ) + M n +1 ] where originally, M is mean zero, uncorrelated, bounded variance noise. Barlas O˘ guz (UCB) MURI 2009 September 2009 18 / 29

Stochastic Approximation Under suitable stability conditions (e.g. sup | x n | < K ), the recursion can be approximated by ODE x ( t ) = h ( x ( t )) ˙ Which can be shown to converge if � a n = ∞ � a 2 n < ∞ Barlas O˘ guz (UCB) MURI 2009 September 2009 19 / 29

Applications Many DSP applications, including adaptive filtering Network control Adaptive routing Service time control in queuing networks In network applications, we wish to run control algorithms based on the values of the flows. However, these might not be directly observed, might be available as noisy estimates. It has been observed empirically that queues and flows in large computer networks exhibit heavy tailed distributions or long range dependence. Barlas O˘ guz (UCB) MURI 2009 September 2009 20 / 29

Alpha stable Levy motion Take X i i.i.d. symmetric, P ( | X 1 | > x ) = x − α L ( x ) then S nt → d S α S 1 ( nL ( n )) α (symmetric α -stable Levy motion) 0 Barlas O˘ guz (UCB) MURI 2009 September 2009 21 / 29

Alpha-stable Levy Motion properties stationary, α -stable, i.i.d. increments. Distribution of S nt √ n → ∞ (long range dependence) Var( S t ) = ∞ Self-similarity: S nt = d n 1 α S t Samorodnitsky, Taqqu. “Stable Non-Gaussian Random Processes: Stochastic models with infinite variance” Barlas O˘ guz (UCB) MURI 2009 September 2009 22 / 29

Fractional Brownian Motion Fractional Brownian Motion is the unique Gaussian H-sssi process. | t 1 | 2 H + | t 2 | 2 H − | t 1 − t 2 | 2 H � Cov( B H ( t 1 ) , B H ( t 2 )) = 1 � Var( B H (1)) 2 H-sssi fBM limit Let Cov ( X 1 , X n ) = n − α L ( n ) regularly varying. And { X i } zero-mean Gaussian. n H → d B H ( t ), where H = (1 − α Then, S nt 2 ). Barlas O˘ guz (UCB) MURI 2009 September 2009 23 / 29

Fractional Brownian Motion fractional Brownian motion − parameter: 0.9 0 Barlas O˘ guz (UCB) MURI 2009 September 2009 24 / 29

Stochastic Approximation with LRD and Heavy Tailed Noise x n +1 = x n + a ( n )[ h ( x n ) + M n +1 + R ( n ) B n +1 + D ( n ) S n +1 + ξ n +1 ] , where h : R d → R d is Lipschitz, B n +1 := ˜ B ( n + 1) − ˜ B ( n ) , where ˜ B ( t ) , t ≥ 0 , is a fractional Brownian motion with Hurst parameter ν ∈ (0 , 1), S n +1 := ˜ S ( n + 1) − ˜ S ( n ), where ˜ S ( t ) , t ≥ 0 , is a symmetric α -stable process with 1 < α < 2, ξ n is an ’error’ process satisfying sup n || ξ n || ≤ K 0 < ∞ a.s. and ξ n → 0 a.s., Barlas O˘ guz (UCB) MURI 2009 September 2009 25 / 29