Data Streams Tutorial Andrew McGregor University of Massachusetts, - - PowerPoint PPT Presentation

Data Streams Tutorial

Andrew McGregor

University of Massachusetts, Amherst

Data Stream Model

[Morris ’78] [Munro, Paterson ’78] [Flajolet, Martin ’85] [Alon, Matias, Szegedy ’96] [Henzinger, Raghavan, Rajagopalan ’98]

Data Stream Model

[Morris ’78] [Munro, Paterson ’78] [Flajolet, Martin ’85] [Alon, Matias, Szegedy ’96] [Henzinger, Raghavan, Rajagopalan ’98]

• Stream: m elements from some universe of size n

e.g., 3,5,3,7,5,4,8,5,3,7,5,4,8,6,3,2,6,4,7, ...

Data Stream Model

[Morris ’78] [Munro, Paterson ’78] [Flajolet, Martin ’85] [Alon, Matias, Szegedy ’96] [Henzinger, Raghavan, Rajagopalan ’98]

• Stream: m elements from some universe of size n

e.g., 3,5,3,7,5,4,8,5,3,7,5,4,8,6,3,2,6,4,7, ...

• Goal: Estimate properties of the stream, e.g., median,

number of distinct elements, longest increasing sequence.

Data Stream Model

[Morris ’78] [Munro, Paterson ’78] [Flajolet, Martin ’85] [Alon, Matias, Szegedy ’96] [Henzinger, Raghavan, Rajagopalan ’98]

• Stream: m elements from some universe of size n

e.g., 3,5,3,7,5,4,8,5,3,7,5,4,8,6,3,2,6,4,7, ...

• Goal: Estimate properties of the stream, e.g., median,

number of distinct elements, longest increasing sequence.

• The Catch:
• i) Limited working memory, e.g., polylog(n,m)
• ii) Access data sequentially
• iii) Process each element quickly

Data Stream Model

[Morris ’78] [Munro, Paterson ’78] [Flajolet, Martin ’85] [Alon, Matias, Szegedy ’96] [Henzinger, Raghavan, Rajagopalan ’98]

• Stream: m elements from some universe of size n

e.g., 3,5,3,7,5,4,8,5,3,7,5,4,8,6,3,2,6,4,7, ...

• Goal: Estimate properties of the stream, e.g., median,

number of distinct elements, longest increasing sequence.

• The Catch:
• i) Limited working memory, e.g., polylog(n,m)
• ii) Access data sequentially
• iii) Process each element quickly
• Origins in ’70s but has become popular in last ten years...

Why’s it become popular?

Why’s it become popular?

• Practical Appeal:

Faster networks, cheaper data storage, ubiquitous data- logging results in massive amount of data to be processed. Applications to: Network monitoring, query planning, I/O efficiency for massive data, sensor networks aggregation...

Why’s it become popular?

• Practical Appeal:

Faster networks, cheaper data storage, ubiquitous data- logging results in massive amount of data to be processed. Applications to: Network monitoring, query planning, I/O efficiency for massive data, sensor networks aggregation...

• Theoretical Appeal:

Easy to state problems but hard to solve. Links to: Communication complexity, compressed sensing, embeddings, pseudo-random generators, approximation...

• I. What’s Done?
• II. Some Tools
• III. What’s Next?
• I. What’s Done?
• II. Some Tools
• III. What’s Next?

• I. What’s Done?
• I. What’s Done?

• I. What’s Done?
• I. What’s Done?

Basic Problems Problem Variants Model Variants

Families of Problems

Families of Problems

• Numbers:
• Distinct elements, heavy hitters, range queries, quantiles,

frequency moments, matrix problems, ...

Families of Problems

• Numbers:
• Distinct elements, heavy hitters, range queries, quantiles,

frequency moments, matrix problems, ...

Families of Problems

• Numbers:
• Distinct elements, heavy hitters, range queries, quantiles,

frequency moments, matrix problems, ...

• Graphs:
• Connectivity, MST, bipartiteness, matchings, distances,

graph cuts, independent sets, number of triangles, ...

Families of Problems

• Numbers:
• Distinct elements, heavy hitters, range queries, quantiles,

frequency moments, matrix problems, ...

• Graphs:
• Connectivity, MST, bipartiteness, matchings, distances,

graph cuts, independent sets, number of triangles, ...

Families of Problems

• Numbers:
• Distinct elements, heavy hitters, range queries, quantiles,

frequency moments, matrix problems, ...

• Graphs:
• Connectivity, MST, bipartiteness, matchings, distances,

graph cuts, independent sets, number of triangles, ...

• Points:
• Clustering, diameter, convex hulls, minimum enclosing

balls, MST, facility location, earth mover distance, ...

Families of Problems

• Numbers:
• Distinct elements, heavy hitters, range queries, quantiles,

frequency moments, matrix problems, ...

• Graphs:
• Connectivity, MST, bipartiteness, matchings, distances,

graph cuts, independent sets, number of triangles, ...

• Points:
• Clustering, diameter, convex hulls, minimum enclosing

balls, MST, facility location, earth mover distance, ...

Families of Problems

• Numbers:
• Distinct elements, heavy hitters, range queries, quantiles,

frequency moments, matrix problems, ...

• Graphs:
• Connectivity, MST, bipartiteness, matchings, distances,

graph cuts, independent sets, number of triangles, ...

• Points:
• Clustering, diameter, convex hulls, minimum enclosing

balls, MST, facility location, earth mover distance, ...

• Sequences:
• Longest increasing subsequence, pattern matching,

periodicity, time-series histograms, DYCK languages, ...

Problem Variants

Problem Variants

• Sliding

Window:

• Suppose you only want to solve the problem using the last

w elements and have O(polylog w) space.

• Elements could have time stamps and you want to solve the

problem for elements with stamps in the last hour.

Problem Variants

• Sliding

Window:

• Suppose you only want to solve the problem using the last

w elements and have O(polylog w) space.

• Elements could have time stamps and you want to solve the

problem for elements with stamps in the last hour.

• Uncertain Data:
• ith element of the stream is a distribution μi that defines

random variable Xi. Consider random variable g(X1, ... , Xm).

• Problems: What’s the expected value of the max value?

What’s the probability the graph is connected?

Model Variants

Model Variants

• Multiple Passes: Suppose you can take p>1 passes over

the stream. Can we trade-off passes with space?

Model Variants

• Multiple Passes: Suppose you can take p>1 passes over

the stream. Can we trade-off passes with space?

• Example: Most common item in space.

˜ Θ(n/p)

Model Variants

• Multiple Passes: Suppose you can take p>1 passes over

the stream. Can we trade-off passes with space?

• Example: Most common item in space.
• Example: Can find elements of length k increasing

subsequence in space. ˜ Θ(k1+

1 2p−1 )

˜ Θ(n/p)

Model Variants

• Multiple Passes: Suppose you can take p>1 passes over

the stream. Can we trade-off passes with space?

• Example: Most common item in space.
• Example: Can find elements of length k increasing

subsequence in space.

• Random Order: We normally assume that stream is
• rdered adversarially. What if it’s ordered randomly?

˜ Θ(k1+

1 2p−1 )

˜ Θ(n/p)

Model Variants

• Multiple Passes: Suppose you can take p>1 passes over

the stream. Can we trade-off passes with space?

• Example: Most common item in space.
• Example: Can find elements of length k increasing

subsequence in space.

• Random Order: We normally assume that stream is
• rdered adversarially. What if it’s ordered randomly?
• Example: Can find median of a random-order stream

in O(n1/2) space. If adversarial, it takes Ω(n) space. ˜ Θ(k1+

1 2p−1 )

˜ Θ(n/p)

Model Variants

• Multiple Passes: Suppose you can take p>1 passes over

the stream. Can we trade-off passes with space?

• Example: Most common item in space.
• Example: Can find elements of length k increasing

subsequence in space.

• Random Order: We normally assume that stream is
• rdered adversarially. What if it’s ordered randomly?
• Example: Can find median of a random-order stream

in O(n1/2) space. If adversarial, it takes Ω(n) space.

• Example: Estimating Fk takes roughly the same space

in random and adversarial settings. ˜ Θ(k1+

1 2p−1 )

˜ Θ(n/p)

• I. What’s Done?
• II. Some Tools
• III. What’s Next?
• I. What’s Done?
• II. Some Tools
• III. What’s Next?

• II. Some Tools
• II. Some Tools

• II. Some Tools
• II. Some Tools

Sketching Sampling Lower Bounds

First Idea: Sketches

First Idea: Sketches

         f1 f2 . . . fn         

First Idea: Sketches

• Algorithm uses a (random) projection matrix Z such that the

relevant properties of f can be estimated from the sketch Zf.          f1 f2 . . . fn         

    Z    

First Idea: Sketches

• Algorithm uses a (random) projection matrix Z such that the

relevant properties of f can be estimated from the sketch Zf.          f1 f2 . . . fn          =     t1 t2 tk    

    Z    

First Idea: Sketches

• Algorithm uses a (random) projection matrix Z such that the

relevant properties of f can be estimated from the sketch Zf.

• Easy to Update: On seeing “i”, add ith column of Z to sketch

         f1 f2 . . . fn          =     t1 t2 tk    

    Z    

First Idea: Sketches

• Algorithm uses a (random) projection matrix Z such that the

relevant properties of f can be estimated from the sketch Zf.

• Easy to Update: On seeing “i”, add ith column of Z to sketch
• Store Matrix Implicitly: Need to be able to efficiently generate

any entry of Z from a “small” random seed.          f1 f2 . . . fn          =     t1 t2 tk    

    Z    

First Idea: Sketches

• Algorithm uses a (random) projection matrix Z such that the

relevant properties of f can be estimated from the sketch Zf.

• Easy to Update: On seeing “i”, add ith column of Z to sketch
• Store Matrix Implicitly: Need to be able to efficiently generate

any entry of Z from a “small” random seed.

• Gives Õ(k) space algorithm with seed & precision assumptions.

         f1 f2 . . . fn          =     t1 t2 tk    

    Z              f1 f2 . . . fn          =     t1 t2 tk    

Algorithm for Estimating F2

    Z              f1 f2 . . . fn          =     t1 t2 tk    

Algorithm for Estimating F2

Consider a row z of the projection matrix.

    Z              f1 f2 . . . fn          =     t1 t2 tk    

Algorithm for Estimating F2

Consider a row z of the projection matrix. Let entries of z be uniform in {-1,1} chosen with 4-wise independence. Let t=z.f.

    Z              f1 f2 . . . fn          =     t1 t2 tk    

Algorithm for Estimating F2

Consider a row z of the projection matrix. Let entries of z be uniform in {-1,1} chosen with 4-wise independence. Let t=z.f.

    Z              f1 f2 . . . fn          =     t1 t2 tk    

Algorithm for Estimating F2

Consider a row z of the projection matrix. Let entries of z be uniform in {-1,1} chosen with 4-wise independence. Let t=z.f.

Square of entry is concentrated around F2.

    Z              f1 f2 . . . fn          =     t1 t2 tk    

Algorithm for Estimating F2

Consider a row z of the projection matrix. Let entries of z be uniform in {-1,1} chosen with 4-wise independence. Let t=z.f. Expectation: E(t2) = ∑i,j E(zizj)fifj= F2

Square of entry is concentrated around F2.

    Z              f1 f2 . . . fn          =     t1 t2 tk    

Algorithm for Estimating F2

Consider a row z of the projection matrix. Let entries of z be uniform in {-1,1} chosen with 4-wise independence. Let t=z.f. Expectation: E(t2) = ∑i,j E(zizj)fifj= F2 Variance: Var(t2) ≤ ∑i,j,k,l E(zizjzkzl)fifjfkfl < 6F22

Square of entry is concentrated around F2.

    Z              f1 f2 . . . fn          =     t1 t2 tk    

Algorithm for Estimating F2

Consider a row z of the projection matrix. Let entries of z be uniform in {-1,1} chosen with 4-wise independence. Let t=z.f. Expectation: E(t2) = ∑i,j E(zizj)fifj= F2 Variance: Var(t2) ≤ ∑i,j,k,l E(zizjzkzl)fifjfkfl < 6F22 By Chebyshev, setting k=O(ε-2 log δ-1) ensures with

• prob. 1-δ, average of squared entries is (1±ε) F2.

Square of entry is concentrated around F2.

Second Idea: Sampling

Second Idea: Sampling

• Let’s sample from S=[a1, a2, a3, ... , am] where each ai ∈R [n]

Second Idea: Sampling

• Let’s sample from S=[a1, a2, a3, ... , am] where each ai ∈R [n]
• Distribution Sampling: Return i with probability fi/m

Second Idea: Sampling

• Let’s sample from S=[a1, a2, a3, ... , am] where each ai ∈R [n]
• Distribution Sampling: Return i with probability fi/m
• Universe Sampling: Return (i,fi) where i ∈R [n]

Second Idea: Sampling

• Let’s sample from S=[a1, a2, a3, ... , am] where each ai ∈R [n]
• Distribution Sampling: Return i with probability fi/m
• Universe Sampling: Return (i,fi) where i ∈R [n]
• AMS Sampling: Return (i,r) with i chosen w/p fi/m and r ∈R [fi]

Second Idea: Sampling

• Let’s sample from S=[a1, a2, a3, ... , am] where each ai ∈R [n]
• Distribution Sampling: Return i with probability fi/m
• Universe Sampling: Return (i,fi) where i ∈R [n]
• AMS Sampling: Return (i,r) with i chosen w/p fi/m and r ∈R [fi]
• Sample aj for j ∈R [m], let i= aj and compute r=|{j′≥j : aj′=aj}|

Second Idea: Sampling

• Let’s sample from S=[a1, a2, a3, ... , am] where each ai ∈R [n]
• Distribution Sampling: Return i with probability fi/m
• Universe Sampling: Return (i,fi) where i ∈R [n]
• AMS Sampling: Return (i,r) with i chosen w/p fi/m and r ∈R [fi]
• Sample aj for j ∈R [m], let i= aj and compute r=|{j′≥j : aj′=aj}|
• Useful for estimating ∑i g(fi) because E[m(g(r)-g(r-1))] = ∑i g(fi)

Second Idea: Sampling

• Let’s sample from S=[a1, a2, a3, ... , am] where each ai ∈R [n]
• Distribution Sampling: Return i with probability fi/m
• Universe Sampling: Return (i,fi) where i ∈R [n]
• AMS Sampling: Return (i,r) with i chosen w/p fi/m and r ∈R [fi]
• Sample aj for j ∈R [m], let i= aj and compute r=|{j′≥j : aj′=aj}|
• Useful for estimating ∑i g(fi) because E[m(g(r)-g(r-1))] = ∑i g(fi)
• Lp Sampling: Return i with probability fik/Fk

L0 Sampling

L0 Sampling

Suppose we know F0. Pick hash function h:[n]→[F0]

L0 Sampling

Suppose we know F0. Pick hash function h:[n]→[F0] Algorithm: Maintain values c and id, initially 0.

L0 Sampling

Suppose we know F0. Pick hash function h:[n]→[F0] Algorithm: Maintain values c and id, initially 0. For each j in stream: if h(j)=1, c←c+1, id←id+j

L0 Sampling

Suppose we know F0. Pick hash function h:[n]→[F0] Algorithm: Maintain values c and id, initially 0. For each j in stream: if h(j)=1, c←c+1, id←id+j Return id/c if all elts hashing to 1 were same

L0 Sampling

Suppose we know F0. Pick hash function h:[n]→[F0] Algorithm: Maintain values c and id, initially 0. For each j in stream: if h(j)=1, c←c+1, id←id+j Return id/c if all elts hashing to 1 were same Claim: This happens with constant probability.

L0 Sampling

Suppose we know F0. Pick hash function h:[n]→[F0] Algorithm: Maintain values c and id, initially 0. For each j in stream: if h(j)=1, c←c+1, id←id+j Return id/c if all elts hashing to 1 were same Claim: This happens with constant probability. Claim: Need to check elts hashing to 1 were same.

L0 Sampling

Suppose we know F0. Pick hash function h:[n]→[F0] Algorithm: Maintain values c and id, initially 0. For each j in stream: if h(j)=1, c←c+1, id←id+j Return id/c if all elts hashing to 1 were same Claim: This happens with constant probability. Claim: Need to check elts hashing to 1 were same. Run O(log n) copies guessing F0=2i. At least one instantiation works with constant probability.

L0 Sampling

Suppose we know F0. Pick hash function h:[n]→[F0] Algorithm: Maintain values c and id, initially 0. For each j in stream: if h(j)=1, c←c+1, id←id+j Return id/c if all elts hashing to 1 were same Claim: This happens with constant probability. Claim: Need to check elts hashing to 1 were same. Run O(log n) copies guessing F0=2i. At least one instantiation works with constant probability. Algorithm is a sketch and works with deletions!

Third Idea: Lower Bounds

Third Idea: Lower Bounds

x∈{0,1}n y∈{0,1}n

• Many space lower bounds in data stream model use

reductions from communication complexity.

Third Idea: Lower Bounds

x∈{0,1}n y∈{0,1}n

• Many space lower bounds in data stream model use

reductions from communication complexity.

Third Idea: Lower Bounds

x∈{0,1}n y∈{0,1}n

• Many space lower bounds in data stream model use

reductions from communication complexity.

• Example: Alice and Bob have x,y∈{0,1}n and Bob wants to

check DISJOINTNESS i.e., is there an i with xi=yi=1?

Third Idea: Lower Bounds

x∈{0,1}n y∈{0,1}n

• Many space lower bounds in data stream model use

reductions from communication complexity.

• Example: Alice and Bob have x,y∈{0,1}n and Bob wants to

check DISJOINTNESS i.e., is there an i with xi=yi=1?

• Thm: Any 1/3-error protocol for DISJOINTNESS requires

Ω(n) bits of communication.

Third Idea: Lower Bounds

x∈{0,1}n y∈{0,1}n

• Many space lower bounds in data stream model use

reductions from communication complexity.

• Example: Alice and Bob have x,y∈{0,1}n and Bob wants to

check DISJOINTNESS i.e., is there an i with xi=yi=1?

• Thm: Any 1/3-error protocol for DISJOINTNESS requires

Ω(n) bits of communication.

• Corollary: Any 1/3-error stream algorithm that checks if a

graph is triangle-free needs Ω(n2) bits of memory.

Lower Bound for Triangle Detection

Lower Bound for Triangle Detection

Alice and Bob have X,Y∈{0,1}nxn. For Bob to check if Xij=Yij=1 for some i,j needs Ω(n2) communication.

Lower Bound for Triangle Detection

Alice and Bob have X,Y∈{0,1}nxn. For Bob to check if Xij=Yij=1 for some i,j needs Ω(n2) communication. Let A be an s-space alg that checks for triangles.

Lower Bound for Triangle Detection

Alice and Bob have X,Y∈{0,1}nxn. For Bob to check if Xij=Yij=1 for some i,j needs Ω(n2) communication. Let A be an s-space alg that checks for triangles. Consider 3-layer graph (U,V ,W) with |U|=|V|=|W|=n

Lower Bound for Triangle Detection

Alice and Bob have X,Y∈{0,1}nxn. For Bob to check if Xij=Yij=1 for some i,j needs Ω(n2) communication. Let A be an s-space alg that checks for triangles. Consider 3-layer graph (U,V ,W) with |U|=|V|=|W|=n

Lower Bound for Triangle Detection

Alice and Bob have X,Y∈{0,1}nxn. For Bob to check if Xij=Yij=1 for some i,j needs Ω(n2) communication. Let A be an s-space alg that checks for triangles. Consider 3-layer graph (U,V ,W) with |U|=|V|=|W|=n Alice runs A on E1={uiwi: 1≤i≤n} and E2={uivj: Xij=1}

Lower Bound for Triangle Detection

Alice and Bob have X,Y∈{0,1}nxn. For Bob to check if Xij=Yij=1 for some i,j needs Ω(n2) communication. Let A be an s-space alg that checks for triangles. Consider 3-layer graph (U,V ,W) with |U|=|V|=|W|=n Alice runs A on E1={uiwi: 1≤i≤n} and E2={uivj: Xij=1}

Lower Bound for Triangle Detection

Alice and Bob have X,Y∈{0,1}nxn. For Bob to check if Xij=Yij=1 for some i,j needs Ω(n2) communication. Let A be an s-space alg that checks for triangles. Consider 3-layer graph (U,V ,W) with |U|=|V|=|W|=n Alice runs A on E1={uiwi: 1≤i≤n} and E2={uivj: Xij=1}

Lower Bound for Triangle Detection

Alice and Bob have X,Y∈{0,1}nxn. For Bob to check if Xij=Yij=1 for some i,j needs Ω(n2) communication. Let A be an s-space alg that checks for triangles. Consider 3-layer graph (U,V ,W) with |U|=|V|=|W|=n Alice runs A on E1={uiwi: 1≤i≤n} and E2={uivj: Xij=1} Sends memory to Bob who runs A on E3={vjwi:Yij=1}

Lower Bound for Triangle Detection

Alice and Bob have X,Y∈{0,1}nxn. For Bob to check if Xij=Yij=1 for some i,j needs Ω(n2) communication. Let A be an s-space alg that checks for triangles. Consider 3-layer graph (U,V ,W) with |U|=|V|=|W|=n Alice runs A on E1={uiwi: 1≤i≤n} and E2={uivj: Xij=1} Sends memory to Bob who runs A on E3={vjwi:Yij=1}

Lower Bound for Triangle Detection

Alice and Bob have X,Y∈{0,1}nxn. For Bob to check if Xij=Yij=1 for some i,j needs Ω(n2) communication. Let A be an s-space alg that checks for triangles. Consider 3-layer graph (U,V ,W) with |U|=|V|=|W|=n Alice runs A on E1={uiwi: 1≤i≤n} and E2={uivj: Xij=1} Sends memory to Bob who runs A on E3={vjwi:Yij=1} Output of A resolves matrix question so s=Ω(n2).

Useful Communication Results

Useful Communication Results

• Indexing:
• Alice has x∈{0,1}n, Bob has i∈[n]. Bob want’s to learn xi.
• One-way communication requires Ω(n) bits even if Bob

also knows first i-1 bits of x.

Useful Communication Results

• Indexing:
• Alice has x∈{0,1}n, Bob has i∈[n]. Bob want’s to learn xi.
• One-way communication requires Ω(n) bits even if Bob

also knows first i-1 bits of x.

• Gap-Hamming:
• Alice and Bob have x,y∈{0,1}n. Distinguish Δ(x,y)<n/2-√n

from Δ(x,y)>n/2+√n.

• Requires Ω(n) communication.

Useful Communication Results

• Indexing:
• Alice has x∈{0,1}n, Bob has i∈[n]. Bob want’s to learn xi.
• One-way communication requires Ω(n) bits even if Bob

also knows first i-1 bits of x.

• Gap-Hamming:
• Alice and Bob have x,y∈{0,1}n. Distinguish Δ(x,y)<n/2-√n

from Δ(x,y)>n/2+√n.

• Requires Ω(n) communication.
• Multi-Party Disjointness:
• t players have x1,x2, ... , xt ∈{0,1}n. Need to distinguish

x1i=x2i= ... =xti =1 for some i from all vectors orthogonal.

• Requires Ω(n/t) communication.

Bonus! The Fourth Idea

Bonus! The Fourth Idea

• Algorithmic tools will only get you so far, sometimes you

need to come up with neat ad hoc solutions.

Bonus! The Fourth Idea

• Algorithmic tools will only get you so far, sometimes you

need to come up with neat ad hoc solutions.

• Graph Distances: Given a stream of edges, approximate the

shortest path distance between any two nodes.

Bonus! The Fourth Idea

• Algorithmic tools will only get you so far, sometimes you

need to come up with neat ad hoc solutions.

• Graph Distances: Given a stream of edges, approximate the

shortest path distance between any two nodes.

• k-Center: Given a stream of points, find a set of centers that

minimizes max distance from a point to nearest center.

Approximate Distances

Approximate Distances

Edges define shortest path graph metric dG.

Approximate Distances

Edges define shortest path graph metric dG. An α-spanner of G = (V ,E) is a subgraph H = (V ,E’) such that ∀u,v: dG(u,v) ≤ dH(u,v) ≤ αdG(u,v)

Approximate Distances

Edges define shortest path graph metric dG. An α-spanner of G = (V ,E) is a subgraph H = (V ,E’) such that ∀u,v: dG(u,v) ≤ dH(u,v) ≤ αdG(u,v) Algorithm: Let E′ be initially empty Add (u,v) to E′ if dH(u,v) > 2t-1

Approximate Distances

Edges define shortest path graph metric dG. An α-spanner of G = (V ,E) is a subgraph H = (V ,E’) such that ∀u,v: dG(u,v) ≤ dH(u,v) ≤ αdG(u,v) Algorithm: Let E′ be initially empty Add (u,v) to E′ if dH(u,v) > 2t-1 Analysis: Each distance increase by at most factor 2t-1 |E′| = O(n1+1/t) because all cycles of length > 2t

k-Center Clustering

k-Center Clustering

2 approx in O(k) space if you already know OPT.

k-Center Clustering

2 approx in O(k) space if you already know OPT. (2+ε) approx in O(k ε-1 log Δ) space if 1≤OPT≤Δ

k-Center Clustering

2 approx in O(k) space if you already know OPT. (2+ε) approx in O(k ε-1 log Δ) space if 1≤OPT≤Δ Better Algorithm O(k ε-1 log ε-1): Instantiate basic algorithm with guesses 1, (1+ε), (1+ε)2, ... , 2ε−1

k-Center Clustering

2 approx in O(k) space if you already know OPT. (2+ε) approx in O(k ε-1 log Δ) space if 1≤OPT≤Δ Better Algorithm O(k ε-1 log ε-1): Instantiate basic algorithm with guesses 1, (1+ε), (1+ε)2, ... , 2ε−1 If guess r stops working at (j+1)th point: Let q1,...,qk be centers chosen so far. Then p1,...,pj are all at most 2r from some qi. OPT for {q1,...,qk,pj+1,...,pn} is at most OPT+2r.

k-Center Clustering

2 approx in O(k) space if you already know OPT. (2+ε) approx in O(k ε-1 log Δ) space if 1≤OPT≤Δ Better Algorithm O(k ε-1 log ε-1): Instantiate basic algorithm with guesses 1, (1+ε), (1+ε)2, ... , 2ε−1 If guess r stops working at (j+1)th point: Let q1,...,qk be centers chosen so far. Then p1,...,pj are all at most 2r from some qi. OPT for {q1,...,qk,pj+1,...,pn} is at most OPT+2r. Hence, an instantiation with guess 2r/ε can use {q1,...,qk,pj+1,...,pn} rather than {p1,...,pn}.

• I. What’s Done?
• II. Some Tools
• III. What’s Next?
• I. What’s Done?
• II. Some Tools
• III. What’s Next?

• III. What’s Next?
• III. What’s Next?

• III. What’s Next?
• III. What’s Next?

Open Problems Annotations Space-Efficient Sampling

Open Problems

Open Problems

• Lists of Open Problems:
• Original Kanpur List (2007)
• www.cse.iitk.ac.in/users/sganguly/data-stream-probs.pdf
• Bertinoro & Second Kanpur List (2011)
• www.cs.umass.edu/~mcgregor/papers/11-openproblems.pdf

Open Problems

• Lists of Open Problems:
• Original Kanpur List (2007)
• www.cse.iitk.ac.in/users/sganguly/data-stream-probs.pdf
• Bertinoro & Second Kanpur List (2011)
• www.cs.umass.edu/~mcgregor/papers/11-openproblems.pdf

?

Longest Increasing Subsequence: Given a stream of n values, approximate the length of the LIS.

Open Problems

• Lists of Open Problems:
• Original Kanpur List (2007)
• www.cse.iitk.ac.in/users/sganguly/data-stream-probs.pdf
• Bertinoro & Second Kanpur List (2011)
• www.cs.umass.edu/~mcgregor/papers/11-openproblems.pdf

?

Longest Increasing Subsequence: Given a stream of n values, approximate the length of the LIS.

?

Earth Mover Distance: Given a stream of n red points and n blue points in [n]2, approximate min-cost matching.

Annotated Streams

STREAM

Annotated Streams

STREAM ADVICE S T R E A M

Annotated Streams

• Helper: Helper has unbounded memory but can’t tell the
• future. Sends h bits of of advice including the final answer.

STREAM ADVICE S T R E A M

Annotated Streams

• Helper: Helper has unbounded memory but can’t tell the
• future. Sends h bits of of advice including the final answer.
• Verifier: Has v bits of memory to process the stream and
• advice. Need to verify that provided answer is correct.

STREAM ADVICE S T R E A M

Annotated Streams

• Helper: Helper has unbounded memory but can’t tell the
• future. Sends h bits of of advice including the final answer.
• Verifier: Has v bits of memory to process the stream and
• advice. Need to verify that provided answer is correct.
• Example: There exists helper/verifier protocol for median

where h=v=O(√m).

STREAM ADVICE S T R E A M

Annotated Streams

• Helper: Helper has unbounded memory but can’t tell the
• future. Sends h bits of of advice including the final answer.
• Verifier: Has v bits of memory to process the stream and
• advice. Need to verify that provided answer is correct.
• Example: There exists helper/verifier protocol for median

where h=v=O(√m).

?

Open Problem: Testing if a graph is triangle-free.

STREAM ADVICE S T R E A M

Space-Efficient Sampling

STREAM

Space-Efficient Sampling

THE SOURCE STREAM

Space-Efficient Sampling

• Stochastically Generated Stream: E.g., stream is generated by

taking independent samples from unknown distribution μ.

THE SOURCE STREAM

Space-Efficient Sampling

• Stochastically Generated Stream: E.g., stream is generated by

taking independent samples from unknown distribution μ.

• Sample Complexity: How many samples to estimate f(μ)?

THE SOURCE STREAM

Space-Efficient Sampling

• Stochastically Generated Stream: E.g., stream is generated by

taking independent samples from unknown distribution μ.

• Sample Complexity: How many samples to estimate f(μ)?
• Space Complexity: How much memory to process samples?

THE SOURCE STREAM

Space-Efficient Sampling

• Stochastically Generated Stream: E.g., stream is generated by

taking independent samples from unknown distribution μ.

• Sample Complexity: How many samples to estimate f(μ)?
• Space Complexity: How much memory to process samples?
• By increasing sample complexity, we can perhaps decrease

space complexity. This is the case for frequency moments.

THE SOURCE STREAM

Resources

Resources

• Blog:
• http://polylogblog.wordpress.com

Resources

• Blog:
• http://polylogblog.wordpress.com
• Lectures: Piotr Indyk (MIT)
• http://stellar.mit.edu/S/course/6/fa07/6.895/

Resources

• Blog:
• http://polylogblog.wordpress.com
• Lectures: Piotr Indyk (MIT)
• http://stellar.mit.edu/S/course/6/fa07/6.895/
• Books:
• “Data Streams: Algorithms and Applications”
• S. Muthukrishnan (2005)
• “Algorithms and Complexity of Stream Processing”
• A. McGregor, S. Muthukrishnan (forthcoming)

Median with Annotations

Median with Annotations

Define “cumulative frequency” vector: gi=f1+f2+...+fi

Median with Annotations

Define “cumulative frequency” vector: gi=f1+f2+...+fi

1 1 3 5 6 7 8 8 8 8 10 10 11 12 12 15 18 18 19 20 22 22 23 25 25

Median with Annotations

Define “cumulative frequency” vector: gi=f1+f2+...+fi Easy to see i is median iff gi-1<m/2 and gi≥m/2

1 1 3 5 6 7 8 8 8 8 10 10 11 12 12 15 18 18 19 20 22 22 23 25 25

Median with Annotations

Define “cumulative frequency” vector: gi=f1+f2+...+fi Easy to see i is median iff gi-1<m/2 and gi≥m/2 Partition g into v=m1/2 segments of length h=m1/2

1 1 3 5 6 7 8 8 8 8 10 10 11 12 12 15 18 18 19 20 22 22 23 25 25

Median with Annotations

Define “cumulative frequency” vector: gi=f1+f2+...+fi Easy to see i is median iff gi-1<m/2 and gi≥m/2 Partition g into v=m1/2 segments of length h=m1/2 Verifier: a) Construct fingerprint of each segment

1 1 3 5 6 7 8 8 8 8 10 10 11 12 12 15 18 18 19 20 22 22 23 25 25

Median with Annotations

Define “cumulative frequency” vector: gi=f1+f2+...+fi Easy to see i is median iff gi-1<m/2 and gi≥m/2 Partition g into v=m1/2 segments of length h=m1/2 Verifier: a) Construct fingerprint of each segment

• b) Compute last entry in each segment

1 1 3 5 6 7 8 8 8 8 10 10 11 12 12 15 18 18 19 20 22 22 23 25 25

Median with Annotations

Define “cumulative frequency” vector: gi=f1+f2+...+fi Easy to see i is median iff gi-1<m/2 and gi≥m/2 Partition g into v=m1/2 segments of length h=m1/2 Verifier: a) Construct fingerprint of each segment

• b) Compute last entry in each segment
• c) Identify “interesting” segment

1 1 3 5 6 7 8 8 8 8 10 10 11 12 12 15 18 18 19 20 22 22 23 25 25

Median with Annotations

Define “cumulative frequency” vector: gi=f1+f2+...+fi Easy to see i is median iff gi-1<m/2 and gi≥m/2 Partition g into v=m1/2 segments of length h=m1/2 Verifier: a) Construct fingerprint of each segment

• b) Compute last entry in each segment
• c) Identify “interesting” segment

Helper: Presents entirety of interesting segment

1 1 3 5 6 7 8 8 8 8 10 10 11 12 12 15 18 18 19 20 22 22 23 25 25