Streaming Model of Computation A streaming algorithm processes a data - - PowerPoint PPT Presentation

Streaming Model of Computation

A streaming algorithm processes a data stream 𝑇:

• Input is presented as a sequence of items and can be examined in only

a few passes (typically just one).

• The algorithm has limited memory and cannot store the whole input

sequence.

• The algorithm can spend limited processing time per item.
• In some problems we are satisfied with an approximate answer.
• Approximation algorithms can be based on sketches (summaries) of the

data stream in memory.

Streaming Graph Algorithms

In many applications we deal with massive graphs. E.g. (vertices – edges):

• Web-pages – hyperlinks
• Neurons – synapses
• IP addresses – network flows
• People – friendships

Processing such graphs with a classic graph algorithm may be infeasible! But it may be possible to use an algorithm developed for the data stream model.

Streaming Graph Algorithms

In many applications we deal with massive graphs. E.g. (vertices – edges):

• Web-pages – hyperlinks
• Neurons – synapses
• IP addresses – network flows
• People – friendships

Processing such graphs with a classic graph algorithm may be infeasible! But it may be possible to use an algorithm developed for the data stream model. Presentation based on:

• A. McGregor “Graph Stream Algorithms: A Survey” [ACM SIGMOD Record 2014]

Streaming Graph Algorithms

Streaming Graph Algorithms

Data stream model

• The input is given by a stream of data. E.g., the stream could be the graph

edges.

• The algorithm can use a limited amount of memory to process the stream.
• The input stream must be processed in the order it arrives.

Related goals:

• Real-time systems.
• I/O efficiency.
• Trade-off size and accuracy.

Streaming Graph Algorithms

Data stream model How much memory should our model allow in order to be able to process a graph with 𝑜 vertices?

• Most problems are intractable if space is < 𝑜.
• We will work in the semi-streaming model that allows O(𝑜 log𝑙𝑜) memory, for

some constant 𝑙.

• Some algorithms will be randomized. We will say that an event 𝐹 occurs with

high probability if Pr 𝐹 ≥ 1 − 1/𝑜.

Streaming Graph Algorithms

Graph connectivity Data stream 𝑇: Edges of a graph 𝐻 = (𝑊, 𝐹) with 𝑜 = 𝑊 1 2 3 4 𝐻 E.g., data stream 𝑇 = (1,2), (2,3), (1,3), (3,4)

Streaming Graph Algorithms

Graph connectivity Data stream 𝑇: Edges of a graph 𝐻 = (𝑊, 𝐹) with 𝑜 = 𝑊 The goal is to test if 𝐻 is connected, i.e., for any two vertices there is a path that connects them. 𝑦 𝑧 𝐻

Streaming Graph Algorithms

Graph connectivity Data stream 𝑇: Edges of a graph 𝐻 = (𝑊, 𝐹) with 𝑜 = 𝑊 The goal is to test if 𝐻 is connected, i.e., for any two vertices there is a path that connects them. 𝑦 𝑧 𝐻 Simple algorithm: Maintain a set of edges 𝐼. When we read the next edge (𝑣, 𝑤) from the stream, we add it to 𝐼 if there is currently no path between 𝑣 and 𝑤.

Streaming Graph Algorithms

Spanners 𝑏-spanner 𝐼 of a graph 𝐻 = (𝑊, 𝐹): subgraph of 𝐻 such that for all pairs 𝑣, 𝑤 ∈ 𝑊,

𝑒𝐻(𝑣, 𝑤) ≤ 𝑒𝐼(𝑣, 𝑤) ≤ 𝑏 ∙ 𝑒𝐻(𝑣, 𝑤)

𝑒𝐻 𝑣, 𝑤 = length of the shortest path between 𝑣 and 𝑤 in 𝐻 𝑒𝐼 𝑣, 𝑤 = length of the shortest path between 𝑣 and 𝑤 in 𝐼 1 2 3 4 𝐻

Streaming Graph Algorithms

Spanners 𝑏-spanner 𝐼 of a graph 𝐻 = (𝑊, 𝐹): subgraph of 𝐻 such that for all pairs 𝑣, 𝑤 ∈ 𝑊,

𝑒𝐻(𝑣, 𝑤) ≤ 𝑒𝐼(𝑣, 𝑤) ≤ 𝑏 ∙ 𝑒𝐻(𝑣, 𝑤)

𝑒𝐻 𝑣, 𝑤 = length of the shortest path between 𝑣 and 𝑤 in 𝐻 𝑒𝐼 𝑣, 𝑤 = length of the shortest path between 𝑣 and 𝑤 in 𝐼 1 2 3 4 1 2 3 4 𝐻 𝐼 2-spanner

Streaming Graph Algorithms

Spanners Construction of an 𝑏-spanner 𝐼: add next edge (𝑣, 𝑤) if it does not create a short cycle in 𝐼 Greedy Spanner Algorithm 1. 𝐼 ← ∅ 2. for

• r each edge (𝑣, 𝑤) ∈ 𝑇 do

do 3. if if 𝑒𝐼 𝑣, 𝑤 > 𝑏 the then add (𝑣, 𝑤) to 𝐼 4. ret eturn 𝐼

• Does this work?
• What is the size (#edges) of the spanner?

Streaming Graph Algorithms

Spanners Proof that the Greedy Spanner Algorithm works: For any edge (𝑦, 𝑧) of 𝐻 we have 𝑒𝐼 𝑦, 𝑧 ≤ 𝑏. Length of 𝑄 in 𝐻 = 𝑙 = 𝑒𝐻 𝑤0, 𝑤1 + 𝑒𝐻 𝑤1, 𝑤2 + ⋯ + 𝑒𝐻 𝑤𝑙−1, 𝑤𝑙 𝑤0 𝑤1

𝑤𝑙−1

𝑤𝑙 𝑄 𝑒𝐼 𝑤0, 𝑤1 ≤ 𝑏 𝑒𝐼 𝑤𝑙−1, 𝑤𝑙 ≤ 𝑏 Consider a path 𝑄 = 𝑤0, 𝑤1, . . , 𝑤𝑙−1, 𝑤𝑙 in 𝐻

Streaming Graph Algorithms

Spanners Proof that the Greedy Spanner Algorithm works: For any edge (𝑦, 𝑧) of 𝐻 we have 𝑒𝐼 𝑦, 𝑧 ≤ 𝑏. Length of 𝑄 in 𝐻 = 𝑙 = 𝑒𝐻 𝑤0, 𝑤1 + 𝑒𝐻 𝑤1, 𝑤2 + ⋯ + 𝑒𝐻 𝑤𝑙−1, 𝑤𝑙 𝑤0 𝑤1

𝑤𝑙−1

𝑤𝑙 𝑄 𝑒𝐼 𝑤0, 𝑤1 ≤ 𝑏 𝑒𝐼 𝑤𝑙−1, 𝑤𝑙 ≤ 𝑏 Length in 𝐼 ≤ 𝑒𝐼 𝑤0, 𝑤1 + 𝑒𝐼 𝑤1, 𝑤2 + ⋯ + 𝑒𝐼 𝑤𝑙−1, 𝑤𝑙 Consider a path 𝑄 = 𝑤0, 𝑤1, . . , 𝑤𝑙−1, 𝑤𝑙 in 𝐻 ≤ 𝑏 ∙ 𝑒𝐻 𝑤0, 𝑤1 + 𝑏 ∙ 𝑒𝐻 𝑤1, 𝑤2 + ⋯ + 𝑏 ∙ 𝑒𝐻 𝑤𝑙−1, 𝑤𝑙 = 𝑏 ∙ 𝑙

Streaming Graph Algorithms

Spanners How many edges are inserted into 𝐼?

• By a known result in Graph Theory, any such graph has at most

O 𝑜1+ Τ

1 𝑢

• Let 𝑏 = 2𝑢 − 1, for some integer 𝑢.
• Then 𝐼 does not contain cycles of length < 2𝑢.

edges.

Streaming Graph Algorithms

Minimum Spanning Tree Data stream 𝑇: Edges of a weighted graph 𝐻 = (𝑊, 𝐹, 𝑥) with 𝑜 = 𝑊 Greedy MST Algorithm 1. 𝐼 ← ∅ 2. for

• r each edge 𝑓 = (𝑣, 𝑤) ∈ 𝑇 do

do 3. if if 𝑓 creates a cycle 𝐷 in 𝐼 th then 4. find the maximum weight edge 𝑔 ∈ 𝐷 5. add 𝑓 to 𝐼 6. delete 𝑔 from 𝐼 7. ret eturn 𝐼 Construction: if next edge (𝑣, 𝑤) creates a cycle 𝐷 in 𝐼, delete from 𝐼 the maximum weight edge of 𝐷.

Streaming Graph Algorithms

Graph Sparsification Given a graph 𝐻 = (𝑊, 𝐹) we want to construct a weighted subgraph 𝐼 = (𝑊, 𝐹𝐼, 𝑥) of 𝐻 that estimates various (connectivity) properties of 𝐻 E.g.:

• Cut sparsification [Benczur-Karger]
• Spectral sparsification [Spielman-Teng]

Streaming Graph Algorithms

Picture from https://simons.berkeley.edu/sites/default/files/docs/1768/slidessrivastava1.pdf

Streaming Graph Algorithms

Cuts in Graphs Weighted graph 𝐻 = (𝑊, 𝐹, 𝑥). Edge weights 𝑥 ∶ 𝐹 → 𝑆 𝐵-cut: partition of 𝑊 into two sets 𝐵 and 𝑊\𝐵 𝜀𝐻 𝛣 = set of edges in 𝐻 crossing the 𝐵-cut. 𝜀𝐻 𝛣 =

𝑣, 𝑤 ∈ 𝐹 ∶ 𝑣 ∈ 𝐵, 𝑤 ∈ 𝑊\A

Size of 𝐵-cut in 𝐻: 𝜇𝐵 𝐻 = σ𝑓∈𝜀𝐻(𝛣) 𝑥(𝑓)

Streaming Graph Algorithms

Cuts in Graphs 26 17 12 14 7 9 20 4 10 𝑏 𝑐 𝑑 𝑒 𝑔 𝑓 Weighted graph 𝐻 = (𝑊, 𝐹, 𝑥). Edge weights 𝑥 ∶ 𝐹 → 𝑆 𝐵-cut: partition of 𝑊 into two sets 𝐵 and 𝑊\𝐵 𝜀𝐻 𝛣 = set of edges in 𝐻 crossing the 𝐵-cut. 𝜀𝐻 𝛣 =

𝑣, 𝑤 ∈ 𝐹 ∶ 𝑣 ∈ 𝐵, 𝑤 ∈ 𝑊\A

Size of 𝐵-cut in 𝐻: 𝜇𝐵 𝐻 = σ𝑓∈𝜀𝐻(𝛣) 𝑥(𝑓)

Streaming Graph Algorithms

Cuts in Graphs 26 17 12 14 7 9 20 4 10 𝑏 𝑐 𝑑 𝑒 𝑔 𝑓 Weighted graph 𝐻 = (𝑊, 𝐹, 𝑥). Edge weights 𝑥 ∶ 𝐹 → 𝑆 𝐵-cut: partition of 𝑊 into two sets 𝐵 and 𝑊\𝐵 𝜀𝐻 𝛣 = set of edges in 𝐻 crossing the 𝐵-cut. 𝜀𝐻 𝛣 =

𝑣, 𝑤 ∈ 𝐹 ∶ 𝑣 ∈ 𝐵, 𝑤 ∈ 𝑊\A

Size of 𝐵-cut in 𝐻: 𝜇𝐵 𝐻 = σ𝑓∈𝜀𝐻(𝛣) 𝑥(𝑓) 𝜇𝐵 𝐻 = 𝑥 𝑏, 𝑔 + 𝑥 𝑐, 𝑔 + 𝑥 𝑑, 𝑔 + 𝑥 𝑑, 𝑓 + 𝑥 𝑑, 𝑒 = 17 + 10 + 9 + 7 + 20 = 63 𝐵 𝑊\𝐵

Streaming Graph Algorithms

Cuts in Graphs 26 17 12 14 7 9 20 4 10 𝑏 𝑐 𝑑 𝑒 𝑔 𝑓 Weighted graph 𝐻 = (𝑊, 𝐹, 𝑥). Edge weights 𝑥 ∶ 𝐹 → 𝑆 𝐵-cut: partition of 𝑊 into two sets 𝐵 and 𝑊\𝐵 𝜀𝐻 𝛣 = set of edges in 𝐻 crossing the 𝐵-cut. 𝜀𝐻 𝛣 =

𝑣, 𝑤 ∈ 𝐹 ∶ 𝑣 ∈ 𝐵, 𝑤 ∈ 𝑊\A

Size of 𝐵-cut in 𝐻: 𝜇𝐵 𝐻 = σ𝑓∈𝜀𝐻(𝛣) 𝑥(𝑓) 𝜇𝐵 𝐻 = 𝑥 𝑐, 𝑑 + 𝑥 𝑔, 𝑑 + 𝑥 𝑔, 𝑓 = 12 + 9 + 14 = 35 𝐵 𝑊\𝐵

Streaming Graph Algorithms

Cut Sparsification Given a graph 𝐻 = (𝑊, 𝐹) we want to construct a weighted subgraph 𝐼 = (𝑊, 𝐹𝐼, 𝑥) of 𝐻 that estimates the size of each cut of 𝐻 (1 + 𝜁) cut sparsification

1 − 𝜁 ∙ 𝜇𝐵 𝐻 ≤ 𝜇𝐵 𝐼 ≤ (1 + 𝜁) ∙ 𝜇𝐵 𝐻

for all vertex subsets 𝐵 ⊂ 𝑊

Streaming Graph Algorithms

Graph Laplacian Weighted graph 𝐻 = (𝑊, 𝐹, 𝑥). Edge weights 𝑥 ∶ 𝐹 → 𝑆 Laplacian of 𝐻: 𝑜 × 𝑜 real matrix 𝑀𝐻, 𝑜 = |𝑊|

𝑀𝐻 𝑗, 𝑘 = ෍

𝑗,𝑙 ∈𝐹

𝑥 𝑗, 𝑙 , 𝑗 = 𝑘 −𝑥 𝑗, 𝑘 , 𝑗 ≠ 𝑘

where 𝑥(𝑗, 𝑘) = 0 if (𝑗, 𝑘) ∉ 𝐹

Streaming Graph Algorithms

Graph Laplacian 1 2 4 3 1 2 8 9 3 Weighted graph 𝐻 = (𝑊, 𝐹, 𝑥). Edge weights 𝑥 ∶ 𝐹 → 𝑆 Laplacian of 𝐻: 𝑜 × 𝑜 real matrix 𝑀𝐻, 𝑜 = |𝑊|

𝑀𝐻 𝑗, 𝑘 = ෍

𝑗,𝑙 ∈𝐹

𝑥 𝑗, 𝑙 , 𝑗 = 𝑘 −𝑥 𝑗, 𝑘 , 𝑗 ≠ 𝑘

where 𝑥(𝑗, 𝑘) = 0 if (𝑗, 𝑘) ∉ 𝐹 𝑀𝐻 = 14 −3 −9 −2 −3 4 −1 −9 −1 18 −8 −2 −8 10

Streaming Graph Algorithms

Graph Laplacian Weighted graph 𝐻 = (𝑊, 𝐹, 𝑥). Edge weights 𝑥 ∶ 𝐹 → 𝑆 Laplacian of 𝐻: 𝑜 × 𝑜 real matrix 𝑀𝐻, 𝑜 = |𝑊|

𝑀𝐻 𝑗, 𝑘 = ෍

𝑗,𝑙 ∈𝐹

𝑥 𝑗, 𝑙 , 𝑗 = 𝑘 −𝑥 𝑗, 𝑘 , 𝑗 ≠ 𝑘

where 𝑥(𝑗, 𝑘) = 0 if (𝑗, 𝑘) ∉ 𝐹 Let 𝑦 = 𝑦1 ⋮ 𝑦𝑜 be a real vector in ℝ𝑜. Recall that 𝑦𝑈 = 𝑦1 ⋯ 𝑦𝑜

Streaming Graph Algorithms

Graph Laplacian Weighted graph 𝐻 = (𝑊, 𝐹, 𝑥). Edge weights 𝑥 ∶ 𝐹 → 𝑆 Laplacian of 𝐻: 𝑜 × 𝑜 real matrix 𝑀𝐻, 𝑜 = |𝑊|

𝑀𝐻 𝑗, 𝑘 = ෍

𝑗,𝑙 ∈𝐹

𝑥 𝑗, 𝑙 , 𝑗 = 𝑘 −𝑥 𝑗, 𝑘 , 𝑗 ≠ 𝑘

where 𝑥(𝑗, 𝑘) = 0 if (𝑗, 𝑘) ∉ 𝐹 Let 𝑦 = 𝑦1 ⋮ 𝑦𝑜 be a real vector in ℝ𝑜. Recall that 𝑦𝑈 = 𝑦1 ⋯ 𝑦𝑜 Then

𝑦𝑈𝑀𝐻𝑦 = ෍

(𝑗,𝑘)∈𝐹

𝑥(𝑗, 𝑘) 𝑦𝑗 − 𝑦𝑘

2

Streaming Graph Algorithms

Spectral Sparsification Graph 𝐻 = (𝑊, 𝐹) A weighted subgraph 𝐼 = (𝑊, 𝐹𝐼, 𝑥) of 𝐻 is a (1 + 𝜁) spectral sparsifier of 𝐻 if

1 − 𝜁 ∙ 𝑦𝑈𝑀𝐻𝑦 ≤ 𝑦𝑈𝑀𝐼𝑦 ≤ (1 + 𝜁) ∙ 𝑦𝑈𝑀𝐻𝑦

for all real vectors 𝑦 ∈ ℝ𝑜

Streaming Graph Algorithms

Spectral Sparsification Graph 𝐻 = (𝑊, 𝐹) A weighted subgraph 𝐼 = (𝑊, 𝐹𝐼, 𝑥) of 𝐻 is a (1 + 𝜁) spectral sparsifier of 𝐻 if

1 − 𝜁 ∙ 𝑦𝑈𝑀𝐻𝑦 ≤ 𝑦𝑈𝑀𝐼𝑦 ≤ (1 + 𝜁) ∙ 𝑦𝑈𝑀𝐻𝑦

for all real vectors 𝑦 ∈ ℝ𝑜 A spectral sparsifier of 𝐻 can approximate:

• Size of all cuts
• Eigenvalues
• Effective resistances (in the corresponding electrical network)
• Properties of random walks

Streaming Graph Algorithms

Spectral Sparsification Graph 𝐻 = (𝑊, 𝐹) A weighted subgraph 𝐼 = (𝑊, 𝐹𝐼, 𝑥) of 𝐻 is a (1 + 𝜁) spectral sparsifier of 𝐻 if

1 − 𝜁 ∙ 𝑦𝑈𝑀𝐻𝑦 ≤ 𝑦𝑈𝑀𝐼𝑦 ≤ (1 + 𝜁) ∙ 𝑦𝑈𝑀𝐻𝑦

for all real vectors 𝑦 ∈ ℝ𝑜 Theorem [Spielman and Teng] A (1 + 𝜁) spectral sparsifier with O(𝑜 log 𝑜 /𝜁2) edges can be constructed in O(𝑛 polylog(𝑜)/𝜁2), where 𝑜 is the number of vertices and 𝑛 is the number of edges of the input graph.

Streaming Graph Algorithms

Spectral Sparsification Graph 𝐻 = (𝑊, 𝐹) A weighted subgraph 𝐼 = (𝑊, 𝐹𝐼, 𝑥) of 𝐻 is a (1 + 𝜁) spectral sparsifier of 𝐻 if

1 − 𝜁 ∙ 𝑦𝑈𝑀𝐻𝑦 ≤ 𝑦𝑈𝑀𝐼𝑦 ≤ (1 + 𝜁) ∙ 𝑦𝑈𝑀𝐻𝑦

for all real vectors 𝑦 ∈ ℝ𝑜 Theorem [Spielman and Teng] A (1 + 𝜁) spectral sparsifier with O(𝑜 log 𝑜 /𝜁2) edges can be constructed in O(𝑛 polylog(𝑜)/𝜁2), where 𝑜 is the number of vertices and 𝑛 is the number of edges of the input graph. Theorem [Batson, Spielman and Srivastava] A graph with 𝑜 vertices has a (1 + 𝜁) spectral sparsifier with O(𝑜/𝜁2) edges.

Streaming Graph Algorithms

Spectral Sparsification – Construction in the semi-streaming model

• Use as a black box any existing algorithm ALG that returns a (1 + 𝛿)

spectral sparsifier.

• ALG returns a spectral sparsifier with 𝑡𝑗𝑨𝑓(𝛿) = O(𝑜/𝛿2) number of edges.

Streaming Graph Algorithms

Spectral Sparsification – Construction in the semi-streaming model

• Use as a black box any existing algorithm ALG that returns a (1 + 𝛿)

spectral sparsifier.

• ALG returns a spectral sparsifier with 𝑡𝑗𝑨𝑓(𝛿) = O(𝑜/𝛿2) number of edges.

We use the following properties of spectral sparsification

• Mergeable: Suppose 𝐼1 and 𝐼2 are 𝛾 spectral sparsifiers of two graphs

𝐻1 and 𝐻2 on the same set of vertices. Then 𝐼1 ∪ 𝐼2 is a 𝛾 spectral sparsifier of 𝐻1 ∪ 𝐻2.

• Composable: If 𝐼3 is a 𝛾 spectral sparsifier for 𝐼2 and 𝐼2 is a 𝜀 spectral

sparsifier for 𝐼1 then 𝐼3 is a 𝛾𝜀 spectral sparsifier for 𝐼1.

Streaming Graph Algorithms

Spectral Sparsification – Construction in the semi-streaming model Let 𝐻 = (𝑊, 𝐹) be the input graph with 𝑜 = |𝑊| and 𝑛 = 𝐹 Data stream 𝑇 = the 𝑛 edges of 𝐻 Set 𝑢 = 𝑛/𝑡𝑗𝑨𝑓(𝛿). For simplicity assume that 𝑢 is a power of 2 We divide 𝑇 into 𝑢 segments of 𝑡𝑗𝑨𝑓(𝛿) edges 𝐻𝑗

0 = graph that consists of the edges in the 𝑗-th segment

… 𝑡𝑗𝑨𝑓(𝛿) 𝑡𝑗𝑨𝑓(𝛿) 𝑡𝑗𝑨𝑓(𝛿) 𝐻1 𝐻2 𝐻𝑢 𝑇

Streaming Graph Algorithms

Spectral Sparsification – Construction in the semi-streaming model Set 𝑢 = 𝑛/𝑡𝑗𝑨𝑓(𝛿). For simplicity assume that 𝑢 is a power of 2 (𝑢 = 2𝑙, 𝑙 = lg 𝑢) We divide 𝑇 into 𝑢 segments of 𝑡𝑗𝑨𝑓(𝛿) edges 𝐻𝑗

0 = graph that consists of the edges in the 𝑗-th segment

For 𝑗 = 1,2, … , lg 𝑢 and 𝑘 = 1,2, … , 𝑢/2𝑗 define 𝐻𝑗

𝑘 = 𝐻2𝑗−1 𝑘−1 ∪ 𝐻2𝑗 𝑘−1

E.g., for 𝑢 = 4 𝐻1

1 = 𝐻1 0 ∪ 𝐻2

𝐻2

1 = 𝐻3 0 ∪ 𝐻4

𝐻1 𝐻2 𝐻3 𝐻4 𝐻1

2 = 𝐻1 1 ∪ 𝐻2 1 = 𝐻

Streaming Graph Algorithms

Spectral Sparsification – Construction in the semi-streaming model Set 𝑢 = 𝑛/𝑡𝑗𝑨𝑓(𝛿). For simplicity assume that 𝑢 is a power of 2 (𝑢 = 2𝑙, 𝑙 = lg 𝑢) We divide 𝑇 into 𝑢 segments of 𝑡𝑗𝑨𝑓(𝛿) edges 𝐻𝑗

0 = graph that consists of the edges in the 𝑗-th segment

For 𝑗 = 1,2, … , lg 𝑢 and 𝑘 = 1,2, … , 𝑢/2𝑗 define 𝐻𝑗

𝑘 = 𝐻2𝑗−1 𝑘−1 ∪ 𝐻2𝑗 𝑘−1

For each 𝐻𝑗

𝑘 define a weighted subgraph 𝐼𝑗 𝑘 :

• 𝐼𝑗

0 = 𝐻𝑗

• 𝐼𝑗

𝑘 = ALG(𝐼2𝑗−1 𝑘−1 ∪ 𝐼2𝑗 𝑘−1), 𝑘 > 0

Streaming Graph Algorithms

Spectral Sparsification – Construction in the semi-streaming model Set 𝑢 = 𝑛/𝑡𝑗𝑨𝑓(𝛿). For simplicity assume that 𝑢 is a power of 2 (𝑢 = 2𝑙, 𝑙 = lg 𝑢) We divide 𝑇 into 𝑢 segments of 𝑡𝑗𝑨𝑓(𝛿) edges 𝐻𝑗

0 = graph that consists of the edges in the 𝑗-th segment

For 𝑗 = 1,2, … , lg 𝑢 and 𝑘 = 1,2, … , 𝑢/2𝑗 define 𝐻𝑗

𝑘 = 𝐻2𝑗−1 𝑘−1 ∪ 𝐻2𝑗 𝑘−1

For each 𝐻𝑗

𝑘 define a weighted subgraph 𝐼𝑗 𝑘 :

• 𝐼𝑗

0 = 𝐻𝑗

• 𝐼𝑗

𝑘 = ALG(𝐼2𝑗−1 𝑘−1 ∪ 𝐼2𝑗 𝑘−1), 𝑘 > 0

By the mergeable and composable properties 𝐼1

lg 𝑢 is a (1 + 𝛿)lg 𝑢 sparsifier of 𝐻

Streaming Graph Algorithms

Spectral Sparsification – Construction in the semi-streaming model By the mergeable and composable properties 𝐼1

lg 𝑢 is a (1 + 𝛿)lg 𝑢 sparsifier of 𝐻

Set 𝛿 = 𝜁/(2 lg 𝑢) ⇒ (1 + 𝛿)lg 𝑢 ~ (1 + 𝜁) Then 𝐼1

lg 𝑢 is a (1 + 𝜁) sparsifier of 𝐻

Streaming Graph Algorithms

Spectral Sparsification – Construction in the semi-streaming model By the mergeable and composable properties 𝐼1

lg 𝑢 is a (1 + 𝛿)lg 𝑢 sparsifier of 𝐻

Set 𝛿 = 𝜁/(2 lg 𝑢) ⇒ (1 + 𝛿)lg 𝑢 ~ (1 + 𝜁) Then 𝐼1

lg 𝑢 is a (1 + 𝜁) sparsifier of 𝐻

Space required 𝐼𝑗

𝑘 = ALG 𝐼2𝑗−1 𝑘−1 ∪ 𝐼2𝑗 𝑘−1

𝐼2𝑗−1

𝑘−1

𝐼2𝑗

𝑘−1

Delete 𝐼2𝑗−1

𝑘−1 and 𝐼2𝑗 𝑘−1

as soon as 𝐼𝑗

𝑘 is computed

⇒ For each 𝑘 we need to store 𝐼𝑗

𝑘

• nly for two values of 𝑗

Streaming Graph Algorithms

Spectral Sparsification – Construction in the semi-streaming model By the mergeable and composable properties 𝐼1

lg 𝑢 is a (1 + 𝛿)lg 𝑢 sparsifier of 𝐻

Set 𝛿 = 𝜁/(2 lg 𝑢) ⇒ (1 + 𝛿)lg 𝑢 ~ (1 + 𝜁) Then 𝐼1

lg 𝑢 is a (1 + 𝜁) sparsifier of 𝐻

Space required 𝐼𝑗

𝑘 = ALG 𝐼2𝑗−1 𝑘−1 ∪ 𝐼2𝑗 𝑘−1

𝐼2𝑗−1

𝑘−1

𝐼2𝑗

𝑘−1

Delete 𝐼2𝑗−1

𝑘−1 and 𝐼2𝑗 𝑘−1

as soon as 𝐼𝑗

𝑘 is computed

⇒ For each 𝑘 we need to store 𝐼𝑗

𝑘

• nly for two values of 𝑗

So at any given time we need to store ≤ 2 ∙ 𝑡𝑗𝑨𝑓 𝛿 ∙ lg 𝑢 = O(𝑜 lg3𝑜/𝜁2)

Streaming Graph Algorithms

Matchings Graph 𝐻 = (𝑊, 𝐹) Goal: Find a maximum cardinality matching 𝑁∗ Matching: Subset of edges 𝑁 ⊆ 𝐹 such that each vertex is adjacent to at most one edge in 𝑁

Streaming Graph Algorithms

Matchings Graph 𝐻 = (𝑊, 𝐹) Goal: Find a maximum cardinality matching 𝑁∗ Matching: Subset of edges 𝑁 ⊆ 𝐹 such that each vertex is adjacent to at most one edge in 𝑁

Streaming Graph Algorithms

Matchings Graph 𝐻 = (𝑊, 𝐹) Goal: Find a maximum cardinality matching 𝑁∗ Greedy Matching Algorithm 1. 𝑁 ← ∅ 2. for

• r each edge 𝑓 ∈ 𝑇 do

do 3. if if 𝑁 ∪ {𝑓} is a matching the then add 𝑓 to 𝑁 4. ret eturn 𝑁 Matching: Subset of edges 𝑁 ⊆ 𝐹 such that each vertex is adjacent to at most one edge in 𝑁

Streaming Graph Algorithms

Matchings Graph 𝐻 = (𝑊, 𝐹) Goal: Find a maximum cardinality matching 𝑁∗ The Greedy Matching Algorithm computes a matching 𝑁 with cardinality 𝑁 ≥ 𝑁∗ /2 Matching: Subset of edges 𝑁 ⊆ 𝐹 such that each vertex is adjacent to at most one edge in 𝑁

Streaming Graph Algorithms

Matchings Graph 𝐻 = (𝑊, 𝐹) Matching: Subset of edges 𝑁 ⊆ 𝐹 such that each vertex is adjacent to at most one edge in 𝑁 Goal: Find a maximum cardinality matching 𝑁∗ The Greedy Matching Algorithm computes a matching 𝑁 with cardinality 𝑁 ≥ 𝑁∗ /2 𝑣 𝑤 ∈ 𝑁∗ Consider an edge (𝑣, 𝑤) ∈ 𝑁∗ If (𝑣, 𝑤) ∉ 𝑁 then 𝑁 must contain at least one edge 𝑓 adjacent to 𝑣 or to 𝑤 𝑓 is adjacent to at most 2 edges of 𝑁∗ 𝑣′ 𝑤′ ∈ 𝑁∗ ∈ 𝑁 𝑓

Streaming Graph Algorithms

Weighted Matchings Weighted graph 𝐻 = (𝑊, 𝐹, 𝑥). Edge weights 𝑥 ∶ 𝐹 → ℝ+ (𝑥(𝑓) > 0, ∀𝑓 ∈ 𝐹) Goal: Find a maximum weight matching 𝑁∗ As before, we process the edges of the stream 𝑇 as they arrive and try to augment the current matching 𝑁

Streaming Graph Algorithms

Weighted Matchings Weighted graph 𝐻 = (𝑊, 𝐹, 𝑥). Edge weights 𝑥 ∶ 𝐹 → ℝ+ (𝑥(𝑓) > 0, ∀𝑓 ∈ 𝐹) Goal: Find a maximum weight matching 𝑁∗ As before, we process the edges of the stream 𝑇 as they arrive and try to augment the current matching 𝑁 Let 𝑓 be the next edge read from 𝑇. Let 𝐷 be the edges of 𝑁 that are in conflict with 𝑓 : and edge in 𝐷 and 𝑓 are adjacent to a common vertex.

Streaming Graph Algorithms

Weighted Matchings Weighted graph 𝐻 = (𝑊, 𝐹, 𝑥). Edge weights 𝑥 ∶ 𝐹 → ℝ+ (𝑥(𝑓) > 0, ∀𝑓 ∈ 𝐹) Goal: Find a maximum weight matching 𝑁∗ As before, we process the edges of the stream 𝑇 as they arrive and try to augment the current matching 𝑁 Let 𝑓 be the next edge read from 𝑇. Let 𝐷 be the edges of 𝑁 that are in conflict with 𝑓 : and edge in 𝐷 and 𝑓 are adjacent to a common vertex. ∈ 𝑁 ∈ 𝑁 𝑓 𝐷 has at most two edges. Let 𝑥(𝐷) be the total weight of the edges in 𝐷. If 𝑥(𝑓) > 𝑥(𝐷) then we increase the weight of 𝑁 by including 𝑓 and deleting the edges of 𝐷.

Streaming Graph Algorithms

Weighted Matchings Let 𝑓 be the next edge read from 𝑇. Let 𝐷 be the edges of 𝑁 that are in conflict with 𝑓 : and edge in 𝐷 and 𝑓 are adjacent to a common vertex. ∈ 𝑁 ∈ 𝑁 𝑓 If 𝑥(𝑓) > 𝑥(𝐷) then we increase the weight of 𝑁 by including 𝑓 and deleting the edges of 𝐷. 𝑥(𝐷) = total weight of the edges in 𝐷. Greedy Weighted Matching Algorithm 1. 𝑁 ← ∅ 2. for

• r each edge 𝑓 ∈ 𝑇 do

do 3. let 𝐷 be the set of edges that are in conflict with 𝑓 4. if if 𝑥 𝑓 > 𝑥(𝐷) th then add 𝑓 to 𝑁 and delete 𝐷 from 𝑁 5. ret eturn 𝑁

Streaming Graph Algorithms

Weighted Matchings Consider the following scenario 𝑇 = (1,2), (2,3), (3,4), … , (𝑜, 𝑜 − 1) Edge 𝑓𝑗 = (𝑗, 𝑗 + 1) has weight 𝑥(𝑓𝑗) = 1 + 𝑗𝜁, for a small 𝜁 > 0 1 + 𝜁

2 1 4 3 5 6

1 + 2𝜁 1 + 3𝜁 1 + 4𝜁 1 + 5𝜁 𝑁 = {}

Streaming Graph Algorithms

Weighted Matchings Consider the following scenario 𝑇 = (1,2), (2,3), (3,4), … , (𝑜, 𝑜 − 1) Edge 𝑓𝑗 = (𝑗, 𝑗 + 1) has weight 𝑥(𝑓𝑗) = 1 + 𝑗𝜁, for a small 𝜁 > 0 𝑁 = {(1,2)} 1 + 𝜁

2 1 4 3 5 6

1 + 2𝜁 1 + 3𝜁 1 + 4𝜁 1 + 5𝜁

Streaming Graph Algorithms

Weighted Matchings Consider the following scenario 𝑇 = (1,2), (2,3), (3,4), … , (𝑜, 𝑜 − 1) Edge 𝑓𝑗 = (𝑗, 𝑗 + 1) has weight 𝑥(𝑓𝑗) = 1 + 𝑗𝜁, for a small 𝜁 > 0 𝑁 = {(1,2)} 1 + 𝜁

2 1 4 3 5 6

1 + 2𝜁 1 + 3𝜁 1 + 4𝜁 1 + 5𝜁

Streaming Graph Algorithms

Weighted Matchings Consider the following scenario 𝑇 = (1,2), (2,3), (3,4), … , (𝑜, 𝑜 − 1) Edge 𝑓𝑗 = (𝑗, 𝑗 + 1) has weight 𝑥(𝑓𝑗) = 1 + 𝑗𝜁, for a small 𝜁 > 0 𝑁 = {(2,3)} 1 + 𝜁

2 1 4 3 5 6

1 + 2𝜁 1 + 3𝜁 1 + 4𝜁 1 + 5𝜁

Streaming Graph Algorithms

Weighted Matchings Consider the following scenario 𝑇 = (1,2), (2,3), (3,4), … , (𝑜, 𝑜 − 1) Edge 𝑓𝑗 = (𝑗, 𝑗 + 1) has weight 𝑥(𝑓𝑗) = 1 + 𝑗𝜁, for a small 𝜁 > 0 𝑁 = {(2,3)} 1 + 𝜁

2 1 4 3 5 6

1 + 2𝜁 1 + 3𝜁 1 + 4𝜁 1 + 5𝜁

Streaming Graph Algorithms

Weighted Matchings Consider the following scenario 𝑇 = (1,2), (2,3), (3,4), … , (𝑜, 𝑜 − 1) Edge 𝑓𝑗 = (𝑗, 𝑗 + 1) has weight 𝑥(𝑓𝑗) = 1 + 𝑗𝜁, for a small 𝜁 > 0 𝑁 = {(3,4)} 1 + 𝜁

2 1 4 3 5 6

1 + 2𝜁 1 + 3𝜁 1 + 4𝜁 1 + 5𝜁

Streaming Graph Algorithms

Weighted Matchings Consider the following scenario 𝑇 = (1,2), (2,3), (3,4), … , (𝑜, 𝑜 − 1) Edge 𝑓𝑗 = (𝑗, 𝑗 + 1) has weight 𝑥(𝑓𝑗) = 1 + 𝑗𝜁, for a small 𝜁 > 0 𝑁 = {(3,4)} 1 + 𝜁

2 1 4 3 5 6

1 + 2𝜁 1 + 3𝜁 1 + 4𝜁 1 + 5𝜁

Streaming Graph Algorithms

Weighted Matchings Consider the following scenario 𝑇 = (1,2), (2,3), (3,4), … , (𝑜, 𝑜 − 1) Edge 𝑓𝑗 = (𝑗, 𝑗 + 1) has weight 𝑥(𝑓𝑗) = 1 + 𝑗𝜁, for a small 𝜁 > 0 𝑁 = {(3,4)} 1 + 𝜁

2 1 4 3 5 6

1 + 2𝜁 1 + 3𝜁 1 + 4𝜁 1 + 5𝜁

Streaming Graph Algorithms

Weighted Matchings Consider the following scenario 𝑇 = (1,2), (2,3), (3,4), … , (𝑜, 𝑜 − 1) Edge 𝑓𝑗 = (𝑗, 𝑗 + 1) has weight 𝑥(𝑓𝑗) = 1 + 𝑗𝜁, for a small 𝜁 > 0 𝑁 = {(4,5)} 1 + 𝜁

2 1 4 3 5 6

1 + 2𝜁 1 + 3𝜁 1 + 4𝜁 1 + 5𝜁

Streaming Graph Algorithms

Weighted Matchings Consider the following scenario 𝑇 = (1,2), (2,3), (3,4), … , (𝑜, 𝑜 − 1) Edge 𝑓𝑗 = (𝑗, 𝑗 + 1) has weight 𝑥(𝑓𝑗) = 1 + 𝑗𝜁, for a small 𝜁 > 0 𝑁 = {(4,5)} 1 + 𝜁

2 1 4 3 5 6

1 + 2𝜁 1 + 3𝜁 1 + 4𝜁 1 + 5𝜁

Streaming Graph Algorithms

Weighted Matchings Consider the following scenario 𝑇 = (1,2), (2,3), (3,4), … , (𝑜, 𝑜 − 1) Edge 𝑓𝑗 = (𝑗, 𝑗 + 1) has weight 𝑥(𝑓𝑗) = 1 + 𝑗𝜁, for a small 𝜁 > 0 𝑁 = {(5,6)} 1 + 𝜁

2 1 4 3 5 6

1 + 2𝜁 1 + 3𝜁 1 + 4𝜁 1 + 5𝜁 𝑥(𝑁) = 1 + 5𝜁

Streaming Graph Algorithms

Weighted Matchings Consider the following scenario 𝑇 = (1,2), (2,3), (3,4), … , (𝑜, 𝑜 − 1) Edge 𝑓𝑗 = (𝑗, 𝑗 + 1) has weight 𝑥(𝑓𝑗) = 1 + 𝑗𝜁, for a small 𝜁 > 0 𝑁 = {(5,6)} 1 + 𝜁

2 1 4 3 5 6

1 + 2𝜁 1 + 3𝜁 1 + 4𝜁 1 + 5𝜁 𝑥(𝑁) = 1 + 5𝜁 𝑁∗ = { 1,2 , 3,4 , (5,6)} 1 + 𝜁

2 1 4 3 5 6

1 + 2𝜁 1 + 3𝜁 1 + 4𝜁 1 + 5𝜁 𝑥(𝑁∗) = 3 + 9𝜁

Streaming Graph Algorithms

Weighted Matchings Consider the following scenario 𝑇 = (1,2), (2,3), (3,4), … , (𝑜, 𝑜 − 1) Edge 𝑓𝑗 = (𝑗, 𝑗 + 1) has weight 𝑥(𝑓𝑗) = 1 + 𝑗𝜁, for a small 𝜁 > 0 The computed matching 𝑁 has weight 𝑥(𝑁) = 1 + (𝑜 − 1)𝜁 The optimal matching 𝑁 has weight 𝑥(𝑁∗) = σ𝑗 1 + 2𝑗 − 1 𝜁 > (𝑜 − 1)/2 Hence, the approximation ratio is 𝑥(𝑁∗) 𝑥(𝑁) > (𝑜 − 1)/2 1 + 𝑜 − 1 𝜁 ~ 𝑜 2

Streaming Graph Algorithms

Weighted Matchings The problem is that the trailing edges of 𝑇 that were once inserted into 𝑁 but removed later may have much larger total weight than the edges added later. 𝑁 = {(5,6)} 1 + 𝜁

2 1 4 3 5 6

1 + 2𝜁 1 + 3𝜁 1 + 4𝜁 1 + 5𝜁 𝑥(𝑁) = 1 + 5𝜁 trailing edges

Streaming Graph Algorithms

Weighted Matchings ∈ 𝑁 ∈ 𝑁 𝑓 We include 𝑓 in 𝑁 If 𝑥 𝑓 > 𝛾𝑥(𝐷) for some constant 𝛾 = 1 + 𝛿 > 1. Modified algorithm Greedy Weighted Matching Algorithm 1. 𝑁 ← ∅ 2. for

• r each edge 𝑓 ∈ 𝑇 do

do 3. let 𝐷 be the set of edges that are in conflict with 𝑓 4. if if 𝑥 𝑓 > (1 + 𝛿) ∙ 𝑥(𝐷) th then add 𝑓 to 𝑁 and delete 𝐷 from 𝑁 5. ret eturn 𝑁

Streaming Graph Algorithms

Weighted Matchings The problem is that the trailing edges of 𝑇 that were once inserted into 𝑁 but removed later may have much larger total weight than the edges added later. 𝑁 = {(5,6)} 1 + 𝜁

2 1 4 3 5 6

1 + 2𝜁 1 + 3𝜁 1 + 4𝜁 1 + 5𝜁 𝑥(𝑁) = 1 + 5𝜁 trailing edges We include 𝑓 in 𝑁 If 𝑥 𝑓 > 𝛾𝑥(𝐷) for some constant 𝛾 = 1 + 𝛿 > 1. For an edge 𝑓 define

• 𝐷0 = {𝑓}
• 𝐷𝑗 = edges removed when an edge in 𝐷𝑗−1 was added to 𝑁
• 𝑈

𝑓 = 𝐷1 ∪ 𝐷2 ∪ ⋯

Then 𝑥(𝑈

𝑓) ≤ 𝑥(𝑓)/𝛿

Streaming Graph Algorithms

Weighted Matchings It can be shown that 𝑥(𝑁∗) ≤ (1 + 𝛿) ∙ ෍

𝑓∈𝑁

(𝑥 𝑈

𝑓 + 2𝑥(𝑓))

By applying a careful charging scheme we get 𝑥(𝑁∗) 𝑥(𝑁) < 5.828

Streaming Graph Algorithms

Weighted Matchings Multi-pass Algorithm Greedy Weighted Matching Algorithm 𝑇 𝑁 = ∅ 𝑁 We can get a (2 + 𝜁)-approximation with O(𝜁−3) passes over 𝑇, where 𝛿 = O(𝜁)

Streaming Graph Algorithms

Graph Sketches Random linear projection 𝑁 ∶ ℝ𝑜 → ℝ𝑙, where 𝑙 ≪ 𝑜 𝑁 𝒘 = 𝑁𝒘 ∈ ℝ𝑜 ∈ ℝ𝑙 For any vector 𝒘 ∈ ℝ𝑜, the projection 𝑁𝒘 ∈ ℝ𝑙 preserves properties of 𝒘 with high probability Many applications: estimating entropy, heavy hitters, estimating norms, fitting polynomials,… Rich theory: dimensionality reduction, sparse recovery, metric embeddings,… ∈ ℝ𝑙×𝑜

Streaming Graph Algorithms

Graph Sketches Can we use this approach for graphs? That is, can we project the adjacency matrix 𝐵𝐻 of a graph 𝐻 to a smaller matrix 𝑁𝐵𝐻, so that we can use 𝑁𝐵𝐻 to compute properties of 𝐻?

• For a graph 𝐻 with n vertices, 𝐵𝐻 has O 𝑜2 dimensions.
• To work in the semi-streaming model we want 𝑁𝐵𝐻 to have O(𝑜 polylog(𝑜))

dimensions.

Streaming Graph Algorithms

Graph Sketches

Picture from https://people.cs.umass.edu/~mcgregor/711S12/lec-2-2.pdf

Streaming Graph Algorithms

Graph Sketches Dynamic graph stream 𝑇 = 𝑏1, 𝑏2, … where 𝑏𝑗 = (𝑓𝑗, Δ𝜅) 𝑓𝑗 = an edge of the graph Δ𝑗 = ቐ +1, 𝑓𝑗 is inserted −1, 𝑓𝑗 is deleted Multiplicity of edge 𝑓 :

𝑔

𝑓 = ෍ 𝑗∶ 𝑓𝑗=𝑓

Δ𝑗

For simplicity we will assume that 𝑔

𝑓 ∈ 0,1 , for all edges e.

Streaming Graph Algorithms

Graph Sketches Vector of edge multiplicities 𝒈 ∈ 0,1

𝑜 2

Each entry of 𝒈 is a multiplicity 𝑔

𝑓 of a (potential) edge 𝑓 of 𝐻 (a simple graph with

𝑜 vertices has up to 𝑜

2 edges).

1 2 3 𝑓12 𝑓13 𝑓23

𝒈 = 𝑔

𝑓12

𝑔

𝑓13

𝑔

𝑓23

= 1 1

Streaming Graph Algorithms

Graph Sketches Vector of edge multiplicities 𝒈 ∈ 0,1

𝑜 2

Each entry of 𝒈 is a multiplicity 𝑔

𝑓 of a (potential) edge 𝑓 of 𝐻 (a simple graph with

𝑜 vertices has up to 𝑜

2 edges).

Index vector of edge 𝑓 : 𝒋𝑓 ∈ 0,1

𝑜 2 . The only nonzero entry of 𝒋𝑓 is the one that

corresponds to edge 𝑓. 1 2 3 𝑓12 𝑓13 𝑓23

𝒋𝒇𝟑𝟒 = 𝑗𝑓12 𝑗𝑓13 𝑗𝑓23 = 1

Streaming Graph Algorithms

Graph Sketches Vector of edge multiplicities 𝒈 ∈ 0,1

𝑜 2

Each entry of 𝒈 is a multiplicity 𝑔

𝑓 of a (potential) edge 𝑓 of 𝐻 (a simple graph with

𝑜 vertices has up to 𝑜

2 edges).

Sketch of 𝒈 : 𝐵(𝒈) ∈ ℝ𝑒, 𝑒 =dimensionality of the sketch When we read the next item 𝑓, Δ from the stream, we can update the sketch as follows:

𝐵 𝒈 = 𝐵 𝒈 + Δ ∙ 𝐵(𝒋𝑓)

Index vector of edge 𝑓 : 𝒋𝑓 ∈ 0,1

𝑜 2 . The only nonzero entry of 𝒋𝑓 is the one that

corresponds to edge 𝑓.

Streaming Graph Algorithms

Homomorphic Sketches Vector of edge multiplicities 𝒈 ∈ 0,1

𝑜 2

Each entry of 𝒈 is a multiplicity 𝑔

𝑓 of a (potential) edge 𝑓 of 𝐻 (a simple graph with

𝑜 vertices has up to 𝑜

2 edges).

For a vertex 𝑤 let 𝒈𝒘 ∈ 0,1 𝑜−1 be the restriction of 𝒈 to the coordinates that involve 𝑤 (i.e., the 𝑜 − 1 edges that can be adjacent to 𝑤 in 𝐻) 1 2 3 𝑓12 𝑓13 𝑓23

𝒈 = 𝑔

𝑓12

𝑔

𝑓13

𝑔

𝑓23

= 1 1 𝒈𝟐 = 𝑔

𝑓12

𝑔

𝑓13

= 1

Streaming Graph Algorithms

Homomorphic Sketches Vector of edge multiplicities 𝒈 ∈ 0,1

𝑜 2

Each entry of 𝒈 is a multiplicity 𝑔

𝑓 of a (potential) edge 𝑓 of 𝐻 (a simple graph with

𝑜 vertices has up to 𝑜

2 edges).

For a vertex 𝑤 let 𝒈𝒘 ∈ 0,1 𝑜−1 be the restriction of 𝒈 to the coordinates that involve 𝑤 (i.e., the 𝑜 − 1 edges that can be adjacent to 𝑤 in 𝐻)

𝐵 𝒈 = 𝐵1 𝒈𝒘𝟐 ∘ 𝐵2 𝒈𝒘𝟑 ∘ ⋯ ∘ 𝐵𝑜 𝒈𝒘𝒐

The sketches of 𝒈 are formed by concatenation (∘) of the sketches of each 𝒈𝒘 Homomorphic sketches: For each operation on 𝐻 there is a corresponding

• peration on the sketches

Streaming Graph Algorithms

Connectivity via Sketches We wish to maintain a spanning forest of a graph 𝐻 = (𝑊, 𝐹) 1 2 3 4 6 7 5

Streaming Graph Algorithms

Connectivity via Sketches We wish to maintain a spanning forest of a graph 𝐻 = (𝑊, 𝐹) Connectivity Algorithm 1. rep epeat 2. for each vertex 𝑤 of the current graph do do 3. select an edge incident to 𝑤 4. contract all selected edges 5. un until the current graph has no edges Let’s begin with a simple (non-sketch) algorithm

Streaming Graph Algorithms

Connectivity via Sketches We wish to maintain a spanning forest of a graph 𝐻 = (𝑊, 𝐹) 1 2 3 4 6 7 5

Streaming Graph Algorithms

Connectivity via Sketches We wish to maintain a spanning forest of a graph 𝐻 = (𝑊, 𝐹) 1 2 3 4 6 7 5 12 34

567

Streaming Graph Algorithms

Connectivity via Sketches We wish to maintain a spanning forest of a graph 𝐻 = (𝑊, 𝐹) 1 2 3 4 6 7 5 12 34

567

super-vertices

Streaming Graph Algorithms

Connectivity via Sketches We wish to maintain a spanning forest of a graph 𝐻 = (𝑊, 𝐹) 1 2 3 4 6 7 5 12 34

567

12 34

567

Streaming Graph Algorithms

Connectivity via Sketches We wish to maintain a spanning forest of a graph 𝐻 = (𝑊, 𝐹) 1 2 3 4 6 7 5 12 34

567

12 34

567

new super-vertices

Streaming Graph Algorithms

Connectivity via Sketches We wish to maintain a spanning forest of a graph 𝐻 = (𝑊, 𝐹) Connectivity Algorithm 1. rep epeat 2. for each vertex 𝑤 of the current graph do do 3. select an edge incident to 𝑤 4. contract all selected edges 5. un until the current graph has no edges Let’s begin with a simple (non-sketch) algorithm Finds the connected components of 𝐻, and a spanning forest, in O(log 𝑜) rounds

Streaming Graph Algorithms

Connectivity via Sketches To design an algorithm that uses sketches we have to:

• 1. Define an appropriate graph representation
• 2. Apply ℓ0-sampling via linear sketches

Streaming Graph Algorithms

Connectivity via Sketches To design an algorithm that uses sketches we have to:

• 1. Define an appropriate graph representation
• 2. Apply ℓ𝟏-sampling via linear sketches

Let 𝐿 = polylog(𝑂). There is a distribution over matrices 𝑁 ∈ ℝ𝐿×𝑂 such that for any 𝒚 ∈ ℝ𝑂, a random non-zero element of 𝒚 can be reconstructed from 𝑁𝒚 with high probability. ℓ𝟏-sampling

Streaming Graph Algorithms

Connectivity via Sketches To design an algorithm that uses sketches we have to:

• 1. Define an appropriate graph representation

𝒃(𝑘,𝑙)

𝑗

= ቐ +1, −1, 0,

if 𝑗 = 𝑘 < 𝑙 and (𝑤𝑘, 𝑤𝑙) ∈ 𝐹 if 𝑘 < 𝑙 = 𝑗 and (𝑤𝑘, 𝑤𝑙) ∈ 𝐹

• therwise

For each vertex 𝑤𝑗 we define a vector 𝒃𝒋 ∈ −1,0,1

𝑜 2

with entries

Streaming Graph Algorithms

Connectivity via Sketches To design an algorithm that uses sketches we have to:

• 1. Define an appropriate graph representation

1 2 3 4 𝒃𝒋 = 𝒃 1,2

𝑗

𝒃 1,3

𝑗

𝒃(1,4)

𝑗

𝒃 2,3

𝑗

𝒃 2,4

𝑗

𝒃 3,4

𝑗 𝑈

Vector of vertex 𝑗 :

Streaming Graph Algorithms

Connectivity via Sketches To design an algorithm that uses sketches we have to:

• 1. Define an appropriate graph representation

1 2 3 4 𝒃𝒋 = 𝒃 1,2

𝑗

𝒃 1,3

𝑗

𝒃(1,4)

𝑗

𝒃 2,3

𝑗

𝒃 2,4

𝑗

𝒃 3,4

𝑗 𝑈

Vector of vertex 𝑗 : 𝒃𝟐 = 1 1 0 𝑈 𝒃𝟑 = −1 1 0 𝑈 𝒃𝟒 = 0 −1 −1 1 𝑈 𝒃𝟓 = 0 −1 𝑈

Streaming Graph Algorithms

Connectivity via Sketches To design an algorithm that uses sketches we have to:

• 1. Define an appropriate graph representation

1 2 3 4 𝒃𝒋 = 𝒃 1,2

𝑗

𝒃 1,3

𝑗

𝒃(1,4)

𝑗

𝒃 2,3

𝑗

𝒃 2,4

𝑗

𝒃 3,4

𝑗 𝑈

Vector of vertex 𝑗 : 𝒃𝟐 = 1 1 0 𝑈 𝒃𝟑 = −1 1 0 𝑈 𝒃𝟒 = 0 −1 −1 1 𝑈 𝒃𝟓 = 0 −1 𝑈 𝒃𝟐 + 𝒃𝟑 = 1 1 0 𝑈

Streaming Graph Algorithms

Connectivity via Sketches To design an algorithm that uses sketches we have to:

• 1. Define an appropriate graph representation

1 2 3 4 𝒃𝒋 = 𝒃 1,2

𝑗

𝒃 1,3

𝑗

𝒃(1,4)

𝑗

𝒃 2,3

𝑗

𝒃 2,4

𝑗

𝒃 3,4

𝑗 𝑈

Vector of vertex 𝑗 : 𝒃𝟐 = 1 1 0 𝑈 𝒃𝟑 = −1 1 0 𝑈 𝒃𝟒 = 0 −1 −1 1 𝑈 𝒃𝟓 = 0 −1 𝑈 𝒃𝟐 + 𝒃𝟑 = 𝟐 𝟐 0 𝑈

Streaming Graph Algorithms

Connectivity via Sketches To design an algorithm that uses sketches we have to:

• 1. Define an appropriate graph representation

𝒃(𝑘,𝑙)

𝑗

= ቐ +1, −1, 0,

if 𝑗 = 𝑘 < 𝑙 and (𝑤𝑘, 𝑤𝑙) ∈ 𝐹 if 𝑘 < 𝑙 = 𝑗 and (𝑤𝑘, 𝑤𝑙) ∈ 𝐹

• therwise

For each vertex 𝑤𝑗 we define a vector 𝒃𝒋 ∈ −1,0,1

𝑜 2

with entries For any subset of vertices 𝑉 ⊆ 𝑊, let 𝒃 𝑉 = ෍

𝑤𝑗∈𝑉

𝒃𝒋

Streaming Graph Algorithms

Connectivity via Sketches To design an algorithm that uses sketches we have to:

• 1. Define an appropriate graph representation

𝒃(𝑘,𝑙)

𝑗

= ቐ +1, −1, 0,

if 𝑗 = 𝑘 < 𝑙 and (𝑤𝑘, 𝑤𝑙) ∈ 𝐹 if 𝑘 < 𝑙 = 𝑗 and (𝑤𝑘, 𝑤𝑙) ∈ 𝐹

• therwise

For each vertex 𝑤𝑗 we define a vector 𝒃𝒋 ∈ −1,0,1

𝑜 2

with entries For any subset of vertices 𝑉 ⊆ 𝑊, let 𝒃 𝑉 = ෍

𝑤𝑗∈𝑉

𝒃𝒋

The non-zero entries of 𝒃(𝑉) correspond to 𝜀𝐻 𝑉 = the set of edges of 𝐻 that cross the cut (𝑉, 𝑊\U)

Streaming Graph Algorithms

Connectivity via Sketches To design an algorithm that uses sketches we have to:

• 1. Define an appropriate graph representation

For any subset of vertices 𝑉 ⊆ 𝑊, let 𝒃 𝑉 = ෍

𝑤𝑗∈𝑉

𝒃𝒋

The non-zero entries of 𝒃(𝑉) correspond to 𝜀𝐻 𝑉 = the set of edges of 𝐻 that cross the cut (𝑉, 𝑊\U)

Streaming Graph Algorithms

Connectivity via Sketches To design an algorithm that uses sketches we have to:

• 1. Define an appropriate graph representation

For any subset of vertices 𝑉 ⊆ 𝑊, let 𝒃 𝑉 = ෍

𝑤𝑗∈𝑉

𝒃𝒋

The non-zero entries of 𝒃(𝑉) correspond to 𝜀𝐻 𝑉 = the set of edges of 𝐻 that cross the cut (𝑉, 𝑊\U) Thus σ𝑤𝑗∈𝑉 𝑁𝒃𝒋 = 𝑁 σ𝑤𝑗∈𝑉 𝒃𝒋 gives a random edge in 𝜀𝐻 𝑉

Streaming Graph Algorithms

Connectivity via Sketches Connectivity via Sketches Algorithm I: Compute the Sketches in a Single Pass 1. Choose 𝑢 = O(log 𝑜) 2. for

• r 𝑗 = 1,2, … , 𝑜 and 𝑘 = 1,2, … , 𝑢 do

do 3. Construct the random projection 𝑁

𝑘𝒃𝑗

4. for

• r 𝑗 = 1,2, … , 𝑜 do

do 5. Compute 𝐵𝑗 𝒈𝒘𝒋 = (𝑁1𝒃𝑗) ∘ (𝑁2𝒃𝑗) ∘ ⋯ ∘ (𝑁𝑢𝒃𝑗)

Streaming Graph Algorithms

Connectivity via Sketches Connectivity via Sketches Algorithm I: Compute the Sketches in a Single Pass 1. Choose 𝑢 = O(log 𝑜) 2. for

• r 𝑗 = 1,2, … , 𝑜 and 𝑘 = 1,2, … , 𝑢 do

do 3. Construct the random projection 𝑁

𝑘𝒃𝑗

4. for

• r 𝑗 = 1,2, … , 𝑜 do

do 5. Compute 𝐵𝑗 𝒈𝒘𝒋 = (𝑁1𝒃𝑗) ∘ (𝑁2𝒃𝑗) ∘ ⋯ ∘ (𝑁𝑢𝒃𝑗)

• Each sketch 𝐵𝑗 has dimension O(polylog𝑜)
• Since there are n sketches, the required space is O(𝑜 polylog𝑜)

Streaming Graph Algorithms

Connectivity via Sketches Connectivity via Sketches Algorithm II: Emulate Connectivity Algorithm 1. Let ෠ 𝑊 = 𝑊 be the initial set of super-vertices 2. for

• r 𝑗 = 1,2, … , 𝑢 do

do 3. for

• r each super-vertex 𝑉 ∈ ෠

𝑊 do do 4. use σ𝑤𝑗∈𝑉 𝑁𝒃𝒋 to sample an edge between 𝑉 and another super-vertex 𝑋 5. collapse 𝑉 and 𝑋 to form a new super-vertex

Streaming Graph Algorithms

Connectivity via Sketches Connectivity via Sketches Algorithm II: Emulate Connectivity Algorithm 1. Let ෠ 𝑊 = 𝑊 be the initial set of super-vertices 2. for

• r 𝑗 = 1,2, … , 𝑢 do

do 3. for

• r each super-vertex 𝑉 ∈ ෠

𝑊 do do 4. use σ𝑤𝑗∈𝑉 𝑁𝒃𝒋 to sample an edge between 𝑉 and another super-vertex 𝑋 5. collapse 𝑉 and 𝑋 to form a new super-vertex The update time (to process the next edge in 𝑇) is O(polylog𝑜)

Streaming Graph Algorithms

Concluding remarks

• Many graph algorithms in the data stream model are known for

basic problems. E.g., estimating connectivity, approximating distances, finding approximate matchings, counting subgraphs,…

• But limited work on directed graphs!
• Space constraints: semi-stream model not suited for sparse

graphs (𝑛 = O(𝑜 polylog𝑜))

Streaming Architectures

Picture from https://databricks.com/blog/2015/07/30/diving-into-spark-streamings-execution-model.html

Streaming Architectures

Google Cloud Platform

https://cloud.google.com/solutions/architecture/streamprocessing